Skip to content

Testing

Philosophy

  • Code does a lot of data analysis/transformation, which is tricky to test and generate data for.
  • We use faker, factoryboy, and pytest-factoryboy to easily generate test data. The combination allows us to generate fake data for fields our models need, but our tests don't use.
  • For pieces of data that our tested functions will use, we should always set them to static values.
  • This way we reduce test flakiness (passing in one run, but failing the next because of random data).

Running Tests

# Run all tests (parallelized across 4 workers)
just test

# Run a specific test
just test path/to/test_file.py::test_function_name

# Run tests matching a name pattern
just test -k "test_name"

# Run tests with coverage report
just cov

# Run all checks (lint + tests + coverage)
just check-all

IDE Shortcuts (VSCode / Cursor)

Make sure the Python extension is installed and the Testing tab discovers tests correctly.

  • CMD + ; then C — run test at cursor
  • CMD + ; then CMD + C — debug test at cursor
  • CMD + ; then F — run tests in current file
  • CMD + ; then A — run all tests

Dependency Injection with Monkeypatch

The rule: Always use monkeypatch.setattr(). Never use unittest.mock.patch, MagicMock, or @patch decorators.

Why monkeypatch over patch/MagicMock

MagicMock returns a truthy mock object for any attribute access, which hides real bugs. If your code accidentally calls mock.some_method_that_doesnt_exist(), MagicMock silently returns another mock. monkeypatch.setattr forces you to provide an explicit replacement — if your code calls something you didn't mock, it hits the real implementation and fails loudly.

Canonical example

Testing a method that calls an integration:

def test_sync_recipients_calls_klaviyo(monkeypatch):
    """Test that recipient sync extracts profiles from Klaviyo segment."""
    expected_profiles = [{"email": "test@example.com", "first_name": "Test"}]

    # Mock the integration call — use the full module path where it's imported
    monkeypatch.setattr(
        "app.methods.campaigns.recipient_sync.extract_integration_recipients",
        lambda campaign, integration: expected_profiles,
    )

    # Mock dependent functions
    monkeypatch.setattr(
        "app.methods.campaigns.recipient_sync.get_mail_integration",
        lambda organization_id: IntegrationFactory(obj__organization_id=organization_id),
    )
    monkeypatch.setattr(
        "app.methods.campaigns.recipient_sync.process_automation_recipients",
        lambda incoming_recipients, campaign, organization: None,
    )

    campaign = CampaignFactory(obj__status=CampaignStatus.Active)
    sync_automation_recipients(campaign=campaign, organization=campaign.organization_id)

    # For more specific assertions, use a local function to capture call args:
    call_args = {}

    def capture_process(incoming_recipients, campaign, organization):
        call_args["recipients"] = incoming_recipients

    monkeypatch.setattr(
        "app.methods.campaigns.recipient_sync.process_automation_recipients",
        capture_process,
    )

    sync_automation_recipients(campaign=campaign, organization=campaign.organization_id)
    assert call_args["recipients"] == expected_profiles

Common patterns

Monkeypatching a model function: 

monkeypatch.setattr(
    "app.core.models.campaigns.get_campaign",
    lambda campaign_id: CampaignFactory(obj__campaign_id=campaign_id),
)

Monkeypatching an integration client: 

monkeypatch.setattr(
    "app.core.integrations.klaviyo.client.get_segment_profiles",
    lambda segment_id, **kwargs: [{"email": "test@example.com"}],
)

Monkeypatching a utility: 

monkeypatch.setattr(
    "app.core.utils.dates.get_current_timestamp",
    lambda: datetime(2026, 1, 15, tzinfo=timezone.utc),
)

Using a local function for complex mocks: 

def mock_create_mailer(recipient, front, back, size, **kwargs):
    return {"id": "mailer_123", "status": "created"}

monkeypatch.setattr(
    "app.core.integrations.printers.lob_client.generate_mailer",
    mock_create_mailer,
)

DO NOT do this

# WRONG — never use unittest.mock
from unittest.mock import patch, MagicMock

@patch("app.methods.klaviyo.sync_segment")
def test_bad_example(mock_sync):
    mock_sync.return_value = MagicMock()  # Hides bugs
    # ...

# WRONG — never use MagicMock
mock_service = MagicMock(spec=Integration)
mock_service.get_profiles.return_value = [...]  # Fragile, hard to trace

Tenacity and Retrying

  • We use tenacity to retry operations that can fail due to temporary issues (e.g. network errors, rate limits).
  • This is especially useful for operations that depend on external APIs or services.
  • In testing, we have disabled tenacity retries by default to avoid long test runs and flakiness. See app/tests/conftest.py for details.

Factories

  • See existing factory implementation for each model in app/tests/factories/.
  • Where the model takes in an obj: dict for initialization, that makes factories a bit more verbose because each model needs two factories: one for the model itself (taking obj as an argument) and one for the obj dictionary. Example: app/tests/factories/orders.py (see OrderEventFactory and OrderEventObjectFactory).
  • In factories, always try to restrict random values to a reasonable set of values that reflects our business logic (e.g. use int instead of uuid, and set statuses to one of the possible choices from our pre-defined enums, etc).

FactoryBoy vs. pytest-factoryboy

Use FactoryBoy directly when you need multiple instances with different values in the same test:

all_recipients = [
    ReportingRecipientFactory(
        status=CampaignRecipientStatus.Sent,
        campaign_id=campaign_id,
    ),
    ReportingRecipientFactory(
        status=CampaignRecipientStatus.Holdout,
        campaign_id=campaign_id,
    ),
]

Use pytest-factoryboy fixtures when you need a single default instance as a test fixture:

@pytest.mark.parametrize(
    "order_json__shipping_address",
    [{"address1": "bobsmith", "address2": "flat2", "zip": "3456"}],
)
def test_order_address(order_event):
    assert order_event.get_address_str() == "bobsmithflat23456"

Subfactory values

Pass values to nested factories using the double-underscore syntax:

config = OrganizationConfigurationObjectFactory(
    currency_code="USD",
    excluded_campaigns=[],
    enabled=True,
    excluded_attributions=[],
    test_mode=False,
)
org = OrganizationFactory(obj__id=organization_id, obj__configuration=config)

Registering pytest-factoryboy fixtures

Register factories in an appropriate conftest.py that runs before your test — same directory as the test file or any parent up to app/tests/:

from pytest_factoryboy import register

from app.tests.factories.orders import OrderEventFactory, OrderEventRowFactory

register(OrderEventFactory)
register(OrderEventRowFactory)

Creating a new factory

  1. Check app/tests/factories/ for an existing factory for your model
  2. If the model uses __init__(obj: dict), you need two factories:
  3. An "object" factory for the dict (e.g., CampaignObjectFactory)
  4. A model factory that wraps it (e.g., CampaignFactory)
  5. Use factory.LazyAttribute for computed fields
  6. Restrict random values to business-valid sets using enums
  7. Register in the nearest conftest.py if using pytest-factoryboy

DB and Redis in Tests

Tests do NOT hit a real database. Database calls are mocked via monkeypatch.setattr on the model functions (e.g., monkeypatch.setattr("app.core.models.campaigns.get_campaign", ...)).

Redis calls are routed to an in-memory fakeredis.FakeStrictRedis by the autouse fake_redis fixture in app/tests/conftest.py. The fixture patches RedisClient.get_conn, so any code path using RedisClient.get_value(), set_str(), get_dict(), get_client(), bloom filters, pipelines, etc. uses the same per-test fake Redis instance and never opens a real Redis connection.

Most tests do not need to mention Redis at all. When a test needs to seed or inspect Redis state, either use the real RedisClient helper methods or request the fake_redis fixture directly:

from app.core.db.redis_client import RedisClient


def test_uses_cached_value(fake_redis):
    RedisClient.set_str("klaviyo:account_industry:42", "Health & Beauty")

    assert RedisClient.get_value("klaviyo:account_industry:42") == "Health & Beauty"
    assert fake_redis.exists("klaviyo:account_industry:42") == 1

Only monkeypatch Redis methods directly when the behavior under test is a Redis failure or a call that should not happen. Do not create a local MockRedisClient or import a shared Redis mock class for normal cache tests.

Common Gotchas

Fixture not found error: You forgot to register the factory in conftest.py. Add register(YourFactory) in the nearest conftest.

Monkeypatch doesn't seem to work (original function still runs): You're patching the wrong import path. Patch where the function is used, not where it's defined. If app/methods/runner.py imports from app.core.models.campaigns import get_campaign, patch app.methods.runner.get_campaign, not app.core.models.campaigns.get_campaign.

Flaky tests (pass sometimes, fail sometimes): Factory random values are leaking into your assertions. Always set tested fields to static values:

# WRONG — random campaign_id could collide
recipient = CampaignRecipientFactory()
assert recipient.campaign_id == ???

# RIGHT — static value for the field under test
recipient = CampaignRecipientFactory(campaign_id=123)
assert recipient.campaign_id == 123