Synthetic Worlds

Synthetic data-generating processes with known causal ground truth, and the MFF exporter behind generate_synthetic_data.

Synthetic marketing worlds with known causal ground truth.

Originally built as the stress-test battery in tests/synth (which now re-exports from here), these data-generating processes produce realistic synthetic MMM data: confounded budgets, collinear channels, misspecified carryover/saturation, structural breaks, data-entry defects, and geo panels with heterogeneous effectiveness — each with a recorded causal answer key.

Entry points

dgp.SCENARIOS / dgp.build

National violation worlds ("realistic", "clean", "unobserved_confounding", …).

dgp_geo.SCENARIOS / dgp_geo.build

Geography and geography x product panel worlds.

generate_mff

Flatten any scenario to a Master Flat File dataset + JSON answer key.

class mmm_framework.synth.Scenario(name, violates, description, weeks, spend, y, controls, true_contribution, true_roas, representable=True, control_roles=None, notes=<factory>, mu=None, response_fn=None)[source]

Bases: object

A ready-to-fit world plus the ground truth used to grade the model.

name: str
violates: str
description: str
weeks: DatetimeIndex
spend: DataFrame
y: Series
controls: DataFrame
true_contribution: Series
true_roas: Series
representable: bool = True
control_roles: dict | None = None
notes: dict
mu: Series | None = None

Noiseless structural mean per period. y is this plus noise (then floored at 1.0). Grading a forecast interval against y conflates model error with irreducible noise, so an over-wide interval scores as well calibrated; grading against mu does not. Trailing + optional so every existing factory keeps byte-identical y / truth.

response_fn: Callable[[ndarray], ndarray] | None = None

The world’s structural response, spend (n, C) -> mean (n,). Kept so a window’s truth can be RECOMPUTED rather than truncated (see slice).

slice(start, end)[source]

The window [start, end) with its truth RECOMPUTED, not truncated.

This method exists to close a specific trap. true_contribution and true_roas are whole-window TOTALS; truncating them to a sub-window by any proportional rule is simply wrong, because the response is non-linear in spend and carries over across periods. Grading a rolling-origin forecast against a naively truncated total would produce a confident, fictional error.

Carryover semantics: the contribution is evaluated on the FULL spend history and then summed over the window, so spend before start still carries into it — the honest attribution for a forecast or variance window. At start == 0 this coincides exactly with recomputing on the sliced spend.

Requires response_fn (present on every world built by _finish).

Return type:

Scenario

property channels: list[str]
panel()[source]

Build a single-KPI PanelDataset ready for BayesianMMM.

__init__(name, violates, description, weeks, spend, y, controls, true_contribution, true_roas, representable=True, control_roles=None, notes=<factory>, mu=None, response_fn=None)
class mmm_framework.synth.GeoScenario(name, violates, description, weeks, geos, products, spend, y, controls, true_contribution, true_contribution_by_geo, true_roas_by_geo, representable=True, notes=<factory>)[source]

Bases: object

A ready-to-fit panel world plus per-geography causal ground truth.

name: str
violates: str
description: str
weeks: DatetimeIndex
geos: list[str]
products: list[str] | None
spend: DataFrame
y: Series
controls: DataFrame
true_contribution: Series
true_contribution_by_geo: DataFrame
true_roas_by_geo: DataFrame
representable: bool = True
notes: dict
property channels: list[str]
property cells: list[str]
property true_roas: Series
panel()[source]

Build the geo(/product) PanelDataset for BayesianMMM.

national_scenario()[source]

Aggregate the panel to one national series (the pre-geo-data view).

Spend and KPI sum across geographies; the price control averages. The causal truth is unchanged (zeroing a channel everywhere is the sum of the per-geo zero-outs), but the aggregated response is no longer in the model’s family: a sum of per-geo saturation curves is not a saturation curve of the summed spend (Jensen’s inequality), so the national fit carries a structural aggregation error the panel fit does not.

Return type:

Scenario

geo_scenario(geo)[source]

One geography as a standalone national-style scenario.

Used by the per-geo refit pivot: a single geo’s series with that geo’s own causal truth. Exactly representable for the clean and heterogeneous worlds alike (a per-geo fit estimates its own beta).

Return type:

Scenario

__init__(name, violates, description, weeks, geos, products, spend, y, controls, true_contribution, true_contribution_by_geo, true_roas_by_geo, representable=True, notes=<factory>)
mmm_framework.synth.build(name, seed=None, *, n_weeks=None)[source]

Build a scenario by name (uses each factory’s default seed if None).

Return type:

Scenario

mmm_framework.synth.generate_mff(scenario='realistic', *, seed=None, n_weeks=None, geographies=None)[source]

Generate a long-format MFF dataset from a named synthetic world.

Return type:

tuple[DataFrame, dict]

Parameters

scenariostr

A national scenario from dgp.SCENARIOS (e.g. "realistic", "clean", "unobserved_confounding") or a panel scenario from dgp_geo.SCENARIOS ("geo_clean", "geo_heterogeneous", "geo_product"). When geographies is given with a national name, the world is upgraded to a panel: "clean" maps to "geo_clean" and everything else to "geo_heterogeneous".

seedint, optional

Random seed (each factory’s default when omitted).

n_weeksint, optional

Series length in weeks (scenario default when omitted; minimum 52).

geographieslist of str, optional

Geography names for a panel world. Custom names get seeded baseline offsets, budget shares, and (heterogeneous world) effectiveness multipliers.

Returns

(DataFrame, dict)

The MFF-format data and the JSON-safe ground-truth answer key.

mmm_framework.synth.scenario_to_mff(sc)[source]

Flatten a national Scenario to MFF.

Return type:

DataFrame

mmm_framework.synth.geo_scenario_to_mff(sc)[source]

Flatten a panel GeoScenario to MFF.

Return type:

DataFrame

mmm_framework.synth.truth_summary(sc)[source]

JSON-safe answer key: scenario metadata + causal ground truth.

Return type:

dict

mmm_framework.synth.make_awareness_survey(*, n_weeks=104, n_trials=500, retention=0.8, intercept=-0.3, amplitude=1.2, seed=7)[source]

A synthetic brand-awareness survey dataset for the awareness model.

The KPI is a weekly aware-count n_aware ~ Binomial(n_trials, p_t) from a survey of n_trials people, where the aware-rate p_t = sigmoid(intercept + goodwill_t) and goodwill_t is a media goodwill stock that decays at retention ρ each week (Σ_c Σ_{τ≤t} ρ^(t-τ)·βc·sat(spend_τ,c)) — i.e. the exact generative structure the AwarenessStructuralMMM recovers. Two media channels (TV, Search). Returned as an MFF long-format dataframe (KPI Awareness = the count) + an answer key carrying the true ρ / half-life / n_trials / per-channel goodwill coefficients.

Fit it with likelihood={"family": "binomial"} and model_params={"number_of_trials": n_trials} — see the awareness model’s Atelier demo notebook.

Return type:

tuple[DataFrame, dict]

mmm_framework.synth.brand_funnel_mff(seed=21, *, n_weeks=156)[source]

MFF long table for the make_brand_funnel() world, INCLUDING its mediator surveys as extra variables — the format the StructuralNestedMMM fit path consumes.

Beyond the standard Sales/media/Price blocks, the table carries: :rtype: tuple[DataFrame, dict]

  • awareness_count / awareness_trials — the weekly binary tracker (rows only for observed weeks; a missing week = no survey, which the structural builder loads as NaN = unobserved)

  • consideration_cat_1 .. _5 — the weekly Likert category counts (low → high), again with unobserved weeks omitted

Returns (mff_df, answer_key); the key carries the structural truth (rho, betas, mediated shares) plus the survey variable names, so a recovery harness can assemble the DAG spec without magic strings.

Dgp

Structural-violation data-generating processes (DGPs) for the MMM.

Every scenario shares one clean core world (_base_world()) whose media response is built from the model’s exact structural family:

  • geometric carryover with normalized weights (parametric_adstock family),

  • concave saturation 1 - exp(-lambda * x_norm) (the model’s saturation),

  • additive channel contributions,

  • a Fourier-representable seasonality and a linear trend,

  • time-invariant positive coefficients,

  • homoscedastic Gaussian noise,

  • exogenous spend (independent of the latent demand / outcome).

make_clean() is the positive control: data drawn from precisely the model’s assumptions. On it, recovery error should be ~0 and every diagnostic green. Every other factory injects exactly one violation that real marketing data exhibits, holding the rest of the world fixed, so the harness can attribute any degradation to that single broken assumption.

Ground truth is defined the way the model itself reports an effect: the counterfactual zero-out of a channel’s spend, evaluated on the noiseless structural mean (_counterfactual_truth()). This makes “true contribution” exactly comparable to BayesianMMM.compute_counterfactual_contributions – truth and estimate are the same estimand on the same (KPI) scale. For confounded or endogenous worlds, truth is the causal media effect (zeroing spend does not change the demand-driven baseline), so the gap to the estimate is the bias.

A scenario sets representable=False when the truth lies outside the model’s hypothesis space (e.g. a genuinely negative effect under a positive-only prior). There, a large error is expected by construction and is reported as such.

class mmm_framework.synth.dgp.Scenario(name, violates, description, weeks, spend, y, controls, true_contribution, true_roas, representable=True, control_roles=None, notes=<factory>, mu=None, response_fn=None)[source]

Bases: object

A ready-to-fit world plus the ground truth used to grade the model.

name: str
violates: str
description: str
weeks: DatetimeIndex
spend: DataFrame
y: Series
controls: DataFrame
true_contribution: Series
true_roas: Series
representable: bool = True
control_roles: dict | None = None
notes: dict
mu: Series | None = None

Noiseless structural mean per period. y is this plus noise (then floored at 1.0). Grading a forecast interval against y conflates model error with irreducible noise, so an over-wide interval scores as well calibrated; grading against mu does not. Trailing + optional so every existing factory keeps byte-identical y / truth.

response_fn: Callable[[ndarray], ndarray] | None = None

The world’s structural response, spend (n, C) -> mean (n,). Kept so a window’s truth can be RECOMPUTED rather than truncated (see slice).

slice(start, end)[source]

The window [start, end) with its truth RECOMPUTED, not truncated.

This method exists to close a specific trap. true_contribution and true_roas are whole-window TOTALS; truncating them to a sub-window by any proportional rule is simply wrong, because the response is non-linear in spend and carries over across periods. Grading a rolling-origin forecast against a naively truncated total would produce a confident, fictional error.

Carryover semantics: the contribution is evaluated on the FULL spend history and then summed over the window, so spend before start still carries into it — the honest attribution for a forecast or variance window. At start == 0 this coincides exactly with recomputing on the sliced spend.

Requires response_fn (present on every world built by _finish).

Return type:

Scenario

property channels: list[str]
panel()[source]

Build a single-KPI PanelDataset ready for BayesianMMM.

__init__(name, violates, description, weeks, spend, y, controls, true_contribution, true_roas, representable=True, control_roles=None, notes=<factory>, mu=None, response_fn=None)
mmm_framework.synth.dgp.build(name, seed=None, *, n_weeks=None)[source]

Build a scenario by name (uses each factory’s default seed if None).

Return type:

Scenario

Dgp Geo

Multi-geography and geography x product synthetic worlds for the MMM.

Companion to mmm_framework.synth.dgp (national worlds). Every world here is a balanced panel: n_periods x n_geos (x n_products) observations stacked period-major, exactly the layout the MFF loader produces. Ground truth follows the same doctrine as dgp: per-channel truth is the counterfactual zero-out evaluated on the noiseless structural mean — the model’s own estimand — and is recorded per geography (and per cell for geo x product worlds), so panel fits can be graded at the level stakeholders actually read: regional ROI.

The model’s hierarchy (model/base.py) gives each geography/product an additive intercept offset (geo_sigma * geo_offset) while every response parameter — beta, saturation, adstock — is global. Two worlds bracket that hypothesis space:

  • make_geo_clean() — geo differences are level shifts only: exactly the model’s family (the panel positive control).

  • make_geo_heterogeneous() — channel effectiveness differs by geo and budgets chase performance, the way real regional allocation works. One pooled beta cannot represent this; per-geo readouts fail silently while the national total stays plausible.

  • make_geo_product() — a geo x product positive control (level shifts per geo and per product, shared response, product-tilted channel mixes).

class mmm_framework.synth.dgp_geo.GeoScenario(name, violates, description, weeks, geos, products, spend, y, controls, true_contribution, true_contribution_by_geo, true_roas_by_geo, representable=True, notes=<factory>)[source]

Bases: object

A ready-to-fit panel world plus per-geography causal ground truth.

name: str
violates: str
description: str
weeks: DatetimeIndex
geos: list[str]
products: list[str] | None
spend: DataFrame
y: Series
controls: DataFrame
true_contribution: Series
true_contribution_by_geo: DataFrame
true_roas_by_geo: DataFrame
representable: bool = True
notes: dict
property channels: list[str]
property cells: list[str]
property true_roas: Series
panel()[source]

Build the geo(/product) PanelDataset for BayesianMMM.

national_scenario()[source]

Aggregate the panel to one national series (the pre-geo-data view).

Spend and KPI sum across geographies; the price control averages. The causal truth is unchanged (zeroing a channel everywhere is the sum of the per-geo zero-outs), but the aggregated response is no longer in the model’s family: a sum of per-geo saturation curves is not a saturation curve of the summed spend (Jensen’s inequality), so the national fit carries a structural aggregation error the panel fit does not.

Return type:

Scenario

geo_scenario(geo)[source]

One geography as a standalone national-style scenario.

Used by the per-geo refit pivot: a single geo’s series with that geo’s own causal truth. Exactly representable for the clean and heterogeneous worlds alike (a per-geo fit estimates its own beta).

Return type:

Scenario

__init__(name, violates, description, weeks, geos, products, spend, y, controls, true_contribution, true_contribution_by_geo, true_roas_by_geo, representable=True, notes=<factory>)
mmm_framework.synth.dgp_geo.build(name, seed=None, *, geos=None, n_weeks=None)[source]

Build a geo scenario by name (factory default seed when None).

Return type:

GeoScenario

Mff

Convert synthetic DGP scenarios to Master Flat File (MFF) datasets.

The scenario factories in mmm_framework.synth.dgp (national worlds) and mmm_framework.synth.dgp_geo (geo / geo x product panels) return wide, model-ready frames plus causal ground truth. This module flattens a scenario into the 8-column long MFF layout the data loader and the app ingest (Period, Geography, Product, Campaign, Outlet, Creative, VariableName, VariableValue) and emits a JSON-safe “answer key” so a fitted model can be graded against the world’s known causal truth.

mmm_framework.synth.mff.MIN_WEEKS = 52

Minimum series length the violation worlds support (seasonality cycles, mid-series breaks, and seeded error placement all assume at least a year).

mmm_framework.synth.mff.scenario_to_mff(sc)[source]

Flatten a national Scenario to MFF.

Return type:

DataFrame

mmm_framework.synth.mff.geo_scenario_to_mff(sc)[source]

Flatten a panel GeoScenario to MFF.

Return type:

DataFrame

mmm_framework.synth.mff.truth_summary(sc)[source]

JSON-safe answer key: scenario metadata + causal ground truth.

Return type:

dict

mmm_framework.synth.mff.generate_mff(scenario='realistic', *, seed=None, n_weeks=None, geographies=None)[source]

Generate a long-format MFF dataset from a named synthetic world.

Return type:

tuple[DataFrame, dict]

Parameters

scenariostr

A national scenario from dgp.SCENARIOS (e.g. "realistic", "clean", "unobserved_confounding") or a panel scenario from dgp_geo.SCENARIOS ("geo_clean", "geo_heterogeneous", "geo_product"). When geographies is given with a national name, the world is upgraded to a panel: "clean" maps to "geo_clean" and everything else to "geo_heterogeneous".

seedint, optional

Random seed (each factory’s default when omitted).

n_weeksint, optional

Series length in weeks (scenario default when omitted; minimum 52).

geographieslist of str, optional

Geography names for a panel world. Custom names get seeded baseline offsets, budget shares, and (heterogeneous world) effectiveness multipliers.

Returns

(DataFrame, dict)

The MFF-format data and the JSON-safe ground-truth answer key.

mmm_framework.synth.mff.make_awareness_survey(*, n_weeks=104, n_trials=500, retention=0.8, intercept=-0.3, amplitude=1.2, seed=7)[source]

A synthetic brand-awareness survey dataset for the awareness model.

The KPI is a weekly aware-count n_aware ~ Binomial(n_trials, p_t) from a survey of n_trials people, where the aware-rate p_t = sigmoid(intercept + goodwill_t) and goodwill_t is a media goodwill stock that decays at retention ρ each week (Σ_c Σ_{τ≤t} ρ^(t-τ)·βc·sat(spend_τ,c)) — i.e. the exact generative structure the AwarenessStructuralMMM recovers. Two media channels (TV, Search). Returned as an MFF long-format dataframe (KPI Awareness = the count) + an answer key carrying the true ρ / half-life / n_trials / per-channel goodwill coefficients.

Fit it with likelihood={"family": "binomial"} and model_params={"number_of_trials": n_trials} — see the awareness model’s Atelier demo notebook.

Return type:

tuple[DataFrame, dict]