Method Comparisons
Compare estimands and decision rules before comparing numbers.
| Method family | Counterfactual | Typical uncertainty | Main diagnostic focus |
|---|---|---|---|
| GeoSC | SparseSC-weighted individual donors | In-space placebo effects and intervals | Joint pre-fit, donor weights, placebo support, simulation diagnostics |
| Google TBR/GBR and matched-market workflows | Time-based regression on aggregated treatment and control signals | Model-based intervals; design power depends on the implementation | Stable treatment-control relationship, match quality, model form, assignment constraints |
| Meridian GeoX | Design-dependent; the current overview names time-based regression and stratified sampling | Product-specific | Randomisation, balance, TBR stability, and delivery integrity |
| Meta GeoLift | Augmented synthetic control with generalised synthetic-control components | Conformal and package-specific procedures | L2 imbalance, augmentation model, market selection, test fit |
| Bayesian causal-impact tools | Structural time-series counterfactual | Posterior distribution | Prior/model fit and posterior predictive behaviour |
GeoSC’s native power module is simulation-based, but that does not imply a TBR or GeoX MDE is necessarily analytic; implementations vary. GeoSC repeatedly simulates a factor-VAR post period, injects effects, and calls SparseSC. A TBR workflow may use a fitted treatment-control regression, residual variance, analytic formula, resampling, or simulation.
Before comparing MDEs align outcome and transformation, treatment geography, control eligibility, pre-period, measurement and cooldown windows, lift denominator, effect pattern, alpha, target power, one- or two-sided rule, failure handling, and whether placebo assignments are sampled. Before comparing effects align the estimand, scale, aggregation, and uncertainty interpretation.
A material discrepancy is a diagnostic. Decompose it rather than averaging the answers or assuming the larger MDE is conservative. GeoSC is useful when individual donor diagnostics, Python operation, and auditable artefacts matter; it is not automatically superior to a well-specified regression design.
Primary method references
- Google Research: TBR matched-markets design and the archived GeoexperimentsResearch implementation.
- Google: Meridian GeoX overview, which names time-based regression and stratified sampling. Verify the chosen release and design before assigning it an uncertainty contract.
- Meta: GeoLift methodology and confidence-interval explanation.
These references describe their respective implementations. They do not imply that an internal or modified workflow uses the same defaults.