Method snapshot: synthetic marketplace
Choose the assignment before interpreting the effect.
A frozen simulation study from a different project, shown for its method. These are not enterprise results.
The frozen study contains 6 scenarios, 500 replications per scenario and 12,000 method-result rows. Driver A/B, its CUPED analysis, region clustering and region/time switchbacks estimate different contrasts when spillovers or carryover exist.
Policy gap is mean estimate minus the paired all-treated versus all-control policy truth. Under interference, that gap can include an estimand mismatch. Rejection means false positives in null scenarios and power in positive-effect scenarios. Coverage here concerns the simulation's policy truth, not a guarantee for the randomized contrast or a real marketplace.
| Scenario | Method | Policy gap, rides | Coverage | Reject zero |
|---|---|---|---|---|
| carryover effect | driver ab | −0.6 | 95.4% | 37.4% |
| carryover effect | driver ab cuped | −1.8 | 94.2% | 91.2% |
| carryover effect | region cluster | −6.7 | 95.6% | 25.4% |
| carryover effect | switchback | −259.8 | 44.8% | 54.6% |
| carryover null | driver ab | −0.7 | 95.2% | 4.8% |
| carryover null | driver ab cuped | −1.8 | 94.4% | 5.6% |
| carryover null | region cluster | −8.6 | 95.4% | 4.6% |
| carryover null | switchback | 7.5 | 95.8% | 4.2% |
| none effect | driver ab | −1.0 | 95.2% | 43.6% |
| none effect | driver ab cuped | −2.2 | 94.0% | 94.0% |
| none effect | region cluster | −7.1 | 95.8% | 28.2% |
| none effect | switchback | 7.2 | 96.6% | 99.4% |
| none null | driver ab | −0.7 | 95.2% | 4.8% |
| none null | driver ab cuped | −1.8 | 94.4% | 5.6% |
| none null | region cluster | −8.6 | 95.4% | 4.6% |
| none null | switchback | 7.5 | 95.8% | 4.2% |
| spillover effect | driver ab | 235.2 | 90.8% | 68.0% |
| spillover effect | driver ab cuped | 234.0 | 76.6% | 99.4% |
| spillover effect | region cluster | 126.8 | 94.8% | 50.4% |
| spillover effect | switchback | 128.4 | 85.6% | 99.6% |
| spillover null | driver ab | 5.5 | 95.0% | 5.0% |
| spillover null | driver ab cuped | 4.3 | 94.6% | 5.4% |
| spillover null | region cluster | −11.7 | 95.8% | 4.2% |
| spillover null | switchback | 2.2 | 95.2% | 4.8% |
Transferable method: map spillovers, define the target contrast and test the chosen assignment under plausible contamination. CUPED can reduce variance; it cannot repair interference. Account networks require their own simulation.
No numerical transfer: simulated ride counts, variance and coverage do not supply enterprise activation rates, account ICC or renewal power.
Method and checks
The model omits demand response, pricing, retention and participant entry or exit. Carryover has a tail; a short washout does not erase it. No method is a universal winner. The snapshot is descriptive and does not rerun the underlying simulation; the original run used seed 7.
All summary fields and Monte Carlo intervals · Source hashes and projected fields · Separate enterprise assumptions