Method snapshot: public mobility study
Score useful answers and clarification separately.
Observed portfolio results from a different study, shown for its method. They are not enterprise outcomes.
A fixed question set about public Chicago mobility data compared Raw SQL with a governed metric query bundle. Analytical accuracy was 45/55 versus 46/55. Clarification checks passed in 3/5 versus 4/5 cases.
The paired analytical difference was 1.8 percentage points. Its question-bootstrap interval spans −12.7 to 16.4 percentage points. Related, purposively selected questions do not establish population generalization. One retained response per question and arm does not measure variation across model reruns.
| Tier | Raw SQL | Governed metric query |
|---|---|---|
| Tier 1 | 19/20 | 13/20 |
| Tier 2 | 17/20 | 19/20 |
| Tier 3 | 9/15 | 14/15 |
The governed bundle includes metadata and a constrained interface. This comparison does not isolate a metadata-only effect. The automated clarification rubric needs separate judgment of usefulness. Scoring precision affects the comparison; this snapshot retains the original primary scores.
Transferable method: freeze questions, gold answers and scoring rules; retain errors; audit analytical accuracy separately from appropriate clarification. Enterprise task rubrics need their own representative data and permission checks.
No numerical transfer: none of these rates enters the activation power calculation. They describe a different population, task and unit of analysis.
Method and checks
This is a static summary, not the full answer replay. Source results and release hashes identify the original reviewed artifact.
Frozen result projection · Source hashes and projected fields · City attribution and data notice