Keeps this page in sync as the body changes. Pause it any time for a quieter view.
Path /specs/prompt-ab-roi-measurement
Last refresh never
Prompt A/B ROI Measurement
inventory_source local | spec_api /api/spec-registry/prompt-ab-roi-measurement | registry_updated 2026-04-09T03:10:09.001845Z
potential_value 0.00 | actual_value 1.00 | value_gap 0.00
estimated_cost 0.00 | actual_cost 1.00 | cost_gap 1.00
estimated_roi 0.00 | actual_roi 1.00
Missing contributor linkage. Submit a change request with contributor attribution.
Open process view for this spec
task_ids -
branches -
source_files specs/cross-task-outcome-correlation.md, specs/prompt-ab-roi-measurement.md, specs/provider-health-alerting.md, specs/provider-usage-coalescing-timeout-resilience.md, specs/runner-auto-contribution.md, specs/tool-failure-awareness.md
evidence_refs -
Open implementation view for this spec
implementation_refs spec-registry:cross-task-outcome-correlation, spec-registry:prompt-ab-roi-measurement, spec-registry:provider-health-alerting, spec-registry:provider-usage-coalescing-timeout-resilience, spec-registry:runner-auto-contribution, spec-registry:tool-failure-awareness
lineage_ids -
public_endpoints -
summary **Idea**: `agent-prompt-ab-roi` (sub-idea of `coherence-network-agent-pipeline`)
process_summary Outcome recording — persist measurement record per task completion; ROI computation — sum(value_score)/sum(resource_cost) per variant with stats; Variant selection — Thompson Sampling with exploration priority for < 5 samples, block after 3 zeros; Stats endpoint — GET /api/agent/prompt-ab/stats returns per-variant ROI and selection probability; Integration hook — record_prompt_outcome() callable from task execution flow
pseudocode_summary -
implementation_summary api/app/services/prompt_ab_roi_service.py (record_prompt_outcome(), select_variant(), get_variant_stats()); api/app/services/slot_selection_service.py (SlotSelector); api/app/routers/agent_prompt_ab_routes.py (A/B testing endpoints)