Keeps this page in sync as the body changes. Pause it any time for a quieter view.
Path /specs/pipeline-observability-and-auto-review
Last refresh never
... 13 more in Requirements section below
inventory_source local | spec_api /api/spec-registry/pipeline-observability-and-auto-review | registry_updated 2026-04-09T03:10:08.984193Z
potential_value 0.00 | actual_value 1.00 | value_gap 0.00
estimated_cost 0.00 | actual_cost 1.00 | cost_gap 1.00
estimated_roi 0.00 | actual_roi 1.00
Missing contributor linkage. Submit a change request with contributor attribution.
Open process view for this spec
task_ids -
branches -
source_files specs/agent-orchestration-api.md, specs/attention-heuristics-pipeline-status.md, specs/coherence-network-agent-pipeline.md, specs/pipeline-observability-and-auto-review.md, specs/project-manager-pipeline.md, specs/split-review-deploy-verify-phases.md
evidence_refs -
Open implementation view for this spec
implementation_refs spec-registry:agent-orchestration-api, spec-registry:attention-heuristics-pipeline-status, spec-registry:coherence-network-agent-pipeline, spec-registry:pipeline-observability-and-auto-review, spec-registry:project-manager-pipeline, spec-registry:split-review-deploy-verify-phases
lineage_ids -
public_endpoints -
summary Enable measurement, optimization, and self-improvement of the agent pipeline. Capture execution metrics, success rates, usage per task; support A/B testing of prompts, skills, and models; and add automated review of queue, priorities, pipeline health, specs, implementation, and testing. Supports goal tracking and auto-scheduling of improvements.
process_summary Execution time: Per-task duration in agent_runner logs and task log footer (done: `duration_seconds` in task log); Task success rate: Aggregate completed vs failed by task_type, model, executor; store in metrics (JSON/SQLite); Usage per task: Token/cost proxy or API call count per task (when providers expose it); Pipeline metrics endpoint: `GET /api/agent/metrics` returning execution time P50/P95, success rate, usage summary; Prompt variants: Support `context.prompt_variant` or prompt ID in task; log which variant was used
pseudocode_summary -
implementation_summary api/app/services/agent_execution_metrics.py (resolve_cost_controls(), attribution_values_from_output()); api/app/services/agent_execution_hooks.py (register_lifecycle_hook(), dispatch_lifecycle_event()); api/app/routers/agent_status_routes.py (pipeline status endpoints)