Muhtalip Dede · Founder of kprompt3 min readBuilding AI SRE in Public #7: AI Timeline
Incident chronology as typed EvidenceRef[], not chat folklore. Events, rollout revisions, and HPA in one Investigation artifact — with honest degraded[] when Prom/OTel/mesh are missing.
This is episode 7 of Building AI SRE in Public. Episode 6 framed investigation as a gated graph — fan-out, independent verify, merge into a refuse-able artifact. Timeline answers a different question operators ask every night: not “what is broken?” but “in what order did the world change?” Chat scroll is a terrible chronology. Typed EvidenceRef[] is not.
The claim
Postmortems and handoffs need an ordered evidence trail: Events, controller revisions, HPA transitions — pointers to signals, not a novel. AI SRE should emit that trail as data (Investigation.timeline[]), degrade honestly when Prom/OTel/mesh are missing, and never invent timestamps to make the story prettier.
Chat is folklore with timestamps
A forty-turn agent session can contain the truth and still fail institutional memory. “We looked at events, then logs, then scaled” is not reviewable in CI, not mergeable across tools, and not comparable across incidents. Folklore does not jq. Chronology does.
| Chat chronology | Typed timeline |
|---|---|
| Scroll order = claim order | time-sorted EvidenceRef[] |
| Hard to put in a ticket | Same Investigation envelope as investigate/why |
| Invented narrative glue | degraded[] when signals are missing |
| Dies with the session | JSON artifact for humans and pipelines |
What a timeline walks (MVP)
- Events on the target workload (Deployment, StatefulSet, DaemonSet, or Pod) and related pods
- ReplicaSet revisions for Deployments (deployment.kubernetes.io/revision)
- ControllerRevision history for StatefulSets and DaemonSets where available
- HPA targeting the Deployment (status + condition transitions)
Primary payload is timeline[] of EvidenceRef — the same ADR-0014 Investigation kind used by investigate and why. Findings may be thin; chronology is the product. Window defaults to something like 1h so “what happened” stays bounded.
Chronology, not chat
kprompt "timeline for api" -n payments
kprompt "what happened to ledger" -n payments -o json
kprompt "timeline for StatefulSet db" -n payments
kprompt "timeline for api" -n payments -o json \
| jq '{summary, degraded, timeline: [.timeline[] | {type, reason, message, ts: .timestamp}]}'vs investigate / why
| investigate | why | timeline | |
|---|---|---|---|
| Focus | Multi-hop RCA | Cause tree | Chronology |
| Primary field | findings | Symptom→Cause | timeline[] |
| Trigger | “investigate X” | “why is X pending” | “timeline / what happened” |
Same envelope, different jobs. Investigation Graph (ep.6) may fan out signals; Timeline orders what already happened. Neither replaces PlanResult for mutate — chronology informs the plan; approval still gates apply.
Honesty: degraded beats fiction
MVP lists prometheus, otel, and mesh in Investigation.degraded. Timeline does not invent metric spikes or trace spans to fill gaps. That is the same compiler temperament as PlanResult: refuse to fake structure. When those hops land, they append EvidenceRef kinds — they do not rewrite the past into a prettier story.
Why this belongs in an AI SRE series
Classic AIOps dashboards had timelines and still burned trust when they auto-acted. LLM agents have narratives and still burn trust when they skip artifacts. AI SRE needs both: a time-ordered evidence trail and a refuse-able plan. Timeline is the trail. PlanResult is the proposal. Approval is the boundary. Verify closes the loop (ep.6 / Medium “Investigate → Verify”).
What ships vs building
- Shipped: timeline / what happened → Investigation with timeline[] (Events, RS/ControllerRevision, HPA)
- Shipped: same --output json contract as investigate/why; degraded[] for Prom/OTel/mesh
- Building: metric/trace/mesh hops on the chronology
- Building: richer app/timeline viewer surfaces on Team (when enrolled)
- Non-goal: chat-only RCA as the source of truth; inventing signals to fill gaps
Try the chronology
Non-prod drill
# kind + a broken workload (see kprompt-examples)
make break SCENARIO=01-crashloop
kprompt "timeline for api" -n payments
kprompt "investigate api" -n payments # RCA beside chronology
# Prefer reading JSON timeline[] before any suggested mutateIf your AI ops tool cannot export “what happened” as ordered evidence without scraping a chat UI, you still have folklore — just with better English.
Next
Episode 8 is Cluster Memory — durable local facts that bias Observe context without becoming sole proof. The hub tracks the rest of the arc.
Related posts
Building AI SRE in Public #9: Knowledge Graph
Read-only service topology as typed nodes/edges — Ingress, PVC, Secret/ConfigMap names, reverse impact — not a Secret CMDB or chat “who depends on what” folklore. Honest degraded when OTel/mesh are missing.
Read articleBuilding AI SRE in Public #8: Cluster Memory
Namespace dependency facts that bias Observe without becoming root-cause proof. Local or in-cluster stores — never cloud dumps as fake authority — with AG-034 confidence caps.
Read articleBuilding AI SRE in Public #10: Autonomous SRE — and why not yet
Why unsupervised auto-remediation is not the destination. Observe by default, Autopilot propose-only, reality anchors, and investigate → plan → approve → verify as the load-bearing loop — not a fleet of agents that apply because the model sounded sure.
Read article