One change, checked against everything already decided

A walkthrough of provLedger: a task goes to the coding agent, the check runs before anything is edited, and the reason it fires is already on the record.

●~/churn-model — claude
terminal
01The task
~/churn-model on main
>
🏠 Home 🎯 Outcomes 📋 Task · read only
CHURN-MODEL

Retrain the churn model on an 80/20 split

churn-t118 published 14:22 4 steps 3 things touched

Findings

2 blocking
⚠
This experiment has been run before
80/20 was tried on 2026-08-14 and rejected: the holdout leaked week-52 promotions, so the lift was the promotion and not the model.
pkg.training.split_dataset 3 hits open the record →
⚠
weekly_report depends on this split
Computed downstream: split_dataset → train_churn → weekly_report → the churn figure in the Monday deck.
3 steps · 5 consumers open the graph →
1 unanswered

Decisions relied on

  • reason:96Stratify on churn, not on tenure — the tenure buckets are not balanced after 2026-06.
  • reason:104Seed is fixed at 20260614 so a rerun is comparable.
  • reason:118Holdout must not contain a promotion week.
1read the current split and its history
2answer the findings before editing
3change the split and retrain
4compare against the recorded expectation
🏠 Home 🎯 Outcomes 🧬 Node ledger · read only
CHURN-MODEL

pkg.training.split_dataset

derived upstream 2 · downstream 5 14 records · 6 analysis runs

Change history

2026-09-17
Proposed: move the split to 80/20 and retrain.
churn-t118open
stated
2026-08-14
Rejected: 80/20 leaked week-52 promotions into the holdout. Keep 70/30 until the promotion calendar is a column.
churn-t094rejected alternativesource level verbal
2 hits stated
“We tried eighty-twenty in August and the holdout had week fifty-two in it — the lift was the promotion, not the model. Leave it at seventy-thirty until the promotion calendar is its own column.”
tier
stated — the user's own words, with the span
source level
verbal · the words themselves
recorded in
churn-t094, 2026-08-14 11:07
surfaced
3 times · adopted by 1 plan
provledger why pkg.training.split_dataset
2026-06-02
Stratify on churn, not on tenure — the tenure buckets stopped being balanced.
churn-t061reason
5 hitsstated
2026-05-19
Signature changed: returns (train, test, meta) instead of (train, test).
run 71measured from the source
derived
2026-04-03
No reason on record for the original 70/30 choice.
run 44gap, kept as a gap
unstated

Constraints in force

Do not move to 80/20 until the promotion calendar is a column of its own. — from churn-t094
Seed stays 20260614 so two runs can be compared. — from churn-t104
🏠 Home 🎯 Outcomes 🕸 Project state graph · read only
CHURN-MODEL

run 87 · 2026-09-17 · 612 nodes · 188 with records

Data flow — what is read, what processes it, what comes out. Only what has a story is drawn.
FocusStory Data flowFull
read processed produced data/orders.parquet 18 columns data/customers.parquet 11 columns feeds/promotions external pkg.training.split_dataset 14 records · 2 constraints pkg.training.train_churn 9 records pkg.features.promo_flag declared pkg.rollup.weekly_report Monday deck metric: churn_rate tracked since 2026-03

What this change would reach

3 steps downstream 5 consumers 1 metric on the end of it
Computed from the graph on this run and compared with the run before it — not reported by the agent.
0:00 / 1:12

Interface wording is the product's own; the ledger shown here is a worked example, not a live database. The dashboard is read-only and opens the database read-only, so it can never block or change what it is showing — the check records and surfaces, it never vetoes. Try it with claude plugin marketplace add yizhao95/prov_ledger.

↗ github.com/yizhao95/prov_ledger