Cost you can reconcile: why we derive spend from traces instead of storing it

Most AI cost dashboards are a second source of truth that quietly disagrees with the invoice. They disagree because they store cost as a number written at request time, and that number is wrong the moment a price changes, a model is renamed, or a retry is counted twice.

We do not store cost. We derive it from the traces, on demand, and rebuild it whenever the inputs change. This post is why, and what it lets you measure that a token dashboard cannot.

The drift problem

The obvious design is to compute cost when the call happens and write it to a column. It is fast to query and simple to reason about, and it goes wrong in four ordinary ways.

Each of these is individually small. Together they are why the AI line in a finance review never matches the AI line in an engineering dashboard, and why the conversation ends with someone exporting both to a spreadsheet.

Derive instead

The trace already holds the facts that do not change: which model, how many input tokens, how many output tokens, when. Price is a separate, versioned thing. Cost is a function of the two.

flowchart LR
    T["llm_traces
model, tokens, timestamp
immutable facts"] --> F{{"price lookup
versioned, by model
and effective date"}}
    F --> C["cost_records
derived"]
    C --> B["Per-actor budgets"]
    C --> O["Cost per business outcome"]
    C --> FC["Forecast"]
    R["Price change,
token fix,
new model alias"] -.->|rebuild| C

    style T fill:#e2f5f9,stroke:#0e97b0
    style F fill:#efecfe,stroke:#4b34e0
    style C fill:#efecfe,stroke:#4b34e0
    style R fill:#fbede3,stroke:#b4531b

The practical benefit is that reconciliation becomes possible. When finance says the invoice was $14,200 and your dashboard says $12,850, you can rebuild against the actual price schedule and find out whether the difference is a pricing assumption, an untracked service, or a genuine discrepancy — instead of arguing about which dashboard to trust.

The metric nobody else reports

Here is the part that changes conversations. Once cost is derived and every run is threaded to a business reference, you can divide.

Not dollars per thousand tokens. Not dollars per run. Dollars per bound policy. Dollars per settled claim. Dollars per submission cleared.

These are the numbers a CFO can act on, because they sit next to a number they already know: what the same work costs today. A submission that costs $0.31 of model spend to clear is trivially justifiable against a loaded hourly rate. A submission that costs $14 is a design problem you want to find before it is at volume.

Tokens per run is an engineering metric. Cost per cleared submission is a business metric. Only one of them survives contact with a budget meeting.

The mechanism is unglamorous: because every run carries the business reference as its correlation id, you group cost by that reference and join to the outcome. There is no separate attribution pipeline — it falls out of the threading model.

Budgets per actor, not per API key

The other thing derived cost enables is attribution to the actor rather than the credential.

In our carrier, spend is not "the claims service used $400 this month". It is "CL-DOKAFOR, the adjuster, used $38 across 17 runs, of which 11 escalated". That is actionable in a way a service-level figure is not: you can see that escalation-heavy work costs more because the agent does more analysis before handing off, and decide whether that is worth it.

Sorting by cost per outcome rather than by total spend is the useful ordering. The agent with the largest bill is usually just the busiest one. The agent with the worst cost per outcome is the one with a problem — a retry loop, an oversized context, a tool that returns too much.

What this costs you

Being honest about the trade-off: derived cost is slower to query than a stored column, and it needs a rebuild job. For most teams neither matters — the rebuild is incremental and the query runs against a rollup table that is itself derived. But if you are computing cost across billions of spans in an interactive dashboard, you will want a materialised layer, and then the discipline is to make sure that layer is only ever rebuilt, never edited in place.

The rule we hold to: facts are recorded, money is computed. Anything you can recompute, recompute. The number that cannot be reconstructed is the number that will eventually be wrong and unfixable.

What this looks like in InsightLense

Because cost is derived rather than stored, InsightLense can group it by whatever the run was about. The view that changes conversations is not spend by service — it is spend by outcome, with the outcome count as the denominator.

The rebuild is what keeps this honest. When a provider changed prices mid-quarter, InsightLense recomputed the whole period against the correct schedule and every figure in this view moved together — per-actor budgets, forecasts and cost per outcome stayed consistent with each other because they are all derived from the same traces rather than maintained separately.

Three questions to ask your current setup

• If a provider changed prices last quarter, can you restate last quarter? If not, your historical cost is fiction and your year-on-year comparison is worse.
• Do unpriced models show as zero or as unpriced? Zero is the dangerous answer.
• Can you produce cost per business outcome? If your traces are not threaded to a business reference, you cannot — and that is a threading problem, not a cost problem.