The renewal engine calculated the rate change correctly. It stored it. It surfaced it on a dashboard, where it appeared as a healthy green metric. Then it renewed the policy regardless, because nothing in the code path ever read the number back.
This is the most dangerous defect shape we know of, because every signal you have says the control is working. It is the reason we now treat "the dashboard is green" as evidence of a dashboard, not of a control.
What the code looked like
Simplified, but structurally faithful:
def process_renewal(policy, quote):
rate_change = compute_rate_change(policy, quote) # correct
log.info("rate change %.1f%%", rate_change * 100) # logged
metrics.gauge("renewal.rate_change", rate_change) # on the dashboard
record_rate_change(policy, rate_change) # in the database
# ... and then:
issue_renewal(policy, quote) # unconditional
# The threshold that was supposed to force a referral above
# +25% is never compared to anything.Read that top to bottom and it looks like a function that cares deeply about rate change. Four lines mention it. The number is right in all four. The fifth line ignores it.
Every artefact said the control existed. The only thing missing was the
if.
Why this survives review
Three reasons, all of them about how humans read code.
Density reads as intent. A reviewer skimming that function sees rate change computed, logged, measured and persisted. The density of references creates a strong impression that the value is load-bearing. Nobody scans for the absence of a branch.
The diff that introduced it looked like an improvement. These functions usually start with the enforcement in place and lose it during a refactor — an early return added for a different case, a branch moved into a helper that is no longer called, a feature flag that defaulted open and was never cleaned up. The commit that breaks the control is a commit that adds something.
The metric goes up. After the change, referrals drop and throughput rises. On any operational dashboard that is a win. The control failing open is indistinguishable from the business getting easier.
What it looks like in the ledger
This is where it becomes visible, and the shape is distinctive.
flowchart LR
subgraph healthy["What a working control looks like"]
A1["120 renewals"] --> B1{"rate change
above +25%?"}
B1 -->|"14 yes"| C1["Referred
authority record"]
B1 -->|"106 no"| D1["Renewed
business event"]
end
subgraph broken["What ours looked like"]
A2["120 renewals"] --> B2["rate change computed
14 above threshold"]
B2 --> D2["120 renewed
business events"]
C2["0 authority records"]
end
style B1 fill:#efecfe,stroke:#4b34e0
style C1 fill:#fbede3,stroke:#b4531b
style C2 fill:#fbede3,stroke:#b4531b
style D2 fill:#e2f5f9,stroke:#0e97b0
This is the diagnostic that generalises: for every control you believe you have, there should be a non-zero count of it firing. A control with zero events in ninety days is either protecting against something that never happens, or it is not running. Those two cases look identical from a dashboard and completely different in a ledger.
The detection query
You can run a version of this today against whatever you have.
-- For each capability that has a threshold, compare the number of
-- actions that should have tripped it against the number of authority
-- records actually written.
SELECT b.capability,
COUNT(*) AS actions_over_threshold,
COUNT(a.id) AS authority_records,
COUNT(*) - COUNT(a.id) AS unaccounted
FROM business_events b
LEFT JOIN authority_decisions a
ON a.correlation_id = b.correlation_id
AND a.capability = b.capability
WHERE b.amount > threshold_for(b.capability)
GROUP BY b.capability
HAVING COUNT(*) - COUNT(a.id) > 0; -- anything here is a holeAny row this returns is an action that crossed a threshold without a corresponding decision. In a healthy system the result set is empty. Ours had one row, and that row was fourteen renewals.
The fix, and the second fix
The immediate fix is the missing branch. The durable fix is structural: make it impossible to perform the write without going through the check.
# Before: the check and the write are siblings, and the write can
# be reached without the check.
rate_change = compute_rate_change(policy, quote)
issue_renewal(policy, quote)
# After: the write requires a decision object it cannot fabricate.
decision = policy_engine.check_renewal(policy, quote) # returns Decision
if not decision.allowed:
return escalate(decision)
issue_renewal(policy, quote, decision=decision) # required argumentMaking the decision a required argument of the write is what stops this recurring. A future refactor that drops the check no longer compiles — or in a dynamic language, fails immediately and loudly rather than silently permitting everything. The type system does the remembering.
A control that is adjacent to an action will eventually be separated from it. A control that is required by the action cannot be. Prefer designs where the permission result is an input to the operation rather than a preceding statement.
Where else to look for this
Once you know the shape, it turns up in familiar places:
- Feature flags that default open. The flag was added to roll out the control gradually, the rollout finished, nobody removed the flag, and a config reset turned it off.
- Validation that returns a result nobody checks. A function returning a list of errors, called for its side effects, with the return value discarded.
- Async checks racing the write. The check runs in a background task; the write does not wait for it. Everything is computed, nothing is enforced.
- Warn-only modes that were never promoted. Shipped as "log only for two weeks while we tune it", still log-only two years later.
- Dashboards built for a control that is not wired. If a metric exists for a rule, confirm the rule has a branch. Our whole post is this case.
What this looks like in InsightLense
The diagnostic is a count of zero where a count of some was expected, and it only exists if authority decisions and business events live in the same place. InsightLense compares what should have tripped a control against what actually produced a record.
| Control | Threshold | Over threshold | Records | Unaccounted | Status |
|---|---|---|---|---|---|
| settle_claim | > $50,000 | 11 | 11 | 0 | enforced |
| post_reserve | > $250,000 | 41 | 41 | 0 | enforced |
| renewal_rate_change | > +25% | 14 | 0 | 14 | not enforced |
| cede_to_layer | capacity | 312 | 312 | 0 | enforced |
This view is the answer to the uncomfortable question the post ends on. Rather than asking whether you believe a control is working, InsightLense gives you the count, per control, and flags any control whose count is zero when the underlying actions say it should not be.
The uncomfortable question
For each control you believe protects your agents, can you produce the count of times it fired in the last ninety days?
If the answer is a number, good. If the answer is "it would have shown up in the logs" or "the dashboard looks fine", you have a dashboard. Whether you have a control is currently unknown, and the only way to find out is to look at what actually happened rather than at what was computed.
Where do your agents already act on real records?
Tell us that, and what you would need to prove about those actions to an auditor. We will set up a hands-on walkthrough within two weeks.
Request a demo →