Langfuse, Helicone and InsightLense: where LLM observability stops

If you are running LLM features and you use Langfuse or Helicone, you probably do not need us. They are good tools, built by people who understand the problem, and for most of what teams are shipping today they are the right answer.

This post is about the specific point where that stops being true, and it is not a capability gap so much as a category difference. It arrives the moment your agent is allowed to write to a system of record.

What they do well, honestly

Worth being specific, because a comparison post that will not name a competitor's strengths is not worth reading.

Langfuse has the strongest open-source story in the category. Self-hostable, genuinely good trace visualisation, prompt management with versioning, a mature evaluation and dataset workflow, and an active community. If your problem is "we are building an LLM product and need to see what the model is doing and whether it is getting better", it is an excellent default and the fact that you can run it yourself removes most of the procurement objection.

Helicone optimised for the thing that keeps teams from adopting observability at all: integration effort. A proxy-based setup means you get request logging, caching, rate limiting and cost tracking by changing a base URL. For a team that wants visibility this afternoon rather than after a sprint, that trade is often correct.

Both are better than us at the thing they built for. We ingest standard OTLP precisely so you do not have to choose on the instrumentation layer — the same exporter can feed both.

The line

Here is the distinction as plainly as we can put it.

A model-call log tells you what was said. An accountability record tells you what changed, under whose authority, and what was refused.

For a chatbot, a summariser, a copilot that drafts text a person then accepts — those are the same thing. The model output is the product. Logging the call captures the whole event.

For an agent that posts a reserve, issues an endorsement, releases a payment or cedes a loss to a treaty layer, they diverge completely. The model call is the least consequential part of what happened. The consequential parts are: a number in a database changed, a permission check ran, and a human either was or was not asked.

What is structurally absent from a model-call log

Not missing features — structurally absent, because the data model has no place for them.

That fourth row is the one we would emphasise. In a system with meaningful human oversight, the approval is the point. A tool that only sees model calls is blind at exactly the moment an auditor cares about most — and no amount of better trace visualisation fixes that, because there was no trace.

The two layers, side by side

flowchart TD
    subgraph L1["Layer 1 — the model call"]
        A["Prompt"] --> B["Completion"]
        B --> C["Tokens, latency, cost,
model version"]
    end

    subgraph L2["Layer 2 — the consequence"]
        D["Tool invoked"] --> E{"Permission check"}
        E -->|allowed| F["Record changed
50k to 250k"]
        E -->|refused| G["Refusal recorded"]
        E -->|escalated| H["Human approves"]
        H --> F
    end

    C -.->|"same correlation_id"| D
    F --> I["Auditable ledger"]
    G --> I
    H --> I

    style L1 fill:#f6f6fb,stroke:#6b7396
    style E fill:#fbede3,stroke:#b4531b
    style G fill:#fbede3,stroke:#b4531b
    style H fill:#fbede3,stroke:#b4531b
    style F fill:#e2f5f9,stroke:#0e97b0
    style I fill:#e2f5f9,stroke:#0e97b0

An honest capability comparison

Where we are genuinely behind, and where we are genuinely ahead.

Not either / or

The framing we would actually defend: these are different layers, and running both is reasonable. Because we accept standard OTLP, one exporter can fan out to both — your engineers keep the tracing UI they like, and your risk and audit functions get a ledger they can read.

What we would push back on is the assumption that a model-call log will grow into an audit trail. It will not, for the same reason an application log never became one: the shape is wrong. An audit trail needs entities, values, actors, authority and immutability. Those are schema decisions, and schema decisions are the ones you cannot retrofit onto eighteen months of history.

What this looks like in InsightLense

The clearest way to show the difference is to put both layers on the same incident. Here is one settlement as a model-call log sees it, and as InsightLense sees it.

Which is why we would genuinely run both. InsightLense ingests standard OTLP, so one exporter can feed your existing tracing UI and this ledger at the same time — engineers keep the tool they like, and risk and audit get records that have somewhere to put a human decision.

The question that decides it

Ask what happens when an agent does something wrong and expensive.

If the answer is "we look at the trace and see what the model said", a model-call log is sufficient, because the model saying the wrong thing was the whole failure.

If the answer involves a number in a database, a limit that should have stopped it, and a person who should have been asked — then you need records that a model-call log has no column for, and the time to have them is before the incident rather than after.