What Counts as a Verified Resolution

If you bill per resolution, the entire commercial relationship rests on one definition: what counts as resolved? It is tempting to keep this fuzzy, because precision creates obligations. But fuzziness is exactly what turns a clean idea into an untrustworthy invoice. So it's worth stating plainly what a verified resolution should and should not mean.

Start with what it is not. It is not the conversation ending. It is not the customer going quiet. It is not the AI declaring itself successful. It is not a sentiment classifier guessing the customer seems happy. Each of these is a proxy, and each can be satisfied while the customer's actual problem remains unsolved. Worse, several of them are satisfied most reliably in your failure cases — the frustrated customer who gives up looks, to a naive meter, exactly like a resolved one.

A verified resolution has to clear a higher bar. There should be an identifiable question or task the customer brought. There should be a concrete response the AI gave to that task — not a deflection to an article, but an actual answer or action. There should be evidence the response was grounded in real sources or system state rather than fabricated. And there should be a confirmation signal: the customer indicating the issue is handled, or an outcome in the system (an order changed, a refund issued, a ticket the customer agreed to close) that demonstrates the task was done.

Crucially, the boundary cases should resolve in the customer's favor, not the vendor's. If the AI handed off to a human, that is not a resolution by the AI, even if the human later closed it. If the AI answered but the customer reopened the same issue, the original "resolution" was wrong and the credit should be reversed. If confidence was low and the system hedged, that is honest behavior, but it is not a billable resolution. The rule of thumb: when in doubt, it's not a resolution. A meter that errs toward charging is a meter you cannot trust.

Then there's the receipt. A verified resolution should produce an artifact you can actually open — what the customer asked, what was answered, which sources grounded the answer, the confidence at the time, and why the system concluded the issue was resolved. This is the difference between a number on an invoice and an auditable charge. The receipt is what lets your finance team reconcile spend, lets your support leads spot-check quality, and lets you dispute a charge with a specific factual basis rather than a vague complaint.

Disputes are the other half of verification, and they're the part most people skip. A resolution isn't truly verified if the verification is one-sided. You need a path to challenge a specific charge and have it adjudicated against its own evidence — and you need reversals to actually happen when the evidence doesn't hold up. A judging mechanism that no one can appeal is not verification; it's just automated confidence. The dispute flow is what keeps the judge honest.

It's worth being candid about the hard parts. Verification has its own failure modes: a model judging whether a resolution occurred can itself be wrong, grounding checks can miss subtle fabrication, and "the customer confirmed" can be gamed by leading prompts. None of this is solved by declaring victory. It's managed by layering signals, keeping a human in the loop during alpha, biasing every ambiguous call toward not charging, and treating the dispute rate as a quality metric rather than a nuisance. We'd rather under-bill and learn than over-bill and erode trust.

So: what counts as a verified resolution? A real task, a real answer, real grounding, a real confirmation, a receipt you can read, and a charge you can dispute. Strip away any one of those and you no longer have verification — you have a confident guess with a price attached. The whole reason to make the definition strict is that the strictness is the product. It's what makes "pay only when it works" mean something.