August 14, 2026

The number went up and nobody spent more.

The hardest measurement conversation is not the one where a number falls. It is the one where a number rises for a reason that has nothing to do with performance. A rollup dropped a grain, a whole geographic breakdown had been reporting a fraction of reality, and the fix makes it jump. Nothing was overspent. Nothing improved. And you have just proven to a client that your pipeline can be quietly and plausibly wrong in a direction nobody would have thought to question.

Everybody in measurement is trained for the conversation where a number falls. We have the framing ready before the meeting starts: here is the definition that changed, here is the lookback, here is what the full funnel says, here is what we are doing about it. It is uncomfortable and it is familiar.

Almost nobody is trained for the harder one, which is a number rising for a reason that has nothing to do with performance.

I had that week. A rollup in a reporting pipeline was dropping a grain — city-level spend was mostly failing to aggregate up into the state breakdown, which meant an entire geographic view had been reporting a fraction of reality for as long as anyone had been reading it. Understated by a multiple, not a margin. The fix ships, the numbers jump, and the jump is a reporting correction rather than a spend change.

The engineering is a footnote. The interesting part is what happens next.

Why up is harder than down

When a number falls, you are apologizing, and apologizing is a known shape. Everyone has been in that room.

When a number rises because the pipeline was wrong, you are handing a client a reason to distrust every number you gave them before — including the good ones. You have just demonstrated, with a concrete example, that your reporting can be quietly and plausibly wrong in a direction nobody would ever have thought to question. Nobody audits a number for being too low. That is the whole problem.

So a downward correction costs you a conversation, and an upward one costs you a piece of the background assumption that your numbers are right. That assumption is not a nice-to-have. In this business it is the product. Everything else — the strategy, the optimization, the recommendations — sits on top of a client’s willingness to take the measurement as given.

That is the asymmetry, and it is the reason I think this is worth writing down rather than filing as a bug.

Why the bug survives at all

A number that is wrong by a multiple can still look completely reasonable.

That is the entire mechanism. If the geographic breakdown had shown a negative number, or a figure ten times the total spend, it would have been caught the first afternoon. Instead it showed a plausible number, inside the range where nobody looks twice, and it kept showing one every day for months. No alert fired because nothing was out of bounds. No reviewer objected because there was nothing objectionable on the screen.

This is why review does not catch this class of bug. Review reads the output, and the output is fine. It is caught by somebody changing the grain — by asking the same question at a different level of aggregation and noticing the two answers disagree.

Which gives you the one concrete check I would take from this. Reconcile totals against a grain you did not aggregate. If the state view is built by summing city rows, sum the city rows independently and compare the two. If a channel total is built from campaign rows, total the campaigns separately. It takes fifteen minutes and it is the only thing that would have caught this, because it is the only check that does not trust the aggregation path that broke.

And a large upward jump immediately after a pipeline fix is not an anomaly to investigate. It is the signature of this bug class. If you see it, the story is almost certainly that a grain was being dropped, and the correction is real.

The sequencing is the job

The night before the fix shipped, I wrote myself an email. The subject line was: read before a client asks.

Reading it back, that subject line is most of the professional content, and the body is almost incidental. The technical facts were four sentences. The judgment was entirely in the ordering.

The same fact — a number moved for a structural reason — lands as two completely different things depending on who says it first. From you, before they look: a competence signal. Evidence that you audit your own pipeline, catch your own errors, and tell them things they did not ask about. From their own dashboard, before you say anything: a credibility event. And once it is a credibility event, you are not explaining anymore. You are defending, and the substance of your explanation stops mattering nearly as much as the fact that it arrived second.

There is no clever version of this. Get there first, name the class of bug plainly, say explicitly that it is a reporting correction and not a spend change, and say what you changed so it does not recur. Do not soften the size of it, and do not lead with the fix as though the fix were the news. The news is that a number they were using was wrong.

The other two shapes

This is one of three shapes, and it is worth recognizing the others, because they all come to the same doorstep.

There is the downward version you do not control. A major analytics platform shortened its default acquisition-conversion lookback from 90 days to 30 earlier this year. Some accounts watched reported search conversions fall roughly 18% while their sales teams saw steady, unchanged lead volume. Nothing about performance moved. A default moved, chosen for an account shape that was not theirs, and the reported number followed. Same event as mine with the sign flipped and the cause outside the building.

And there is the version where nothing changes at all and the definition was simply always wrong. I had that one too, in the same stretch: a headline metric that ignored the date picker sitting directly above it, reporting a fixed trailing window regardless of the range selected. We corrected it. And the consequence worth generalizing is that every explanation anyone had ever given about what that number meant became wrong on the day it started behaving correctly. The number never lied more than it had the day before. It just stopped, and all the guidance built around it had to be retracted.

Three shapes, one job: a number moved for a reason that has nothing to do with performance, and somebody has to say so before the client notices.

The takeaway

Build the check that changes the grain, because it is the only one that catches a plausible wrong answer, and plausible wrong answers are the ones that live for months. Then treat the conversation as the deliverable rather than the fix. Get to the client before the correction does, say plainly that it is a reporting correction and not a spend change, and accept that an upward correction is the harder conversation precisely because nobody audits a number for being too low. The pipeline will be wrong sometimes; that is not the thing that damages trust. What damages trust is the client finding out first.

#operator-essays#measurement#attribution#ad-tech
Share