Where the arithmetic actually lives

The rule I run every system on is one I go into more in a companion piece: the model never authors a number a user will act on. This week I wanted to go one level deeper, the fundamentals-series treatment: what do the primary sources actually say about models and arithmetic, and what does the layer that replaces the model's math look like in real code?

What the sources say

The GPT-3 paper (Brown et al., 2020) has a section people forget about: arithmetic as a benchmark. The pattern in their results is the useful part. The model handles small-digit addition well and degrades as the numbers get longer, which is exactly what you'd expect from a system that has seen "2 + 2 = 4" thousands of times and "38,294 + 71,806" approximately never. It's pattern completion over digit strings. There's no adder inside.

Three years later, Toolformer (Schick et al., 2023) is the field conceding the point in the most direct way possible: the authors taught a language model to recognize when it's about to need arithmetic and emit a call to a calculator API instead of guessing. The research answer to "can models do math" turned out to be "stop asking them to." Every function-calling and tool-use API since is the same concession, productized.

That's the whole theoretical foundation for this week, and it's short. A language model produces the most plausible next token. "1,847" after the word "calories:" is a plausible token. Plausible and correct only coincide.

The tiny build, and how it breaks

The toy version of the mistake looks like this:

// Version A: the model authors the number
const answer = await llm(
  `Total the macros for: 2 eggs, 150g cooked chicken breast, 1 banana`
);
// "Roughly 460 calories, 52g protein..."

Version A often returns something close. Close is the trap, and it breaks in two ways, one obvious and one worse.

The obvious break: sometimes the number is wrong, and because it's usually only a little wrong, it survives review. A macro total off by 8% gets eaten every day by someone trusting the app to manage that number.

The worse break has nothing to do with accuracy. Suppose version A returned the right total every single time this month. You still can't write a test that proves it will tomorrow, because there's nothing to pin. Same inputs, and the output is allowed to vary. The property you actually need, "this total is the sum of these entries," isn't a behavior you can prompt for. It's an invariant, and invariants have to live in code that can hold them.

// Version B: the model chooses, the code computes
const items = await llm.propose(request, candidates); // food IDs + quantities only
const total = items.reduce(
  (sum, i) => sum.add(nutritionFacts(i.foodId).scale(i.grams)),
  Macros.zero()
);

Version B is boring and testable. nutritionFacts is a lookup, scale and add are arithmetic, and a unit test proves the whole chain forever.

What the production version adds: a resolver with four layers

The toy hides the real question, which is where nutritionFacts gets its numbers. In my fitness app the answer is a resolver that tries four sources in order, and the order is the design:

1. Coach-verified plan items. If a food is part of a plan a human already verified, those macros win. Highest trust, narrowest coverage.
2. USDA FoodData Central, cached. Public, versioned nutrition data for anything the plan doesn't cover.
3. A deterministic fallback table. About 30 common foods with rounded label values, each with an explicit basis: per 100 grams, or per unit with the unit's gram weight pinned in code. An egg is 50 grams. A banana is 118. Cooked chicken breast is 165 kcal and 31 grams of protein per 100 grams. These numbers are hardcoded on purpose, because a constant is the most testable data source there is.
4. An LLM estimate, flagged. Last resort, for the long tail the other three miss, and it never pretends to be anything else: the estimate carries a low-confidence flag that follows it to the UI.

Layer 4 is the part worth staring at. There is a place for the model to produce a number in this system, and it's at the bottom of the chain, reached only when three deterministic sources have failed, and marked so the user knows what they're looking at. The rule was never "the model must not touch numbers." It's that a model-authored number can't be allowed to impersonate a computed one.

1

Model interprets the request

“Swap the rice for something lower carb that still gets me to 40 grams of protein.” It proposes food IDs and quantities, never totals.

2

Resolver finds the facts

Coach-verified plan item, then cached USDA data, then the 30-food fallback table with pinned unit weights.

3

Plain code does the arithmeticgate

Scale by grams, sum the macros. Same inputs, same totals, forever, and a unit test proves it.

4

Only if everything misses: LLM estimate, flagged

The one model-authored number in the system, and it's labeled as one.

The same split, two more systems

Once you draw the line as "models choose, code computes," you find it load-bearing everywhere.

In Enacted, my Ontario law tracker, the diff between two versions of a regulation is computed by the diff library with three lines of context per hunk. The model writes prose about a diff it was handed, and a deterministic gate rejects any summary containing a date or citation that wasn't in its input. A date is a number too, and the model doesn't get to author those either.

In PermitCheck, the verdict on a zoning dimension is a comparison between an extracted value and a limit resolved for that specific property. The comparison is a pure function with plausibility ceilings built in, so an implausible read (a decimal-slip 3,300% lot coverage) becomes "not verifiable" instead of a confident fail. The model reads drawings. It never decides pass or fail.

The question to carry out of week 2

For every number that appears on a screen you ship: which function produced it, and does that function have a test? If the answer to the first is "a prompt," you don't have an answer to the second, and that's the whole problem. The model's job is to understand what the user wants. The arithmetic lives somewhere with an assert.