W1: Auto-tagging

In plain terms. When a card swipe, payout, or vendor bill lands in Reap, somebody has to decide which ledger account, tax code, and tracking categories it belongs to. Today a controller does it by hand, one line at a time. This skill does it automatically. It first checks whether the same vendor has been coded before for this customer — if yes, it reuses that coding for free. Only when the system has not seen the vendor before does it ask a language model, and the model is forced to answer in a strict shape it cannot break out of. Every coding is saved with the evidence the system used, and every time a controller corrects one, that correction becomes a rule for next time. This is the only one of the four skills where the language model does real work, which is exactly why the model is the smallest possible part of the pipeline.

For context, the assignment brief this skill is answering is folded into the panel below. It is worth opening once — the pain it describes (month-end close stalling on uncoded entries, silent miscoding corrupting P&L and VAT filings) is what every design choice on this page is reacting to.

Assignment brief — Workflow 1

Agent that codes corporate-card and bill-pay transactions to the right general ledger account, tax code, and tracking categories, syncs to the customer's accounting system, and learns from every accountant override.

Mid-market multi-currency businesses on Reap process ~250 transactions/month. Today, those land in Xero/QuickBooks Online/NetSuite as uncoded entries that a controller or outsourced bookkeeper hand-classifies — typically 30–90 seconds per line for routine items, longer for ambiguous ones. Month-end close stalls on this queue. Silent miscoding is the worst outcome: it corrupts P&L, VAT/GST filings, and management reporting, and is costlier to unwind than a delayed entry.

Here is what a controller actually sees today when this skill is running on a tenant's transactions — the queue is the surface where every coding lands for review.

Queue surface: the reviewer sees outcome, confidence, evidence, version stamps, and the override path in one register.

Notice what is not on that queue: there is no "trust me" answer. Every row carries the evidence the system used, the confidence it had, and the version of the chart of accounts it was checking against. That is the surface the rest of this page is about.

Submission proof

Before walking through how the pipeline works, four numbers fell out of the build that are worth knowing up front. They are the easiest way to answer "did this actually get done, and how much of it is real?"

37golden auto-tagging cases

4refusal reason families

$0model cost on vendor-rule hits

1override loop into vendor rules

The shape of those numbers matters as much as the numbers themselves. Thirty-seven golden cases is enough to catch regressions across the four refusal families. Zero model cost on a vendor-rule hit is the proof that the cheapest path is actually free. And exactly one override loop — not three, not zero — is what keeps the learning surface auditable.

What runs

Now the pipeline itself, drawn left to right. An event arrives, the system tries the cheapest answer first, calls the language model only when there is no shortcut, and gates the result before anything is logged. The model — the part most readers expect to be the centrepiece — is the third stop on a four-stop tour.

EVFinanceEventCard, payout, and bill events arrive in one typed shape.

→

RULEVendor memoryKnown vendor + tenant rule emits a deterministic coding.

→

ModelTyped classifierOnly misses call generateObject with chart-of-accounts and vendor evidence.

→

GATEAutonomy policyConfidence maps to apply, suggest, or refuse before logging.

Read left to right, the diagram is also a cost gradient: stage 1 and 2 are effectively free, stage 3 costs an API call, and stage 4 is the only place an autonomy decision is taken. The order is deliberate — it means the bulk of real-world traffic never reaches the parts that cost money or require trust.

Why I made this the lighthouse skill

Looking at that pipeline, a fair question is: why route a "boring" classification problem through this much machinery? Four reasons, each tied to a property of the build you can verify in the demo.

Large language model earns its placeClassification has language ambiguity, vendor-tail risk, and tenant-specific chart-of-accounts mapping. I use a model here because it is useful, but only inside constraints.Schema-checked output only.

Rules beat promptsSecond encounter with a corrected vendor can skip the model entirely.Cost drops, reversals drop.

Refusal is visibleOff-chart-of-accounts, missing input, ambiguous match, or unsafe output become structured refusals.No silent miscoding.

Review compoundsOperator override writes the corrected coding and supersedes the vendor shortcut.The reviewer’s work becomes future automation.

The four rows are arranged from "why we keep the model" to "why the reviewer is the most valuable part of the system." That arc is the whole argument for the design: the model earns its keep, but the deterministic rules and the human override are what make the model safe to use.

Testing & evals

The auto-tagging eval surface — 37 golden cases, four-provider sweep, refusal metrics, calibration, and reversal-rate replay — lives on its own page so it can be diffed across prompt versions without scrolling past the overview. Sonnet 4.5 hits 86.7% top-1 and 100% refusal precision; refusal recall caps at 33% — the load-bearing weakness.

• Auto-tagging — evals — model sweep, refusal metrics, and the slices that load-bear the result.

Read next

If you want to see this skill on a single live row — Anthropic card authorization, $266.03, with the system refusing to guess and asking for a receipt — go to the worked example. The skill's own evals page carries the multi-model sweep; the cross-skill harness sits inside Operations.

• Worked example — the Anthropic row end-to-end, plus the design patterns that hold the skill together.
• Auto-tagging evals — multi-provider sweep, refusal metrics, slices.
• Eval harness — the cross-skill harness, dangers & tradeoffs (in Operations).

Next skill in this section: W2: Policy enforcement — where I deliberately take the language model out of the live path because the verdict has to be explainable and sub-millisecond.