Accounts payable — evals

~3 min read · two-layer testing on the highest-stakes path

Money movement is the highest-stakes path in the system, so this skill is tested in two layers and refuses to ship a plan unless both pass. The 18-case runtime golden corpus — 14 schedule, 2 overdue, 3 discount, 2 prompt-injection, 2 refusal — exercises the planner against the same FinanceEvent shape the queue uses. The 8 AP-optimizer boundary cases then test the foreign-exchange / discount trade-off math directly via Vitest at tests/ap-optimizer-eval.test.ts, separating optimizer correctness from end-to-end planning.

Coverage

Accounts payable · 18 runtime + 8 boundary14 schedule, 2 overdue, 3 discount, 2 injection, 2 refusal. 8 AP-optimizer boundary cases pin the corridor / discount math via Vitest.

18runtime golden cases

8AP-optimizer boundary cases · Vitest

2refusal cases · missing FX, hostile invoice

≥ $10kdual-control trigger asserted in tests

The two refusal cases are the load-bearing ones. Missing-FX refuses rather than guessing a corridor rate; a fabricated invoice line item refuses rather than scheduling the wrong amount. The duplicate-invoice and bank-account-change cases on the worked example are encoded in the golden corpus so they cannot regress silently. Reproduce with pnpm eval -- --skill accounts-payable for the runtime corpus and pnpm test ap-optimizer-eval for the optimizer boundary cases.

Slices worth growing next

• Schedule correctness: paymentDate is one banking day before dueAt, urgency tier matches daysUntilDue.
• Source-of-funds selection: United States Dollar Coin corridors → reap-pay, same-currency → reap-card, idle-cash → reap-optimize.
• Refusal: bill with no dueAt (missing_input).
• Escalation: overdue bill → escalate(controller).
• Constraint-violation rate: the new accounts-payable optimizer corpus asserts no selected sleeve breaches its expected minimum remaining cash buffer. Normalized discounted cumulative gain against a controller-ranked golden is still a future production eval.
• AP-optimizer boundary cases (evals/ap-optimizer.jsonl, 8 cases) cover same-currency card routing, cross-currency Reap Pay routing, large same-currency Optimize routing, fallback routing when the preferred sleeve is underfunded, insufficient-funds escalation candidates, and early-pay discount timing. The Vitest gate asserts schedule, selected sleeve, shortfall, discount timing, and minimum remaining cash buffer for each case.

Read next

• Overview — the recommendation-only posture and dual-control rails.
• Worked example — the duplicate invoice and the bank-change cases, end to end.
• Eval harness — the cross-skill harness (in Operations).