pilot-expense-tracker-v0 3 / 3 Shipped 1 / 3 Followed spec exactly Pilot · n=1 per stack

Expense Tracker App Benchmark: Claude Code vs Codex CLI vs Antigravity CLI

Our pilot: three agent stacks got the same 7-requirement spec for a single-file expense tracker and one headless run each, with zero human follow-up. All three shipped a working app. Only Codex CLI kept the running total as a total of all expenses when a category filter was on. Claude Code and Antigravity CLI quietly turned it into a filtered subtotal.

3 Stacks 1 Run Each
106s – 231s Wall-Clock Span
0 Interventions
Sep 1, 2026 Test Date

📊 Results by agent stack

1 prompt · 0 follow-ups · n=1
Agent stack Verdict Total under filter Safe localStorage parse Wall time Output tokens Reported cost
Claude Code 2.1.252 + Claude Opus 5
claude-opus-5 · default reasoning
Ship Becomes subtotal Yes 171s 11,916 $0.86
Codex CLI 0.151.0 + GPT-5.6 Sol
gpt-5.6-sol · reasoning medium (machine default)
Ship Correct Yes 231s 8,758 unknown
Antigravity CLI + Gemini 3.1 Pro
gemini-3.1-pro-high · version not recorded
Ship Becomes subtotal No 106s 12,076 unknown

Tokens and cost come from each CLI's own log. Claude Code reports a list-price cost; Codex CLI and Antigravity CLI report tokens only (subscription billing), so cost is unknown. Console errors on load, re-checked Oct 31, 2026 in headless Chrome: 0 for all three.

Expense tracker built by Claude Code, Codex CLI and Antigravity CLI. After filtering to Food, Claude Code and Antigravity CLI show the Food subtotal of $91.90 as the total, while Codex CLI keeps the grand total of $1,365.90.
Same data, one filter click. Claude Code and Antigravity CLI change the total to the Food subtotal. Codex CLI keeps the grand total, as the spec asked.
🔒 1p1a.com/pilot-expense-tracker-v0/apps/claude.html
Open Fullscreen ↗

📜 The frozen benchmark prompt

Sent verbatim to each agent CLI as its only input. Task version v0, frozen.

Build a single, self-contained HTML file called `index.html` for a personal expense tracker.

Requirements:
1. One HTML file only. No build step, no server, no external network requests, no external CSS or JS files — all CSS and JavaScript must be inline in the single file.
2. A form to add an expense with: description (text), amount (number, dollars), category (select: Food, Transport, Housing, Entertainment, Other), and date (date picker, defaulting to today).
3. A table listing all added expenses, newest first, showing description, amount, category, date, and a delete button per row.
4. A running total of all expenses, prominently displayed, updating live as rows are added or deleted.
5. A filter dropdown to show only one category at a time, or all.
6. All data must persist in the browser via localStorage, so a page refresh keeps the data.
7. Clean, readable visual design. Currency values formatted as $X.XX.

Do not ask any clarifying questions. Build the complete file, verify it works, and stop.

🔬 What went wrong, and where

1. Running Total Under Category Filter Logic Edge Case

Requirement 4 specified "A running total of all expenses, prominently displayed, updating live as rows are added or deleted."
• Codex CLI computed the total from the full data list, so it stays the grand total when the list is filtered to Food.
• Claude Code and Antigravity CLI computed it from the filtered rows, so the "total" drops to the Food subtotal.

2. Defensive JSON Storage Parsing Crash Vulnerability

• Claude Code & Codex CLI enclosed JSON.parse(localStorage.getItem(...)) within a try/catch block with fallback defaults.
• Antigravity CLI executed raw JSON.parse without protection. If corrupted data existed in localStorage, the page suffered an uncaught exception on boot.

3. Why a simple spec still matters Takeaway

The spec has 7 short requirements and every agent said it was done. Two of three still changed what the total means, independently, in two different model families. That is the kind of silent spec drift a screenshot comparison never shows, and the reason the next 1P1A benchmark ships with an automated test suite. Methodology · All results side by side