sheets.fast evals

Can an agent do everything a spreadsheet user needs?

We measure it. 60 tasks written as real user intents, covering data entry, editing, formulas, formatting, structure, SQL analysis, webhooks, organization, import/export, automation, safety, composites. Real models run them against the live MCP endpoint on this exact infrastructure, and every task is graded on the final state of the workbook through the public API.

Last run 2026-08-30 · latest result per task (a failure triggers a product fix, then a re-run) · fixtures created and destroyed on production.

ModelTasks passedAvg tool calls / taskMedian task time
Free-tier model (OpenRouter)59/60 3.415.8s

Model IDs: openrouter/free (via OpenRouter).

Why small models pass here

The whole tool surface is 9 tools, ~2,523 tokens. That is every word an agent must read before it can act. For comparison: mcp-google-sheets ≈ 13k tokens, Notion MCP ≈ 17k tokens. A tool surface an agent can hold in its head is a tool surface it uses correctly. Even a free-tier model completes real spreadsheet work here.

Evals drive the product

Every failure is triaged: a missing capability becomes a shipped feature, a confusing error becomes a better message, a misused tool becomes a clearer description. Append, drag-fill, style read-back, workbook duplication, folder filing, and CSV/XLSX export URLs all shipped because an agent stumbled without them. Then the suite ran again until it stopped stumbling.

Try it yourself: connect Claude or ChatGPT, or read llms.txt and go.