Build the optimal OpenClaw stack
Stop overpaying for Anthropic. We benchmark 30+ models on real OpenClaw tasks — calendar, email, coding, research, and tool use — to find the best model for every activity layer.
“Gemini 3.1 Flash Lite scored 96/100 at $0.016 total — the highest base score across all 36 models tested. A genuine surprise that reshapes the cost-performance frontier.”
— on Gemini 3.1 Flash Lite
“DeepSeek V3 quietly posted 89.4 base at $0.035 — beating Claude Opus 4 while costing 50x less. The dark horse of this benchmark cycle.”
— on DeepSeek V3
“GPT-4o Mini delivers Grade A performance at $0.009 total cost. For high-volume cron jobs and heartbeat tasks, nothing else comes close on cost-efficiency.”
— on GPT-4o Mini
“Claude Sonnet 4.6 scored 91.8 base — the highest Anthropic score, beating even Opus 4.6. More expensive isn't always better.”
— on Claude Sonnet 4.6
“Gemma 4 26B MoE hit 89.4 base at $0.011 — an open-weight model matching frontier closed models. The open-source gap is closing fast.”
— on Gemma 4 26B MoE
“GPT-OSS 120B scored 84.9 at $0.004 total. OpenAI's open-weight entry proves you don't need API keys to get production-grade tool calling.”
— on GPT-OSS 120B
OpenClaw doesn't use one model for everything. Different tasks demand different tradeoffs between intelligence, speed, and cost. The openclaw.json config file assigns a model to each activity layer.
The main conversational agent — handles complex reasoning, multi-step planning, tool orchestration, and nuanced user interactions. This is where frontier models like Claude Opus and GPT-5.4 shine. The brain needs to understand context deeply, chain tools correctly, and know when NOT to use a tool. Cost matters less here because interactions are lower volume but higher stakes.
Config: models.agents.main.primary
Scheduled background tasks — email digests, monitoring checks, data syncs, periodic summaries. These run frequently (every 30 min to daily) and need to be fast and cheap. A model that costs $15/M output tokens will bankrupt your cron budget. Look for models scoring 80+ at under $1/M. Speed tier bonuses in our rubric reward sub-second responses.
Config: models.agents.main.cron
Delegated workers for code generation, research synthesis, file operations, and complex multi-step tasks. Sub-agents run in isolated sessions and need strong tool-calling accuracy plus the ability to handle multi-file code generation and long chains of reasoning. Mid-tier pricing is acceptable since these are task-specific and don't run constantly.
Config: models.agents.main.subagent
The system's pulse — routing classification, health checks, quick status reads, and lightweight triage. Heartbeats fire every 30 minutes and need to be near-instant. The model reads a status file, makes a binary decision, and moves on. This is where $0.05/M models like GPT-5 Nano and Nemotron 3 Nano earn their keep. Every millisecond and every token counts.
Config: models.agents.main.heartbeat
TheChallenge.ai benchmarks AI models on real OpenClaw agent tasks — not academic puzzles. We test tool selection, parameter accuracy, parallel calling, schema compliance, and restraint using OpenClaw Bench. If you're running OpenClaw and wondering whether you really need Claude Opus, this is where you find out.
Every model is scored on 5 dimensions: Tool Selection (35pts), Param Accuracy (25pts), No Hallucination (20pts), Schema Compliance (10pts), and Restraint (10pts). Bonus points for parallel execution (+10), speed (+5), and cost efficiency (+5). Grades range from A+ to F.
Each model runs 38 tests across 8 categories: simple tool calls, multi-step chains, parallel execution, complex orchestration, no-tool restraint, schema compliance, error recovery, and ambiguous requests. All tests are generic — no user-specific data.
Powered by OpenClaw Bench • 0 models • Updated —