quanglee/ field notes
← field notes
23 min 8 live figures #ai#agents#benchmarks

The best agent harness is a while loop and a lot of upkeep

Every coding agent is a model inside a harness, the program that runs the loop, the tools and the context. This post measures how much a harness adds on Terminal-Bench, which ones lead in October 2026, what the top ones share inside and what they are built on. Claude Code and Codex lead every current board, and mini-swe-agent, a 190-line harness, comes within a few points of them.

  1. the loop and the parts around it
  2. terminal-bench 2.0
  3. effort, tries and models
  4. the board today
  5. the shared design
  6. frameworks or from scratch
  7. subscriptions and access
  8. my pick

Every coding agent you can install, Claude Code, Codex, Gemini CLI, Cursor or Droid, is a model inside a harness. The model is what the labs train. The harness is the program around it: it sends the model a request, runs the tools the model asks for, appends the results to the request and calls the model again, until the model answers without asking for a tool.

The loop fits in ten lines, and fig 1 runs it on a small bug fix.

Ten lines of pseudocode run a small coding task. The context starts with a system prompt (4k tokens), tool schemas (5k), AGENTS.md (1.5k) and the task (0.5k). The model asks for a search, reads src/dates.py, edits it, runs pytest (3,800 lines of output cut down to its head and tail), reads the test file and edits again. At 32.3k of a 40k window the context passes 80% and the harness compacts it to 13.5k, replacing the history with a summary. The model runs pytest again, 212 tests pass, and it replies without a tool call, which ends the loop.
fig 1Every harness in this post runs this loop. Each turn appends the model's reply and the tool output, and the stable prefix at the front stays the same so the provider can cache it. The task, the token counts and the 40k window are made up so that compaction happens on screen. The 80% threshold is the one Goose and Mistral Vibe use; Codex compacts at 90% and Gemini CLI at 50%1.

The rest of a harness decides what goes into each request, how a 3,800-line test log gets cut down, what happens when the window fills up and what the model can run without asking. Those choices move scores a lot. On the older Terminal-Bench 2.0 board, Claude Opus 4.6 scores 58.0% in Claude Code and 76.4% in Stanford’s Meta-Harness, on the same 89 tasks2.

Today the two lab harnesses, Claude Code and Codex, top every board that has current models, and mini-swe-agent, a 190-line harness, scores within a few points of them. The numbers below come from the Terminal-Bench leaderboards, Artificial Analysis, Vals, Datacurve, Princeton’s HAL, a dozen papers from 2026 and the source code of ten open-source harnesses, all read on 12 October 2026.

the loop and the parts around it

mini-swe-agent, from the Princeton and Stanford team behind SWE-bench and SWE-agent, describes itself as “just some 100 lines of python for the agent class” and has no tool except bash3. Its agent class is 190 lines today and it scored above 74% on SWE-bench Verified. Terminal-Bench’s reference agent, Terminus 2, is 2,101 lines4.

At the other end, Claude Code’s docs list about 45 tools5, and Anthropic has shipped about 296 releases of its 2.1 line since January6. Most of that work goes into the parts around the loop: permissions, sandboxing, subagents, memory files, hooks, and fixes for each new model.

terminal-bench 2.0

Terminal-Bench 2.0 is where harness makers competed. Anyone could submit a run, the board has 43 different agents, and Opus 4.6 and GPT-5.3-Codex each appear in nine of them2.

Terminal-Bench 2.0 scores, one row per model and one dot per harness. Claude Opus 4.6: Meta-Harness 76.4%, Capy 75.3, Terminus-KIRA 74.7, TongAgents 71.9, Droid 69.9, Crux 66.9, Mux 66.5, Terminus 2 62.9 (verified), Claude Code 58.0 (verified, last). GPT-5.5: NexAU-AHE 84.7, Capy 83.1, Codex CLI 82.2 (verified), clnkr 66.1. GPT-5.3-Codex: SageAgent 78.4 down to Terminus 2 64.7, with OpenAI's Simple Codex at 75.1. Gemini 3.1 Pro: TongAgents 80.2, Terminus-KIRA 74.8, Gemini CLI 61.4. Gemini 3 Pro: Ante 69.4 (verified), SageAgent 65.2, CodeBrain-1.5 62.2, II-Agent 61.8, Droid 61.1, Terminus 2 56.9 (verified), Letta Code 56.0, with no Google harness. Gemini 3 Flash: Junie CLI 64.3, Terminus 2 51.7, Gemini CLI 47.4. GPT-5.2: Droid 64.9, Codex CLI 62.9 (verified), Mux 60.7, Terminus 2 54.0. Claude Opus 4.5: Droid 63.1, Letta Code 59.1, Mux 58.4, Terminus 2 57.8, Goose 54.3, Claude Code 52.1, OpenHands 51.9, OpenCode 51.7. GPT-5, all verified: Codex CLI 49.6, OpenHands 43.8, Terminus 2 35.2, mini-swe-agent 33.9. Claude Sonnet 4.5: CAMEL-AI 46.5, Goose 43.1, Terminus 2 42.8, OpenHands 42.6, mini-swe-agent 42.5, Claude Code 40.1. Claude Haiku 4.5: Goose 35.5, mini-swe-agent 29.8, Terminus 2 28.3, Claude Code 27.5, OpenHands 13.9. Gemini 2.5 Pro, all verified: Terminus 2 32.6, mini-swe-agent 26.1, Gemini CLI 19.6, OpenHands 16.4. With only verified rows, Opus 4.6's spread falls from 18.4 to 4.9 points.
fig 2For Opus 4.6 the harness is worth 18 points, and Claude Code comes last of nine. Showing only verified rows removes most of the top scores. Terminal-Bench 2.0 leaderboard data2, 89 tasks, 95% intervals of about ±2 to 3 points. "Verified" is the board's own flag; the other rows are self-reported by the harness makers. Gemini 3 Pro has no Google harness on the board. Hover a dot for the harness.

Claude Code scored below the neutral Terminus 2 harness for every Claude model on this board, and came last of nine for Opus 4.6. Gemini CLI also scored below Terminus 2 for the two Gemini models that have both (Gemini 3 Flash and Gemini 2.5 Pro). OpenAI’s harness is the exception: it beats Terminus 2 for every GPT model that has both, by up to 14 points (GPT-5: 49.6% against 35.2%).

Most of the top rows on this board are self-reported. Over all 27 models on the board that have two or more harnesses, the median gap between a model’s best and worst harness is 11.7 points, and 5.4 when only verified runs count. Meta-Harness searched for its harness on the same 89 tasks it reports, with no held-out set7. Terminal-Bench removed two submissions for cheating, one with encrypted solutions inside the agent binary and one that uploaded the test folder, and rescored ForgeCode’s trials to zero where it had “curled solutions from the internet” into its AGENTS.md8.

On Terminal-Bench 2.1, a revision that fixed 28 tasks, Claude Code beats Terminus 2: Fable 5 scores 83.8% in Claude Code and 80.5% in Terminus 22.

effort, tries and models

Fig 3 sets a harness switch next to the other things that change an agent’s score: the reasoning effort, the number of tries and the model itself.

Points gained on Terminal-Bench. Changing the harness: median gap between a model's best and worst verified harness on 2.0, over 22 models, 5.4 points; automated harness search for GPT-5.4 (AHE), 7.3; the same median over all rows and 27 models, 11.7; Claude Code to Meta-Harness for Opus 4.6, 18.4. Raising reasoning effort on 4.0: GPT-6 Astra in Codex from low to max, 7.6; Opus 5 in Claude Code from low to xhigh, 19.1. Counting a pass in any of five tries instead of one, Opus 5.5 on 4.0: 13.9. Moving up a model generation in a fixed harness: Opus 4.1 to Opus 4.5 in Terminus 2, 19.8; Gemini 2.5 Pro to Gemini 3 Pro in Terminus 2, 24.3; Opus 4.8 to Opus 5 in Claude Code on 4.0, 28.2.
fig 3For the median model, the gap between its best and worst verified harness is about five points. A new model generation is worth twenty or more. Terminal-Bench 2.0 and 4.0 leaderboard data2, and the AHE paper for automated search9. Each bar is the gain in points between two runs. The two medians are over every model on the 2.0 board with two or more harnesses, not only the twelve in fig 2.

Going from Opus 4.8 to Opus 5 in the same harness was worth 28 points (23.6% to 51.8%), more than five times the median gap between a model’s best and worst verified harness. Raising Opus 5’s effort from low to xhigh in Claude Code is worth 19. The best harness on the 2.0 board can match a model generation, but only in unverified, benchmark-tuned rows. OpenAI says that keeping the model’s reasoning between turns and compacting context in the Codex harness took GPT-5.6 Sol from 13.3% to 38.3% on ARC-AGI-310.

Studies that control for everything else find smaller effects. Zhang and colleagues ran three models in three harnesses on SWE-bench Verified: the harness moved each model by 8.5 to 13 points and the model choice by 2.5 to 5, because the three models were close peers11. Vats and Golev found gaps of 0 to 8 points between Goose, OpenCode and OpenHands, with confidence intervals that include zero for all but the largest12. Arjmandi found no average advantage for a lab’s own harness (Claude Code for Opus 4.8, the Codex SDK for GPT-5.5) over a neutral one, −1.25 and +1.25 points13. METR couldn’t separate Claude Code and Codex from its own scaffolds either14.

The harness matters more for cost. The same Vats and Golev runs spent up to 40 times more tokens per solved task in one harness than in another12, and MIT FutureTech, working from Princeton’s HAL data, found that efficiency on a benchmark can vary 100 times between scaffolds, “roughly two years of model progress”15. On HAL’s CORE-Bench Hard, Opus 4.5 scores 77.8% in Claude Code for $87 and 33.3% in HAL’s generalist agent for $12716. A weak harness can also break a good model: o3 solves none of SWE-bench Verified Mini in HAL’s generalist agent and 46% in SWE-agent.

the board today

Terminal-Bench 4.0 came out in August with 66 harder tasks, run five times each. Only labs have submitted to it so far: Claude Code, Codex, xAI’s Grok Build and Meta’s Muse Code, with Gemini running in mini-swe-agent2. Claude Code with Opus 5.5 leads at 64.85%, and Codex rows with GPT-6 Astra and GPT-6.1 Sol tie around 58%.

Artificial Analysis runs every harness itself for its coding agent index, which averages DeepSWE, Terminal-Bench 4.0 and SWE-Atlas17. Claude Code with Sonnet 5.5 leads at 68.4. Cognition’s Devin Fusion CLI, which pairs Claude Fable 5.1 with Cognition’s own SWE-2, is the only harness among the index’s ten best rows not made by Anthropic, OpenAI or Google. OpenCode, the best independent open-source harness, scores 53.6 with GLM-5.3.

Best score per harness. Artificial Analysis coding agent index: Claude Code 68.4 (Sonnet 5.5 max), Antigravity CLI 63.8 (Gemini 4 Argon high), Codex 62.9 (GPT-6.1 Sol xhigh), Devin Fusion CLI 61.7 (Fable 5.1 with SWE-2), Grok Build 56.3, Muse Code 54.3, OpenCode 53.6 (GLM-5.3), Kimi Code CLI 51.9 (Kimi K3). Terminal-Bench 4.0: Claude Code 64.85% (Opus 5.5 max), Codex 58.18% (GPT-6 Astra max), Grok Build 37.58% (Grok 4.7), mini-swe-agent 19.09% (Gemini 3.8 Flash), Muse Code 14.55%.
fig 4The lab harnesses take the top three places on both boards. Artificial Analysis coding agent index v1.5, data of 11 October17, and the official Terminal-Bench 4.0 leaderboard2, best row per harness. Artificial Analysis marks the Antigravity CLI row as not publicly available.

Terminal-Bench only lists the runs people submit. Cursor, Droid, Warp, Junie and OpenHands all submitted to version 2.0, and Cursor’s CLI also appears on 2.1 and 3.0, but on 4.0 only the labs have submitted so far2. Artificial Analysis picks the harnesses it runs, and its current index includes none of them17. Mistral Vibe has no row on any Terminal-Bench version. So the independent harnesses that beat the labs on 2.0 haven’t been tested on the new tasks or with the new models.

OpenCode shows how far popularity and benchmarks have drifted apart. It is the most-starred coding harness on GitHub, with 212,864 stars against 150,341 for Claude Code’s repo and 128,786 for Codex18. It was downloaded 11.8 million times from npm last month19, and 13.6% of the 12,255 developers who answered Stack Overflow’s 2026 question on coding agents had used it, seventh overall and first among tools not tied to a model lab20. Its only Terminal-Bench rows are on 2.0, where it scores 51.7% with Opus 4.5 against 52.1% for Claude Code2. MiniMax’s model card puts M2.5 at 76.1% on SWE-bench Verified in OpenCode and 80.2% in Claude Code21. Artificial Analysis has run it only with GLM-5.3, and nobody has submitted it to Terminal-Bench 4.0.

Vals and Datacurve run every model in mini-swe-agent, so their numbers can be set next to each model in its vendor’s harness.

Each model in its vendor's harness and in mini-swe-agent. Terminal-Bench 4.0: Claude Opus 5.5 in Claude Code 64.85% vs 65.15% in mini-swe-agent; Sonnet 5.5 61.82 vs 64.14; Fable 5.1 57.88 vs 58.08; GPT-6 Astra in Codex 58.18 vs 59.60; GPT-6.1 Sol 58.18 vs 55.05; GPT-6 Sol 49.39 vs 44.44; GLM-5.3 in Claude Code 41.82 vs 38.89; Grok 4.7 in Grok Build 37.58 vs 28.79; Qwen3.8 Max in Claude Code 26.97 vs 34.34. DeepSWE: GPT-5.6 Sol in Codex 72.3 vs 73; GPT-6 Astra 67.6 vs 74; Kimi K3 in Kimi Code CLI 68.4 vs 69; Grok 4.6 in Grok Build 64.9 vs 67; Claude Opus 5 in Claude Code 62.5 vs 74; GLM-5.3 in OpenCode 61.4 vs 69.
fig 5On Terminal-Bench 4.0 mini-swe-agent lands within about three points of the vendor's for six of nine models. On DeepSWE it scores higher for all six. Terminal-Bench 4.0: official leaderboard2 against Vals22. DeepSWE v1.1: Artificial Analysis's runs of each vendor CLI17 against Datacurve's mini-swe-agent runs23. Different teams ran the two sides, with different attempt counts and effort settings (Grok 4.6 is xhigh against medium), so a few points either way is noise.

Grok Build is worth about 9 points to Grok 4.7, and Qwen3.8 Max loses 7 by running in Claude Code instead of a neutral harness. Most models sit closer. The Terminal-Bench team ran the same comparison on their Harbor-Index tasks and found that harness choice “rarely” made a statistically significant difference, though it changed which tasks got solved24.

the shared design

Under the product differences, the open-source harnesses share most of their design, and the docs of the closed ones describe the same parts.

An agent harness drawn as layers around the model: loop, context, tools, output limits, compaction, subagents, sandbox and extensions, with how five harnesses build each one. Loop: Claude Code gathers context, acts and verifies, and Esc steers it mid-turn; Codex CLI loops until a reply has no tool call, and stop hooks can resume it; Gemini CLI allows 100 turns per run and has a loop detector; OpenCode has an optional step limit, after which tools are switched off; mini-swe-agent runs while True with step, cost and time limits. Context: Claude Code reads CLAUDE.md and loads MCP tool schemas only when needed; Codex reads AGENTS.md up to 32 KiB and only appends new items; Gemini CLI reads GEMINI.md, also for each folder it touches; OpenCode reads AGENTS.md and adds reminders at the tail to keep the cache; mini-swe-agent sends the system prompt, the task and the history. Tools: Claude Code has about 45 tools, edits by swapping an exact string and searches with grep in bash; Codex CLI has a shell and apply_patch and searches with rg; Gemini CLI has 27 tools with replace edits and glob and grep tools; OpenCode has 12 tools and picks string edits or patches per model; mini-swe-agent has only bash. Output limits: Claude Code about 30k characters for Bash, the rest saved to a file; Codex 10k tokens, head and tail; Gemini CLI 40k characters; OpenCode 2,000 lines or 50 KiB; mini-swe-agent the first and last 5k characters. Compaction: Claude Code near the limit, clearing old tool output first; Codex at 90% of the window; Gemini CLI at 50%, keeping the latest 30%; OpenCode keeps max(10%, 16k) free; mini-swe-agent never compacts. Subagents: Claude Code has Explore and Plan agents, worktrees and a depth of 3; Codex spawns up to 6 threads; Gemini CLI has codebase_investigator and other built-in agents; OpenCode has explore and general subagents; mini-swe-agent has none. Sandbox: Claude Code uses seatbelt or bubblewrap and its auto mode asks a classifier; Codex uses seatbelt or landlock with 3 sandbox and 4 approval modes; Gemini CLI uses docker, podman or seatbelt with plan and yolo modes; OpenCode has allow, ask and deny rules and no sandbox in its core; mini-swe-agent runs in docker or locally with confirm or yolo. Extensions: MCP, hooks, skills and plugins in Claude Code and Codex (plus an app server); MCP, hooks, skills, extensions and ACP in Gemini CLI; MCP, plugins, skills and ACP in OpenCode; a YAML config file in mini-swe-agent.
fig 6Pick a layer to compare five harnesses. mini-swe-agent shows how much of the stack is optional. Read from the source of Codex, Gemini CLI, OpenCode and mini-swe-agent1 and from Claude Code's docs5, 12 October 2026. Claude Code doesn't publish its compaction threshold.

The edit tool follows the model’s training. Cursor’s team writes that “OpenAI’s models are trained to edit files using a patch-based format” while “Anthropic’s models are trained on string replacement”, and its harness gives each model the format it knows25. Codex constrains its apply_patch tool with a grammar; Claude Code’s Edit replaces one exact, unique string.

The context is ordered for the cache. The system prompt, tool schemas and memory files come first and don’t change, and new items are only appended. The team behind Manus called the cache hit rate “the single most important metric” for an agent26, and OpenAI’s write-up of the Codex loop says “cache hits are only possible for exact prefix matches”27. Long tool output gets cut to its head and tail, with the full text saved to a file the agent can grep later. When the window fills, a summarizer writes a handoff note and the old history is dropped. Codex’s and Vibe’s compaction prompts both open with “CONTEXT CHECKPOINT COMPACTION”.

The harness also runs the multi-agent side. Claude Code ships read-only Explore and Plan subagents, runs subagents in the background or in their own git worktree, and lets them nest three levels deep28. Codex can spawn up to six agent threads, Gemini CLI has built-in subagents and can call remote agents over A2A, and Cline runs teams of agents that message each other through a shared mailbox1. The pattern is the same everywhere: a subagent gets a fresh context window, does its part and hands back a summary of “often 1,000-2,000 tokens”, in Anthropic’s words, so the main agent’s context stays small29.

Harnesses also lose parts as models improve. Claude Code no longer ships separate Glob and Grep tools on macOS and Linux, and searches with find and grep in Bash5. Amp, the coding agent Sourcegraph spun out as its own company in December 202530, removed its to-do tool in January because Opus 4.5 no longer needed it31, replaced compaction with a “handoff” and then brought compaction back32. Anthropic says “every component in a harness encodes an assumption about what the model can’t do on its own”33, and the context resets it built for Sonnet 4.5 became dead weight with Opus 4.534.

frameworks or from scratch

None of the leading coding harnesses is built on an agent framework. Gemini CLI doesn’t use Google’s own ADK; its core depends on the plain Gemini client35. The independent open-source harnesses write their own loop as well and borrow only a model client: LiteLLM for Aider, mini-swe-agent, Terminus 2 and OpenHands, and the Vercel AI SDK’s provider packages for Cline and OpenCode.

A lineage diagram. Claude Code is shipped as a library, the Claude Agent SDK, and Codex CLI as the Codex SDK. Qwen Code is a fork of Gemini CLI. Terminus 2 was forked into Terminus-KIRA, which Meta-Harness's automated search started from. SWE-agent was rewritten as mini-swe-agent. OpenClaw, Letta Code and Kimi Code reuse parts of pi. NexAU-AHE is an automated search over the NexAU harness. Deep Agents is built on LangGraph, and Deep Agents Code on Deep Agents. The original Go version of OpenCode continues as Charm's Crush. Below, the borrowed model clients: LiteLLM under Aider, mini-swe-agent, Terminus 2 and OpenHands; AI SDK providers under Cline and OpenCode; pi-ai under Letta Code; and their own clients for Claude Code, Codex, Gemini CLI, Mistral Vibe and Kimi Code.
fig 7The harnesses build on each other, and only LangChain's own coding agent sits on a framework. From each project's README and dependency files, read 12 October 202635.

What people build on now is a lab’s harness shipped as a library. The Claude Agent SDK bundles the Claude Code CLI and drives it from Python or TypeScript36. It was downloaded 51 million times from npm and 30 million from PyPI in the last month, against 9 and 12 million for the OpenAI Agents SDK37. The Codex SDK spawns the Codex CLI the same way, and Cursor, Cline and OpenHands now ship their harnesses as SDKs too. OpenCode has its own history of forks: it started as a Go project, and when its original author joined Charm in 2025 that code became Crush, while the SST team kept the name and rewrote OpenCode in TypeScript38.

Outside coding agents, many teams do build on frameworks. Stack Overflow’s 2026 survey asked developers which agent frameworks they had used in the past year, and of the 4,400 who answered, 30.2% named LangChain, 23.8% the OpenAI Agents SDK, 18.8% LangGraph and 14.0% Google’s ADK20. LangChain was at 32.9% the year before, CrewAI fell from 7.5% to 5.5% and AutoGen from 12.0% to 4.5%, while LangGraph rose from 16.2%39.

Two views of agent framework popularity. Stack Overflow 2026, share of 4,400 developers who had used each in the past year: LangChain 30.2% (32.9% in 2025), OpenAI Agents SDK 23.8%, Terminal Use 18.9%, LangGraph 18.8% (16.2%), Llama Stack 17.4%, Google ADK 14.0%, Hermes Agent 10.2%, Vercel AI SDK 9.0%, Pydantic AI 8.9%, CrewAI 5.5% (7.5%), Semantic Kernel 5.0% (6.0%), AutoGen 4.5% (12.0%). PyPI downloads in the 30 days to 11 October 2026: langchain 170.5 million, of which 36.8% were the current 1.x versions; langgraph 44.8 million; strands-agents 37.1 million; claude-agent-sdk 29.8 million; openai-agents 12.4 million; google-adk 10.1 million; pydantic-ai 5.4 million; crewai 2.4 million; autogen-agentchat 0.4 million.
fig 8LangChain leads both views, but only about a third of its downloads are versions released in the last year. Stack Overflow Developer Survey 2026 and 20252039, multi-select, so the bars don't add up to 100%. The 2026 list had no Claude Agent SDK and no "I wrote my own loop" option, and changed options since 2025. PyPI counts from pypistats and version shares from pepy.tech40; downloads include CI runs and packages installed as dependencies.

Downloads are a noisier signal. LangChain was installed 170 million times from PyPI last month, more than any other agent library, but 33% of those downloads were 2023 releases and only 37% the current 1.x versions40. Since April, downloads of the Claude Agent SDK grew 2.7 times and of AWS’s Strands 6.7 times, while CrewAI fell to a third and AutoGen to about a quarter. LangChain’s own 1.0 release in October 2025 cut the library back to an agent loop, after “consistent feedback” that “LangChain’s abstractions were sometimes too heavy”41.

Production agents look different again. The Berkeley-led “Measuring Agents in Production” study interviewed 20 teams running agents in production: 17 built “custom in-house implementations with direct API calls”, and two more had started on CrewAI and moved to their own code before shipping42. The teams gave three reasons: frameworks didn’t fit their own infrastructure, “core agent loops are straightforward to implement directly”, and some security rules ban external libraries. The same paper’s survey of 29 deployed agents found 61% on a framework, so the split isn’t settled, and nobody has compared framework-built and hand-written agents on the same tasks.

The evidence points to prototyping with whatever gets you there fastest, then shipping a single agent on a small loop you own, or on a lab’s SDK if you’re committed to that lab’s models. A graph framework like LangGraph is worth it when the job needs state that survives long runs or steps where a person reviews the work, which is where its published production stories sit43. Anthropic’s guide says the same: “the most successful implementations use simple, composable patterns rather than complex frameworks”44.

subscriptions and access

The labs now fight over which harness may use their models on a subscription. Anthropic blocked third-party clients that impersonated Claude Code in January45, barred subscription use in third-party harnesses such as OpenClaw in April, then in May allowed it again against a separate credit for programmatic use46. That change was paused on 15 June, the day it was due47. Google moved its paid-plan and free users from the open-source Gemini CLI to Antigravity CLI, cutting them off on 18 June48, and The Register reports the new CLI is not open source49.

OpenCode shows what this means for a harness that runs any model. It is built by Anomaly, the company behind the SST framework38, and many of its users signed in with Claude Pro and Max accounts. The day after Anthropic’s January block, OpenCode 1.1.11 added sign-in with ChatGPT and Codex accounts50. In March a pull request titled “anthropic legal requests” removed Claude sign-in from the code51, and Claude models now need an API key there.

Claude Code itself is closed source: its public GitHub repository holds the changelog, issues and plugins, and its license reads “All rights reserved”52. OpenAI went the other way. It publishes the Codex CLI, app server and SDK as open source10. One part stays closed: with OpenAI’s own models, Codex sends the history to OpenAI’s servers for compaction and gets back an encrypted item it can only pass along, while with other models it summarizes locally with an open prompt27. OpenAI also added “Sign in with ChatGPT” so third-party apps can use a ChatGPT plan53.

Meanwhile other labs benchmark their models in Claude Code. Z.ai evaluated GLM-5.3 “in Claude Code 2.1.207” on most of its agent benchmarks54, and its docs show how to point Claude Code at Z.ai’s endpoint55. MiniMax and Qwen report results in Claude Code as well.

my pick

JetBrains surveyed more than 15,000 developers between May and July: 39% use Claude Code at work, up from 18% in January, and 16% use Codex, up from 3%. GitHub Copilot fell to 21% and Cursor to 12%56.

For a closed stack I’d pick Claude Code with Opus 5.5 or Sonnet 5.5, which lead both current boards. Codex with GPT-6.1 Sol scores 6.7 points lower on Terminal-Bench 4.0, and its full run cost $634 against $4.7k for Opus 5.5 in Claude Code2.

For a fully open stack, with an open-source harness and open weights, I’d pick OpenCode with Z.ai’s GLM-5.3. It is the best open pair on Artificial Analysis’s index at 53.6, ahead of Moonshot’s Kimi Code with Kimi K3 at 51.9 and Codex with DeepSeek V4 Pro at 43.1, and 14.8 points behind Claude Code’s best17.

sources
  1. Source code of Codex, Gemini CLI, Goose, Cline, opencode, mini-swe-agent and Mistral Vibe on GitHub (read 12 October 2026)
  2. Terminal-Bench leaderboards, versions 2.0, 2.1 and 4.0 (read 12 October 2026)
  3. SWE-agent team, mini-swe-agent on GitHub (read 12 October 2026)
  4. Harbor, Terminus 2 source (read 12 October 2026)
  5. Claude Code docs, tools reference (read 12 October 2026)
  6. Anthropic, Claude Code changelog (read 12 October 2026)
  7. Lee, Nair, Zhang, Lee, Khattab and Finn, Meta-Harness, arXiv 2603.28052 (30 March 2026)
  8. Terminal-Bench, Leaderboard integrity update (19 April 2026)
  9. Lin et al., Agentic Harness Engineering, arXiv 2604.25850 (April 2026)
  10. OpenAI, Codex as a platform: build on the open agent harness (August 2026)
  11. Zhang et al., Stop Comparing LLM Agents Without Disclosing the Harness, arXiv 2605.23950 (7 May 2026)
  12. Vats and Golev, The Scaffold Effect in Coding Agents, arXiv 2607.22585 (2026)
  13. Arjmandi, native and neutral harnesses on a contamination-controlled suite, arXiv 2609.11987 (8 September 2026)
  14. METR, Measuring time horizon using Claude Code and Codex (13 February 2026)
  15. Gundlach, Brown, Lynch and Thompson (MIT FutureTech), Just a wrapper? How much do scaffolds matter? (26 June 2026)
  16. Princeton HAL, CORE-Bench Hard and SWE-bench Verified Mini leaderboards (read 12 October 2026)
  17. Artificial Analysis, Coding Agent Index v1.5 (data of 11 October 2026)
  18. Anomaly, opencode on GitHub, and the Claude Code and Codex repositories (stars read 12 October 2026)
  19. npm, opencode-ai downloads, 12 September to 11 October 2026
  20. Stack Overflow, Developer Survey 2026, AI section (6 October 2026)
  21. MiniMax, MiniMax-M2.5 model card (12 February 2026)
  22. Vals AI, Terminal-Bench 4.0 (updated 7 October 2026)
  23. Datacurve, DeepSWE v1.1 leaderboard (updated 22 September 2026)
  24. Terminal-Bench, Harbor-Index 1.0 (read 12 October 2026)
  25. Cursor, Continually improving our agent harness (30 April 2026)
  26. Manus, Context engineering for AI agents: lessons from building Manus (2025)
  27. OpenAI, Unrolling the Codex agent loop (23 January 2026)
  28. Claude Code docs, subagents (read 12 October 2026)
  29. Anthropic, Effective context engineering for AI agents (29 September 2025)
  30. Amp, Amp Inc. (December 2025)
  31. Amp, Todos are done (12 January 2026)
  32. Amp, Neo (6 May 2026)
  33. Anthropic, Harness design for long-running apps (24 March 2026)
  34. Anthropic, Scaling Managed Agents: decoupling the brain from the hands (8 April 2026)
  35. Gemini CLI, packages/core/package.json, and the other projects' README and dependency files (read 12 October 2026)
  36. Anthropic, claude-agent-sdk-python README (read 12 October 2026)
  37. npm and PyPI monthly download counts, 12 September to 11 October 2026
  38. Tech Funding News, OpenCode: the background story on the most popular open-source coding agent (13 January 2026)
  39. Stack Overflow, Developer Survey 2025, AI section (July 2025)
  40. pypistats and pepy.tech download counts and version shares, 30 days to 11 October 2026
  41. LangChain, LangChain and LangGraph 1.0 (October 2025)
  42. Pan et al., Measuring Agents in Production, arXiv 2512.04123, ICML 2026 (v4, 4 June 2026)
  43. LangChain, Built with LangGraph (read 12 October 2026)
  44. Anthropic, Building effective agents (19 December 2024)
  45. VentureBeat, Anthropic cracks down on unauthorized Claude usage by third-party harnesses (January 2026)
  46. Techstrong.ai, Anthropic reverses third-party agent ban, introduces Agent SDK credit tier (May 2026)
  47. DevOps.com, Anthropic hits pause on Claude Agent SDK billing change, for now (18 June 2026)
  48. Google for Developers, An important update: transitioning Gemini CLI to Antigravity CLI (19 May 2026)
  49. The Register, Bye bye Gemini CLI: Google nudges devs toward Antigravity (20 May 2026)
  50. OpenCode, v1.1.11 release notes (10 January 2026)
  51. OpenCode, pull request #18186, anthropic legal requests (merged 19 March 2026)
  52. Anthropic, claude-code repository license (read 12 October 2026)
  53. The New Stack, Sign in with ChatGPT (2026)
  54. Z.ai, GLM-5.3 model card (25 August 2026)
  55. Z.ai docs, Claude Code (read 12 October 2026)
  56. JetBrains Research, AI coding agent adoption in 2026 (August 2026)