Notes · harness internals

Two ways to make an LLM edit code: aider vs pi

The model isn't where edits get reliable; the harness is. aider forgives the parser, pi forgives the arguments. I read both repos to find exactly where each one absorbs a model's mistakes.

Jason · June 2026

A coding-agent harness is the few hundred lines that turn whatever the model emitted into a real change on disk, and keep working when the model gets it slightly wrong. The reliability you feel is invested there, not in the weights. aider (Python) and pi (TypeScript, from earendil-works) put that robustness in mirror-image places: aider in front of the model, in a forgiving parser; pi behind it, in forgiving arguments. The clearest way to see it is to read pi's edit path, because it's the one the surveys skip. Rombaut et al.'s scaffold taxonomy (arxiv.org/html/2604.03515v2) maps thirteen open-source agents but does not include pi; Fabian Hertwig's edit-mechanism survey (fabianhertwig.com) walks Codex, aider, OpenHands, RooCode, and Cursor, again without pi. So this is pi's edit path, read from source, against aider's.

pi: the edit is a tool call, the arguments are forgiving

pi has no text edit format and no grammar anywhere in its harness. Every edit is a native function call to a single edit tool whose schema is one object: a path and an array of { oldText, newText } replacements (packages/coding-agent/src/core/tools/edit.ts). The model calls it through the provider's real tool-calling channel; pi's OpenAI-completions provider hands the tool's TypeBox schema straight to the API as the function's parameters and reads structured tool_calls back out of the stream (packages/ai/src/providers/openai-completions.ts).

edit({
  path: "src/app.ts",
  edits: [
    { oldText: "const port = 3000", newText: "const port = 8080" }
  ]
})

So where does robustness live, if not in a parser? In the arguments, in two layers, and this is the part worth reading closely. First, before the tool runs, prepareEditArguments handles the case where a model sends the edits array as a JSON string instead of a real array, by parsing it back into an array; the source comment names the exact models observed doing this ("Some models (Opus 4.6, GLM-5.1) send edits as a JSON string instead of an array"). It also folds a legacy single {oldText, newText} shape into the array form. Underneath that, the generic validateToolArguments path (packages/ai/src/utils/validation.ts) runs schema-guided coercion, walking the JSON schema and converting, for instance, a stringified number back to a number, before validating. If validation still fails, the precise per-field error is what goes back to the model.

Second, the match itself is normalized, and this is where the public record is wrong. pi's own tool description says "exact text replacement," and independent comparisons have repeated that pi does exact string replacement rather than fuzzy matching; respan.ai's aider-vs-pi page is one such comparison (respan.ai/market-map/compare/aider-vs-pi-coding-agent). The source is more careful. fuzzyFindText (packages/coding-agent/src/core/tools/edit-diff.ts) tries an exact indexOf first; if that misses, it runs both the file and the oldText through normalizeForFuzzyMatch, which applies NFKC, strips trailing whitespace per line, and folds smart quotes, the various Unicode dashes (U+2010 through U+2015 plus U+2212 minus), and exotic spaces (NBSP, the U+2002 family, ideographic space) down to ASCII, then matches in that normalized space. That is a fuzzy-tolerance path, just a tightly scoped one: it forgives encoding noise, not edit distance.

What keeps it safe is a uniqueness contract. A helper counts occurrences, and zero matches or more than one each raise a distinct, instructive error ("Could not find the exact text..." or "Found N occurrences... please provide more context to make it unique"). Multiple edits in one call are matched against the original content, sorted, checked for overlap (overlapping edits raise rather than silently corrupt), then applied back-to-front so earlier offsets stay valid. That unique-match-or-error contract is, notably, the same pattern as Anthropic's str_replace for SWE-bench (anthropic.com/engineering/swe-bench-sonnet); pi's contribution is the normalization layer wrapped around it, not the contract itself.

pi's recovery loop falls out of this for free. A failed edit is just a tool result with isError set; the agent loop appends it like any other tool output, and the model sees the precise error next turn and corrects. There's no reflection counter or hand-built retry prompt, because in a tool-calling loop a bad edit and a good edit travel the same path. And the surrounding prompt is deliberately tiny, which is what makes the rest affordable: pi's authors describe a lean system prompt that advertises skills as just a name plus a one-line description and loads detail on demand (Mario Zechner, mariozechner.at; Armin Ronacher, lucumr.pocoo.org; a deeper breakdown from Tensorlake, tensorlake.ai). The lean prompt is what makes one universal tool-call format cheap enough for every model instead of tuning a contract per model.

aider: the edit is text, the parser is forgiving

aider makes the opposite bet. Its default editor, EditBlockCoder, asks the model to emit a SEARCH/REPLACE block as plain text in its reply (a filename, a fence, the lines to find, a divider, the replacement; full format at aider.chat/docs/more/edit-formats.html). There is no function-calling and no grammar; aider just parses whatever comes back, and the parser, find_original_update_blocks, is deliberately lenient: it tolerates marker-length drift, hunts up to three lines back to recover a misplaced filename, and carries a dedicated branch for one specific way DeepSeek Coder mangled its fences. It is a parser written by someone who has watched a lot of models almost get it right.

Applying a parsed block is its own cascade in replace_most_similar_chunk (aider/coders/editblock_coder.py): an exact line match, then a whitespace-tolerant match (models love to re-indent), then a handler for blocks where the model elided the middle with .... Here is the kind of detail you only get from a source read: the edit-distance fuzzy fallback, replace_closest_edit_distance, is in the file, but a bare return sits immediately above the call, so on the apply path it is unreachable. The same SequenceMatcher scoring still runs elsewhere to build the "did you mean these lines?" hint on failure. So the marketing-level summary ("aider falls back to fuzzy matching") describes code that, on the current apply path, never runs; the fuzzy logic survives only as a diagnostic.

When a block still doesn't match, aider raises a ValueError whose message is the next prompt: it names the failed SEARCH, shows the closest actual lines, and asks for a resend. That becomes self.reflected_message, fed back by the main loop, bounded at max_reflections = 3. And aider picks the edit format per model: models.py pins each known model to diff, udiff, or whole, and toggles whether examples ship as a system message or a fake prior exchange. The harness adapts its text contract to the model in front of it.

Same problem, mirror-image answers

Lined up, the two harnesses rhyme almost perfectly, inverted at every step; the axis the published taxonomies miss is the last row, where in the pipeline the robustness sits.

That last row is the dimension the surveys don't name. aider spends its effort before the model, deciding which text contract a given model can follow and then catching every way it fumbles that contract. pi spends its effort after the model, accepting one universal tool call and repairing the encoding on the way in. Forgive the parser, or forgive the arguments.

Is aider's bet aging well?

aider's public rationale for staying in text is that models are fluent in diffs from pretraining, whereas wrapping code in JSON forces error-prone escaping. It's a real argument, but worth dating honestly: the headline result behind it is a 2023 measurement on GPT-4 Turbo (aider.chat/docs/unified-diffs.html, benchmarks), and the academic finding often cited alongside it (Tam et al. 2024, arxiv.org/html/2408.02442v1) is about structured output hurting reasoning, not code edits. Treating "structured outputs are worse for editing" as a settled, current fact is the part that has aged.

The other side of the bet now has vendor momentum. OpenAI ships a native apply_patch tool, a structured edit primitive supported by the GPT-5.1 family (developers.openai.com/api/docs/guides/tools-apply-patch), and dedicated apply-models like Morph report strong numbers on exactly the diff-application task (morphllm.com). So this is a live question, not a retrospective: aider's text bet buys maximum portability, while pi's tool-call bet is increasingly the one the providers are building for.

Neither is obviously right; they answer slightly different questions. aider optimizes for "run against anything that emits tokens." pi optimizes for "stay small and clean on endpoints that behave, and repair the rest." Reading them back to back, against a taxonomy that left one of them out, is the clearest tutorial I've found in what a harness is actually for: not the model, but the deeply specific layer that turns a probabilistic text generator into something that reliably changes a file.

All claims here are from reading the repositories in June 2026: aider's editblock_coder.py and models.py; pi's edit.ts, edit-diff.ts, validation.ts, and openai-completions.ts. The symbols named (prepareEditArguments, fuzzyFindText, normalizeForFuzzyMatch, validateToolArguments, replace_most_similar_chunk) are the ones I verified at that commit.

Source read in June 2026. aider: github.com/aider-ai/aider. pi: github.com/earendil-works/pi.