Writing

Supervised agents beat autonomy for legacy modernization

I've been thinking about the difference between generating code and shipping code, and Mistral's recent write-up on modernizing a Fortran 77 codebase to C++ puts it in sharp focus. The headline result is that 40,000 lines were translated, but that number only matters because of what surrounded it: a numerical parity harness and more than a hundred documentation agents working under explicit constraints.

The first sprint did not try to boil the ocean. It covered only the core of a 300,000-line system. That kind of scoping decision sounds obvious after the fact, but it rarely happens by default. Left to a general-purpose agent, the temptation is to wander across the codebase, touching files outside the agreed boundary. Human review gates kept the work honest—and they weren't an afterthought. They were built into the workflow so that each stage had to pass before the next one began.

For scientific or engineering code, the real requirement is not syntactic correctness. It is numerical equivalence. A rewrite that silently drifts from the original physics is a liability, not a delivery. A parity harness is to AI migration what a test suite is to refactoring. It turns a fuzzy goal—'modernize this'—into a measurable one: every translated routine must produce the same outputs, within tolerance, for the same inputs.

Parity harness first. Always.

That principle changes how you design the workflow. Instead of giving an agent a large goal and asking it to be autonomous, you structure the work into stages. You constrain the agents to a bounded scope. You measure every output against ground truth. Autonomy is not the goal; verified progress is.

The documentation agents in Mistral's case are another underappreciated detail. Over a hundred of them ran to extract and summarize the original system's behavior. That creates the ground truth the parity harness needs. Without that, you would be asking an AI to migrate code based on superficial reading, with no way to know if the physics survived.

The point of the harness is not to slow things down. It is to make failure visible early. If a translated routine drifts from the original outputs, the discrepancy shows up in the harness long before it can compound through a larger system. That is the same reason we write tests before refactoring: the test does not prevent every mistake, but it catches the ones that matter at the moment they happen.

In practice, this means three things.

• Structure the workflow into bounded stages.
• Constrain each agent to a specific scope and set of deliverables.
• Measure every translated unit against a numerical parity harness.

Most AI migration demos stop at a diff. They show you thousands of lines changed and call it success. That is generating code. Shipping code means the system still works for the people who depend on it. The parity harness is the line between those two outcomes. If you are planning a legacy modernization with AI agents, put the harness first. It will feel slower at the start, but it is the only way to trust the result.

Originally covered at mistral.ai ↗