Agents Maintaining Agents: Notes from the Field
What it actually looks like when a non-engineering ops team lets AI propose, evaluate, and ship changes to its own agents.
Most of the AI-in-GTM content out there is about agents doing work. Almost none of it is about agents maintaining the systems that do the work — which is the part that actually determines whether any of this survives past the demo.
Here’s the pattern that’s held up for us:
- Version everything. Every agent, prompt, and eval lives in a repo, not a UI-only builder. If it’s not in git, it doesn’t exist.
- Evals before merges. No change to a production agent ships without a passing eval run. This is the single highest-leverage habit for a non-engineering team.
- Small, reviewable diffs. Agents propose changes as diffs a human can actually read, not full rewrites.
None of this is exotic. It’s just discipline that most ops teams haven’t had a reason to build until now — because until now, “the system” wasn’t something you could hand off to be edited by something else.
Where it breaks
The failure mode isn’t the agent writing bad code. It’s a non-engineer approving a change that looks reasonable but silently changes behavior in a case nobody tested for. Evals catch this if — and only if — the eval set actually covers the cases that matter, which means someone still has to do the unglamorous work of writing good test cases.