👋 Hey, it’s Stephane. I help engineers become great engineering managers - whether you want to become one or are already leading a team.
Paid subscribers get 50 Notion Templates, The EM’s Field Guide, and access to the complete archive.
Two engineers I know tried the same AI coding tool the same month. Same model, same version, same company even.
One of them told me it pretty much one-shotted her feature. The other told me it was overhyped garbage that kept wrecking his codebase and he’d gone back to writing his feature by hand.
I believed both of them. Because the tool was never the variable.
The first engineer had spent a weekend setting it up - a context file at the root of the repo, a few notes per service about how that part of the system worked, the test command scoped so the agent didn’t run the whole suite on every change. The second one had installed it, pointed it at a million-line monorepo, and asked it to “fix the auth bug”. They were bound to get different results.
That gap is the whole story of AI coding right now. And most engineering managers are looking at the wrong half of it.
The model is not the thing you’re buying
When I built a SaaS with Claude Code, the lesson that stuck with me had nothing to do with how smart the model I was using was. It was how much the result depended on everything around the model - the context I gave it, the way I’d structured the work, the guardrails I set.
There’s a name for this now. Martin Fowler’s team at Thoughtworks calls it harness engineering: the idea that the scaffolding around a coding agent - the context files, the tooling, the way it searches and tests - shapes its output as much as the model itself. Anthropic makes the same point in their own writing on working in large codebases, where they say that the harness matters as much as the model.
That should change how you think.
Your team is going to judge these tools the way the second engineer did. They’ll try them on a real task, in a real codebase, with minimal setup. It’ll produce something mediocre at best. They’ll conclude the tool is overhyped and might stop using it if they were sceptical to begin with. The verdict will feel like it’s about the model. It will actually be about the absence of a harness.
The data backs this up from the other direction. The METR study found experienced developers were about 19% slower using AI on their own mature codebases - while believing they were 20% faster. The 2025 DORA report found AI amplifies whatever your team already is: it speeds up a healthy system and speeds up a broken one. In both cases the model wasn’t the deciding factor. The setup around it was.
That makes it your job, not your platform team’s
If the harness is what determines the result, then the harness needs an owner. And on most teams, nobody owns it.
Addy Osmani, who leads engineering on Chrome at Google, wrote a piece with a title that should be on a poster in every EM’s head: “Your AI coding agents need a manager”. His argument is that AI coding at scale stops being a prompting problem and becomes a management problem. The skills that make someone a good tech lead - setting context, defining the work clearly, reviewing output, knowing when to step in - are exactly the skills that make AI coding work.
The thing standing between your team and real gains from these tools is not a better model. It’s a management responsibility that currently has no owner on your team.
The instinct, when something needs adopting, is to mandate it. LeadDev did a whole piece on how AI coding mandates are driving developers to the brink. Push hard enough and you won’t get adoption - you’ll get resentment.
Owning the harness means someone makes the setup good enough that using the tool is the path of least resistance.
What good setup is actually protecting you from
This is the part that turns it from a tooling chore into a real management priority. Better code out of the agent is the obvious payoff. The bigger one is that the harness is your first line of defense against the three problems AI creates on a team.
Review capacity. This is the big one. When writing code gets cheap, reviewing it doesn’t. Faros AI looked at data from more than 10,000 developers and found that after AI adoption, pull request volume nearly doubled and review time climbed by 91%. I wrote before about how fast PRs can hide shallow understanding - AI pours fuel on exactly that fire. A senior engineer cannot meaningfully review a thousand lines that took five minutes to generate. The bottleneck has moved from coding to reviews from your most experienced people, and they’re the ones you can least afford to burn out.
Your juniors. AI is very good at the small, well-defined tickets that used to be how junior engineers learned the system. If the agent eats those, what’s left for a junior to grow on? This is a real problem, and good setup is part of the answer - the harness can be tuned to leave room for people to learn, not just to ship.
Trust. The Stack Overflow developer survey found that while 84% of developers now use AI tools, but trust in their accuracy fell to 29%. Two-thirds say the answers are “almost right, but not quite” - which is the most expensive kind of wrong, because it costs time to find. I’ve written about how AI is changing the way teams build trust. The harness is where you decide what the tool is allowed to do unsupervised and what it isn’t.
None of these three problems will magically get solved by a smarter model. All three are shaped by whether someone owns the setup.
And here’s the cost of not owning it. The team that skips the harness doesn’t get the upside of AI and avoid the downside. They get the worst of both worlds: more code than before, less understanding than before, seniors drowning in reviews, juniors with nothing to learn on, and a tool everyone blames for the mess.
The teams that own the harness get the opposite. The agent does the work that should be cheap, the humans spend their attention on the work that shouldn’t be, and the setup is what draws that line.
The question is who on your team owns the harness, what good setup actually looks like, and how you roll it out.
Here’s the playbook I’d use, the setup I’d own, and who I’d put in charge of it:
The playbook
Below is what “owning the harness” actually means in practice - the four layers of setup that matter, who owns them, the maintenance cadence it needs, and the how to roll it out without getting resentment back.