👋 Hey, it’s Stephane. I help engineers become great engineering managers - whether you want to become one or are already leading a team.
Paid subscribers get 50 Notion Templates, The EM’s Field Guide, and access to the complete archive.
You see a PR authored by your PM.. It’s 700 LOC and touching on 3 services. They picked up one of the customer complains and thought they could fix it by just clauding their way into a solution.
Now, someone from your team needs to review it and decide whether it’s safe to merge.. and deploy it? but what if something’s wrong with it? would whoever approves it now be responsible for fixing that?
At Shopify, the company’s internal AI agent, River, exists in Slack, and employees ask it to read code, write code, open pull requests, and query production data. It’s super popular as well! In a recent month 5,938 people used it across 4,450 channels.
About one in eight merged pull requests in Shopify’s main codebase started that way.
Shopify CEO gave that pattern a name
Tobias Lütke calls this a “slop grenade”.
Someone generates something with AI, definitely doesn’t understand it, bearly reads it, and passes it on to a colleague. The time they saved becomes someone else’s work.
When someone that deep into AI-assisted development says AI-generated output that is not well understood is a problem, it’s probably something worth paying attention to.
The problem isn’t the output
A “slop grenade” doesn’t only mean bad AI-generated code.
It’s code that someone created, didn’t properly understand it and then handed to a reviewer to deal with.
If the person who submitted it can’t fully explain the decisions behind it - there’s a real problem.
When that happens, the reviewer needs to do much more than reviewing the code. They have to work out what parts can be trusted in the PR (AI can sound very convincing even when it’s wrong) and potentially also teach the author what their own code does.
Sounds to me like a complete recipe for disaster.
This doesn’t only happen with code
Pull requests and code are only very obvious examples.
The same thing happens when someone generates any type of document really - like an ADR, an RFC, an incident report, meeting notes, acceptance criteria, Slack summaries, a performance review.
In each of these, creating the artefact became easier than it used to be. Reviewing them became harder because reviewers have to decide whether any of it is actual thinking from the author now.
Own it fully!!!!
If you own an artefact, you’re responsible for what it says, whether you wrote all of it yourself or not.
Before you open up a PR (or any document that needs a review) you should understand it well enough to stand behind what’s in it.
If someone questions any part in it, you should be able to explain why you made that decision. You can’t say “AI suggested doing that”.
If there’s a mistake, it’s yours to own and correct. And if you don’t understand part of what your AI produced, the work isn’t ready to be shared with others yet.
Attention is a scarce resource
AI made it much easier to create things. It didn’t make it easier to decide whether those things are of high quality, correct, or ok to ship.
That attention was always limited. The amount of work waiting to be reviewed though, now is growing much faster than the number of people available to review it.
See you in the next one,
~ Stephane