Nothing ships because AI said so

Inside the Build · Part 3 of 4

By Miles Sellyn, VP of Partnerships

‍

Last Thursday I said we would get into what happens when your tools can generate more work than your team can properly evaluate. The day we do so is today.

Somewhere in the middle of building Community with beehiiv, one of our engineering leads wrote a line in Slack that rewired my brain chemistry.

(As an aside, this has happened to me more times in the last 10 months than any period in my life since ages 11-14, which was for entirely different reasons)

He was explaining a new rule around how we manage the SDLC (software development lifecycle). Every new ticket lands in triage first. Not the backlog, not in a sprint, but in triage. A human reads it, verifies it, and only then does it become work. The reason he gave was that most of the tickets are now AI generated, and some of them contain assumptions the AI made. He wanted to prevent the team building something just because the AI said so.

As of today, that is probably the biggest problem with AI assisted delivery in about fifteen words, and the fix he shipped was functionally an inbox lol

What you’ll find in today’s issue:

Why cheap generation creates an expensive new problem
The triage rule, and why it is more of a governance model than a workflow tweak
A three question filter for anything your tools generate

Making is $, judgement is $$$$

I have written about this from a few angles this year, but it really is the primary theme of building stuff in 2026. The cost of producing work has continued to collapse. The cost of judging whether the work deserves to exist has continued to inflate like a Paw Patrol birthday balloon.

A downstream manifestation of this: what it does to a backlog.

In practice, this ends up looking like an engineer describing a feature to a tool - and they then get back fourteen well formed tickets with acceptance criteria.

On the surface, they look great! They are specific, they are consistently formatted, they use the right vocabulary - the crowd gasps at such efficiency. But upon closer inspection: a few of them reference a decision nobody made.

Why? The AI needed a value for something the description did not specify, so it picked a sensible one, and now that sensible one makes its way into your product without it ever actually being made.

And the truth is, no one is going to catch that in sprint planning. The ticket looks exactly as legitimate as its neighbors because reviewing the details of every ticket usually happens while you’re writing it, not when you’re doing sprint planning.

None of this is an argument against the tools, because this project, and all of our projects, use them heavily and support us in shipping things on or ahead of schedule. The tools work, there isn’t really much debate about that anymore. But what is changing is where you need to load up the effort.

Why adding a triage step is effective

So, the inbox solution I mentioned before.

Every ticket, regardless of origin, starts in “triage”. Nothing goes from triage to the backlog without a human confirming that a real user needs it and that the assumptions inside it are ones the team actually made. Sprint planning pulls from the verified backlog only, and never from triage.

It is, as far as process goes, a pretty unglamorous and unsexy piece of process - but it does something quite important! It puts human judgment at the one point in the pipeline where it (as of right now) cannot be skipped.

If you put this at the end during review, when the work already exists and killing it feels wasteful, you’ll never end up not pushing it live. But at the front, killing it costs nothing, so you’re more likely to actually exercise your judgement.

It’s also important to point out what the rule does not do - it doesn’t slow generation down. Anyone can still produce forty tickets in an afternoon. It just means the forty tickets have not become commitments yet, so they aren’t guaranteed to make it into the product as bloat.

“Miles, I don’t code, why are you telling me this?” Oh reader, you’re so locked in and astute, I love you. I’m telling you this because I think there is a version of this for every function, and I suspect most people will need one within about a year.

Picture your marketing team. Somebody asks for landing page copy for a campaign and has nine variants back before they finish their next Slack message. They are all decent. They all use the right product names and the right tone. But wait! Two of them make a claim about what the product does that is slightly ahead of what the product currently does, because the tool inferred it from the roadmap deck it was handed as context, and the roadmap deck is aspirational, because roadmap decks are, you know, forward facing.

Nobody reads nine variants closely, because that is where we’re all at right now. Somebody picks one. It goes live with, at best, an exaggeration - at worst, a straight up lie.

That is the same failure as the ticket with the invented default applied to a different business unit, and the fix is the same - a checkpoint between generated by AI and committed to by a human.

A filter you should use tomorrow

Two questions, for any piece of work your tools hand you.

Can I name the user who needs this? Not a segment. A person, or a role, doing a specific thing on a specific day. If the ticket cannot survive that, the requirement came from a pattern in the training data rather than from your business, which means it adds basically nothing
Which of these decisions did we actually make? Read the ticket with a fine eye for embedded choices. Defaults, thresholds, states, copy. Every one of those is either a decision your team made or a decision the tool made on your behalf, and the title of the ticket will not tell you which - but the details of it will. Your job is to be able to flag each decision as something human-generated

Anything that clears both is something you actually decided, with the process made easier by AI. Everything else stays in the inbox, because it wasn’t something real that you agreed to implement in the first place.

That's it for today. If you found this valuable, forward it to whoever is currently drowning in a backlog that tripled in size without anyone hiring anybody (you don’t hire agents, we need to stop saying that).

Next Thursday is the last one in this series, and it is the receiptsssssss. What shipped, when, and why it mattered. I’ll also go 3/3 on promises!