Product Sense Now Means Knowing How Your Model Fails
The old product sense was empathy for users. The new one is fluency in failure. Meta added it to the PM interview loop. Here's what it actually is and how to build it.
Table of contents
A friend who interviews PMs at Meta told me they'd quietly added a new dimension to the loop: "Product Sense with AI." I asked what they were actually probing for. Her answer stuck with me.
"Old product sense was: can you tell me what users want. The new one is: can you tell me how this thing breaks."
That's not a tweak to the rubric. It's a different muscle, and a lot of very good PMs haven't started training it yet.
What product sense used to mean
For twenty years, product sense meant taste plus empathy. Julie Zhuo wrote beautifully about it as the ability to consistently make good product decisions without much data — a gut, built from watching thousands of people use software. You could look at a flow and feel where someone would get stuck. You knew which feature would land and which was a vanity project.
That skill still matters. It's just no longer sufficient, because the thing underneath you stopped being predictable.
When the system was deterministic, "how does it behave" had one answer, and you authored it. You designed the happy path, handled the error states, shipped. Behavior was something you wrote. Now behavior is something you discover, because the model does things you didn't specify and can't fully foresee. Empathy for the user is still table stakes. The new requirement is a working theory of the machine.
The new product sense, defined
Here's the cleanest framing I've found, from Marily Nika: every AI feature has a failure signature — the specific pattern of breakdowns it reliably falls into when the world gets messy. New product sense is the ability to predict that signature before your users find it.
AI product sense is the ability to anticipate how a model behaves under ambiguity, where users will lose trust when it breaks, and what its failures cost at scale — before shipping.
Concretely, it's being able to answer four questions about a feature you've never shipped:
- How does it behave when the input is ambiguous, not the clean demo case?
- What does the user actually experience at the moment it gets something wrong?
- Where is trust earned, and where is it lost for good?
- How do the costs and the failure rate change when usage goes up 100x?
An old-sense PM can sketch the feature. A new-sense PM can sketch the feature and tell you the three ways it'll embarrass you in production.
Why this is the part that gets tested
Because it's the rare half. Plenty of people can prompt a model into a flattering demo. Almost nobody can look at that same demo and say, calmly, "here's where this falls apart, and here's what the user sees when it does."
That second person is who you want owning an AI product. That's why it's showing up in interview loops — anyone can be optimistic about a model on its best input; the signal is whether you can be specific about its worst.
There's a useful borrow here from engineering. Netflix popularized chaos engineering — deliberately breaking your own systems in production to learn how they fail. New product sense is chaos engineering pointed at behavior instead of infrastructure. And it rhymes with Gary Klein's premortem: you assume the thing has already failed, then work backward to how. Pessimism, used as a tool, is the whole craft.
You can't fake this with a framework. It's reps. The fastest way to build it is to push a model into its failure modes on purpose, over and over, until you can call the signature on sight.
How to start building it this week
Pick one AI feature — yours, or a competitor's. Before you touch it, write down the three ways you predict it'll fail. Then spend twenty minutes actively trying to break it: ambiguous inputs, contradictory instructions, the edge of its domain.
Now compare what happened to what you predicted. Two things show up. Some failures you predicted don't reproduce — fine, cross them off. And the feature breaks in ways you didn't predict, which are the most valuable findings of the whole exercise, because those are the ones that would've reached a customer.
The gap between your prediction and reality is the exact size of the product sense you're missing. Close it, then do it again next week with a different feature. A month of that and you'll start seeing failure signatures the way you used to see confusing flows.
The line that matters
Old product sense: I know what users want.
New product sense: I know what users want — and I know exactly how this model will let them down before it does.
The first one designs the feature. The second one decides whether it's safe to ship, which, in a world where quality is a distribution and not a guarantee, is the part of the job that's quietly getting harder.
Next: map a feature's failure modes in 30 minutesRelated posts
How to Map a Feature's Failure Modes in 30 Minutes
Most teams discover how an AI feature breaks after a customer does. This is a 30-minute ritual that surfaces the failures first. You walk out with a one-page failure signature.
MVP Walks. MVQ Decides Whether You Ship.
Minimum viable product tells you how small you can build. It says nothing about whether the thing is good enough to trust. For AI features, that second question is the one that ships you or sinks you.