The Cheap Yes

The cheap yes

The last full market-validation study I ran cost $44.04. I know because I keep a log. A study like it – new market, real segmentation, competitive analysis, the works – used to be a six-figure line item and a quarter of calendar time. Mine ran overnight.

That collapse is real, and it's not just research. Getting a working, demo-able product in front of users used to mean $15,000 to $50,000 and two to four months with an agency. In 2026 the same first step runs somewhere between $50 and $2,000 in tooling and days to weeks of your time.⁴ The raw compute underneath it fell roughly a thousandfold in three years.² By 2024, using AI tools was majority developer behavior,¹ and by early 2025 a quarter of one Y Combinator batch had codebases that were about 95% AI-generated.³

So every instinct says just build it. Your engineers say it, your own excitement says it, and the AI certainly says it – ask any model whether your idea is good and watch it agree with you.

I say a version of it too. I've built six software tools in the past few months, and I'm not an engineer. The cheap yes is genuinely wonderful. It's also the most expensive thing in your company right now, and this paper is about why.

Here's the claim: AI made building cheap. It didn't make being wrong cheap. The cost of a bad bet didn't shrink – it moved downstream, into places that are harder to see and slower to bill you. A yes that costs nothing gets none of the scrutiny that a six-figure yes used to earn, so you say more of them, and the bill arrives later with no itemization.

One more tally before we get into the evidence, offered as exactly what it is – my own small sample, not a benchmark. Across twenty-some years of product work, the bets I was part of hit at roughly the industry rate: around a third moved the metric they were meant to move. The bets I've made since flipping the order – evidence first, build second – are running closer to nine in ten. Small numbers, one practitioner. But the direction matches everything the research below says about what actually changed, and what didn't.

Where the cost went

The natural assumption is that if building got two orders of magnitude cheaper, being wrong got two orders of magnitude cheaper too. It's the assumption behind “fail fast,” behind “just ship it and watch the numbers,” behind every roadmap that quietly doubled in ambition since 2023. The data doesn't support it. Four independent lines of evidence – code-structure telemetry, delivery surveys, peer-reviewed security research, and enterprise ROI studies – all point the same direction: the write-time got compressed, and the cost slid downstream.

The Anatomy of a Cheap Yes

The build is the part you can see

Everything below the line arrives after you ship – and none of it got cheaper.

The tip is an order-of-magnitude band; the masses below are documented cost destinations, not to scale. Sources and caveats are in the references. Tap the graphic to enlarge.

Start with the base rate the cheap yes runs into. Most product bets miss, and they missed before AI. Ron Kohavi, who ran experimentation at Microsoft and Bing, found that only about a third of well-designed ideas improved the metrics they targeted – and in mature, heavily optimized products like Bing and Google, the win rate drops to 10–20%.⁵ Usage in shipped software is just as skewed: the majority of features draw minimal engagement. Pendo's 2019 telemetry analysis put it at 80% of features “rarely or never” used, though you should know how they counted – “rarely” meant the feature sat in the last 5% of aggregate click volume, which makes the exact number softer than the headline suggests.⁶ The skew itself is not in dispute. Cheap building doesn't raise the hit rate. It raises the number of swings, which means more misses to absorb – and the absorbing is where the money is.

When AI writes more of the code, the downstream gets heavier. GitClear analyzed 211 million changed lines of code and found that alongside AI adoption, copy-pasted code rose from about 8% of changes to over 12%, refactoring collapsed by more than half, and churn – code revised within weeks of being written – roughly doubled. GitClear sells code analytics, so treat their numbers as shape rather than gospel,⁸ but the peer-reviewed tier points the same way with cleaner methods: roughly 40% of Copilot's completions in security-sensitive scenarios were vulnerable,⁹ and a Stanford lab study found AI-assisted developers wrote less secure code while being more confident it was secure.¹⁰ That last finding is the whole cheap-yes problem in one sentence.

Delivery didn't speed up – stability got worse. Google's DORA research, surveying tens of thousands of practitioners, found in 2024 that more AI adoption was associated with slightly worse delivery throughput and meaningfully worse delivery stability. In 2025 the throughput finding reversed – this evidence is still settling – but the instability finding held for the second consecutive year.⁷ The individual coding gains are real but modest and uneven: McKinsey's published figures show 35–50% time savings on routine work and under 10% on high-complexity work, which is usually the work that sets your ship date.¹³ The most careful study we have on experienced developers, METR's randomized trial, actually measured them 19% slower with AI on their own codebases – with a follow-up a year later suggesting the same developers had become faster, a result METR itself flags as unreliable.¹² Whatever the true number is, it is not “so fast that being wrong is free.”

At the enterprise level, the returns mostly aren't there yet. Preliminary signals from MIT's 2025 Project NANDA report suggest that despite $30–40 billion in enterprise GenAI investment, 95% of organizations were seeing zero measurable return – a striking figure that comes with its own health warning attached, since the authors describe it as directionally accurate interview data rather than audited results.¹⁴

And the old costs never left. Maintenance consumed 50–75% of total software cost before AI existed – that's Boehm's number from 1981, backed by a 487-organization study, and modern corroboration says it still holds.¹⁵ AI does not compress that cost. What it compresses is the cost of creating new things to maintain. Every cheap yes enlarges the denominator that 50–75% applies to. The same is true of the costs that were never code in the first place: SOC 2 audits run $20,000–$80,000 all-in, PCI starts around $15,000 and climbs, building enterprise SSO in-house takes teams three to four months – and vendors charging for SSO mark it up 50–500%, with documented outliers above 16,000%.¹⁷ App Store review is a fixed human gate that AI-accelerated volume is actively congesting: submissions grew about 30% in 2025, to roughly 600,000, while Apple's stated review turnaround slipped from 24 to 48 hours.¹⁷

Then there are the costs nobody invoices. Killing a shipped feature is funded engineering work – Google's own engineering handbook treats deprecation as a project, not a delete key – though no public study has ever put a benchmark dollar figure on it. Cancelled projects run 11–15% of the total, and what demoralizes the people on them, per the research we have, isn't failure – it's the sense that the work never mattered.²³ There 's the opportunity cost of the right thing you didn't build while shipping the wrong one: the one publicly named, quantified case measured a single delayed feature at over $200,000 per week, and while that's one consultancy's own client data rather than an audited benchmark, the order of magnitude is the point.¹⁶ And there's a newer cost with no line item at all: when a machine writes most of your system, your team slowly loses its own mental model of how the thing works. Practitioners have started calling it epistemic debt.²²

The expensive no

Play the tape forward. You said the cheap yes. The team shipped it in three weeks instead of three quarters, which genuinely is a miracle. Ninety days later the dashboard is flat. Now you're standing in front of the most expensive question in product: was it the idea, or the execution?

When a shipped bet moves nothing, there are four candidate explanations. Wrong audience – the people you built for don't have the problem. Wrong problem – they have problems, just not this one. Wrong solution – right problem, but this doesn't solve it. Wrong execution – right everything, built badly or half-adopted. Each one calls for a completely different next move: retarget, re-research, redesign, or fix and push. Choose wrong and you pay for the failure twice.

What the research says here is uncomfortable in both directions. No study has ever tested whether teams can correctly make that four-way call after the fact – we looked, hard, and the absence is real. What decision science has established is everything that predicts they can't: hindsight bias, outcome bias, the documented tendency of companies to attribute failure to external causes and success to themselves, and – closest to home – research on startup post-mortems showing that founders and outside experts looking at the same failure systematically name different causes.¹⁹ Even CB Insights, whose “why startups fail” lists get quoted everywhere, walked back their own top answer in 2024, reclassifying “ran out of money” from a cause to a symptom. In the one setting where we do have ground truth – controlled experiments – expert intuition about what would work was wrong roughly two-thirds of the time.⁵ Nothing suggests intuition about why it didn't fares better.

The cheap yes makes this worse, not better, because cheap bets are exactly the ones that ship without the instrumentation, the baseline, or the success criteria that would let you interpret the result. You saved the scrutiny going in, so the answer isn't there coming out. If the expansion fails, it's on you – and you won't even know whether it was the idea or the execution.

There's a related freeze on the front end, and there's real data on it, though it comes from a different room than yours. Matthew Dixon and Ted McKenna analyzed 2.5 million recorded B2B sales conversations and found 40–60% of qualified deals end in no decision at all – buyers who had the pain, saw the solution, and chose nothing, mostly out of fear of messing up rather than love of the status quo.¹⁸ Their data is about buyers evaluating vendors, not CEOs greenlighting internal bets, so carry it across as an analogy rather than a measurement. But the psychology travels. When you can't price being wrong, the safest-feeling move is to decide nothing – and in a market moving this fast, the no-decision is its own expensive bet.

So the founder-CEO in 2026 is squeezed from both sides. Say the cheap yes and you buy a lottery ticket with a hidden carrying cost and an uninterpretable outcome. Freeze, and you pay the cost of delay on everything. The way out of the squeeze isn't building faster or deciding slower. It's making the evidence cheap before you make the bet – which, unlike the rest of this, actually did get cheaper.

What a good answer looks like

You have four realistic ways to answer “what should we build next,” and it's worth being honest about all of them, because each one genuinely wins somewhere. Deciding internally – debate it, or build-and-see – is unbeatable on speed to a first signal. Hiring a senior product leader is the right call when what you actually need is someone to own execution for years. A brand-name consultancy brings a logo your board already trusts. The catches are just as real: internal debate adds no new outside evidence, a senior hire takes three to six months to find and ramps up inside the same information vacuum you're in, and the strategy deck tends to arrive un-buildable – personas your engineers can't estimate from, direction nobody can ship against next sprint.

Whichever way you go – including hiring no one – hold the answer you're buying to the same standard. Six criteria, and the reasoning behind each:

Notice what the criteria do to the four options. Build-and-see fails the first one – it generates telemetry, not understanding. The senior hire fails the second – the calendar math doesn't work when the question is now. The deck fails the third. And nearly everything on the market fails the fourth, because telling a paying client to not do the thing they're excited about is bad for repeat business, which is exactly why it's the criterion worth paying for.

What it costs to answer well

Pricing in this market mostly lives in sales calls, so here's the best cited picture the research could assemble: the 2026 market for an answer to “what should we build next,” with the sourcing caveats attached. The spread covers five orders of magnitude, and the shape of it tells you where the value actually lives.²⁰

Two things jump out of that table. First, the gap between $60 and $400,000 isn't a pricing inefficiency – it's the market telling you that judgment is the expensive part. Tokens got a thousand times cheaper; someone who can tell you not to build the thing did not. Second, everything between the extremes is priced against the size of the mistake it prevents. A $15,000 answer protecting a two-quarter, six-engineer bet – call it a half-million dollars of fully loaded payroll before the downstream costs start – is cheap insurance if, and only if, it meets the six criteria above.

Match the method to the question

Not every direction question calls for outside research. The right next move depends on where you actually are.

If you're pre-product-market-fit, the question is still yours to answer in person. Zero-to-one conviction gets built in the conversations themselves. Research can feed that work – it can't substitute for it.

If the evidence is already in hand, the remaining work is the decision. I've sat with founders who could recite the customer quotes, the churn numbers, and the competitive gap from memory. They didn't need another study; they were one decision away.

If the direction is sound and delivery keeps slipping, the question is execution. That's a real problem with real fixes – it's just a different diagnosis, and buying direction won't touch it.

The argument in this paper is for the case in the middle: post-PMF, a real direction question on the table – a new market, a second product, the next year's bet – and no cheap way to be wrong about it.

Nine months of direction: the Tallo pattern

Tallo is a career-platform company I've worked with since 2025 – I'd worked with Stride, their parent company, back in 2022. They had the classic version of the question: strong core product, a direction decision about where to grow next, and more conviction than evidence about a market of non-college-bound career starters that their intuition – tuned on their core users – couldn't actually read.

The engagement ran the order this paper argues for. Research first – customer interviews and market studies on the new audience before committing the build. Then strategy: the evidence became personas the team actually uses, a problem inventory engineers estimate from, and a roadmap.

One market got a firm no, and it wasn't hypothetical. Tallo had already been investing there – partnerships in motion, staff on it full time – and it wasn't paying off. The research showed why: the segment was smaller than it looked, and serving it well would have taken kinds of support Tallo isn't structured to provide and that sat outside its mission. My recommendation was to stop, and I put my name on it. They wound the work down and moved those people and that focus to the audiences that were hitting their goals.

The result I'd point to is the hit rate. We set KPI targets from benchmarks before each launch, and every feature shipped from this roadmap is clearing them – typically by two to four times. That's the base rate from earlier in this paper, bent the right way. The depth matters too: as of July 2026, the roadmap is built out through April 2027 on research that's now being deepened rather than redone. Direction isn't a deck they bought – it's a position they hold.

Start smaller than you think

You don't have to believe this paper. That's rather the point – you shouldn't take direction claims on faith, mine included. The six criteria stand on their own; apply them to whoever you're evaluating, including us, including nobody.

But if the argument holds – building got cheap, being wrong didn't, and evidence is the one input whose price truly collapsed – then the rational move is to spend a little on evidence before you spend a lot on a bet. That's the whole Fieldway model, and it starts deliberately small: a research engagement, published price, criterion five by design. The details live at fieldway.org/research, and I'm at matthew@fieldway.org if it's worth a conversation.

Most of what the research says is yes, by the way – Tallo's numbers are what the yeses look like. You can trust a yes that was allowed to be a no.

A cheap yes leads to an expensive no. Buy the honest no early, while it still costs almost nothing to act on.

References

Every caveat stays with its citation – it's part of the claim, not fine print.