Micro Experiments
/
September 22, 2026

The Cognitive Exoskeleton Should Make It Cheaper to Be Wrong

AI-powered experimentation should let more ideas reach evidence before investment. Human judgment still decides which deserve to move forward.

AI-powered experimentation changes one of the oldest constraints in innovation: how many ideas we can afford to investigate before we have to choose. For years, we treated early narrowing as discipline because exploration was expensive. Interviews had to be scheduled. Workshops had to be prepared. Research had to be synthesized. Prototypes had to be built. Client teams and consulting teams both had to make time. The question was not only which ideas looked promising. It was how many ideas we could afford to understand properly before the clock ran out.

I learned that constraint repeatedly in venture-building and transformation work. The exploration phase could consume enormous amounts of energy before anything had been validated. Qualitative interviews were valuable, but somebody had to recruit participants, conduct them, interpret the answers and turn them into something a team could act on. Ideation workshops were valuable, but they pulled broad groups of people away from their normal work. Prototypes created evidence, but only after we had decided which ideas deserved the effort of prototyping.

That made prioritization necessary very early. We used scrutiny and compromise to compress the field before real validation began. This was not bad practice. It was rational practice under a scarce-resource model.

AI changes enough of that model that the discipline should change with it.

We used to shortlist before we could really know

In one engagement with a large OEM, the organization had a backlog of 72 ideas. We needed to decide which ones should move forward. So we did what good teams did: we brought people together, worked through the backlog and shortlisted the most promising candidates before investing in deeper testing.

The workshop went well. There was surprisingly little vanity around the ideas, even though they had come from inside the organization. People were willing to challenge them. We created a sensible shortlist and moved on.

What strikes me now is not that the process was wrong. It is that the stopping rule was partly invisible.

We stopped testing when the time ran out.

Some ideas deserved to die early. Others may have deserved a more serious test. But the economics of exploration meant we could not treat all 72 with equal intensity. We had to narrow before we had strong evidence because the act of producing that evidence was itself expensive.

That assumption shaped a generation of innovation methods. Compress the exploration phase. Get to the shortlist. Focus scarce specialists, scarce customer access and scarce executive attention on a small number of candidates. Then validate the survivors properly.

The important question is not whether that logic was sensible. It was. The question is whether we should preserve the same narrowing logic after AI reduces the cost of research, synthesis, comparison, preparation and prototyping.

AI-powered experimentation widens the field before the gate

I have previously described AI as an exoskeleton rather than a prosthesis. The point is not to remove the humans who decide which ideas deserve investment. The point is to increase how much territory those humans can examine before they decide.

Research can be accelerated. Interview material can be synthesized faster. Competing hypotheses can be pressure-tested before the next workshop. Rapid prototyping can be expedited and scaled even further by applying similar production guidelines to AI production, while A/B testing can become A-E testing with AI-assisted evaluation and synthesis. Alternatives that would once have been dropped because nobody had time to develop them can receive a lightweight test instead.

That is a different use of productivity from simply doing the same work faster. It means using the returned capacity to increase the number of plausible paths that encounter evidence.

This is where the AI Capacity Dividend becomes useful as an innovation concept. If AI returns time, attention and iteration capacity, management does not have to spend all of it on throughput. Some of the dividend can buy diversification.

Instead of choosing five ideas because only five can be prepared properly, perhaps twenty can receive a first-pass prototype. Instead of one concept entering customer discovery, several can encounter structured criticism. Instead of asking a team to bet early, we can give more candidates a cheap opportunity to prove that they should not be killed yet.

This does not mean every idea deserves equal investment. It means more ideas can earn or lose the right to receive investment based on evidence rather than on how early we ran out of capacity.

The evidence for cognitive leverage is real, but uneven

There is now enough experimental evidence to take this mechanism seriously without pretending it is universal. In a pre-registered field experiment with 776 professionals at Procter & Gamble working on real product-innovation challenges, individuals using AI matched the performance of teams working without AI. AI users also produced more balanced solutions across commercial and technical backgrounds. The NBER working paper describes AI as changing not only performance but the way expertise sharing manifests inside teams.

That matters for exploration because early-stage innovation is often constrained by the number of perspectives a team can bring together. If AI can help one person consider technical, commercial and customer dimensions that previously required more coordination, the cost of giving an idea a competent first look falls.

Other studies show similarly meaningful gains on bounded knowledge work. In an experiment with 453 college-educated professionals completing writing tasks, generative AI reduced average completion time by 40 percent and increased output quality by 18 percent. The published Science study does not prove that innovation becomes 40 percent faster. It does show that some cognitive production tasks can become materially cheaper.

But cheaper cognition is not the same thing as reliable cognition. The now-published research on the jagged technological frontier makes that boundary explicit. In a field experiment with 758 consultants, AI improved speed, task completion and quality across tasks that sat inside the model’s capability frontier. On a complex task outside that frontier, participants using AI were 19 percentage points less likely to reach the correct answer.

That is the central discipline for AI-powered experimentation. We should use AI aggressively where it expands the field we can investigate, while becoming more, not less deliberate about the point where machine-supported exploration turns into a consequential human decision.

More options do not make more options good

There is an obvious failure mode hiding inside this argument. If AI makes it cheap to generate hypotheses, concepts, prototypes and business cases, organizations can drown themselves in cheap possibility.

I do not think the answer is to preserve the old scarcity artificially. We should not keep expensive workshops, slow analysis or narrow shortlists simply because those mechanisms once forced focus. But we do need to preserve the judgment that scarcity used to impose indirectly.

Before AI, the cost of exploration forced us to ask hard questions early. Which ideas are actually worth the effort? Which assumptions are weak? Which customer problem matters enough? Which concept is somebody defending because it is theirs rather than because evidence supports it?

AI does not make those questions obsolete. It separates them from the production cost of asking them.

That can be a major improvement. A bad idea is still a bad idea even if ten agents can work on it in parallel. A weak value proposition does not become strong because a model can produce a beautiful prototype. An executive should not confuse the cost of generating an option with the quality of the option itself.

There is also evidence that generative AI can create convergence as well as variety. A 2024 experiment on creative writing found that AI-assisted stories were rated as more creative and better written, particularly for less creative writers, but they also became more similar to each other. The Science Advances study is not an innovation-management experiment, but the warning transfers: individual improvement can coexist with reduced collective diversity.

So the objective is not simply to ask AI for more ideas. It is to use AI to investigate a wider field while deliberately protecting human challenge, independent evidence and alternative frames.

Fail fast should now mean fail with better evidence

I have argued before for failing fast and failing cheap. AI makes the phrase more operational.

Cheap failure should not mean producing a large pile of disposable prototypes. It should mean discarding an idea before it consumes transformation-scale money, organizational attention and political capital.

Sometimes what fails is one assumption. Sometimes it is the proposed solution. Sometimes the customer problem is real but the economics are wrong. And sometimes the whole idea deserves to die.

The useful change is that we can now give more ideas enough evidence to fail for a reason.

That is different from an old-fashioned shortlist based mostly on expert instinct. Expert judgment still matters. But it can operate later in the funnel, with more context around each candidate. The human does not have to spend all their scarce cognition producing the material that makes comparison possible. They can spend more of it interpreting what the comparison means.

This extends the logic behind the AI Value Hypothesis: an idea should have permission to begin before certainty exists, but every increase in commitment should require stronger evidence.

More ideas deserve a chance to be tested. Fewer deserve the right to consume serious capital.

Use the Capacity Dividend to diversify, then concentrate

The practical operating model I would use is not complicated. The change is in where we spend scarce human effort.

A four-step experimentation loop

  1. Broaden cheaply. Use AI to give more plausible ideas enough research, synthesis and prototyping to become testable rather than merely imaginable.
  2. Expose them to evidence. Put prototypes, assumptions and value hypotheses in front of customers, operators, domain experts or real data as early as possible.
  3. Cut hard. Use human judgment and explicit evidence gates to remove ideas that remain weak, unnecessary, uneconomic or strategically incoherent.
  4. Concentrate investment. Reserve substantial capital, operating-model change and scarce leadership attention for the ideas that survived because the evidence improved—not because the artifact looked convincing.

This is not permission for indiscriminate experimentation. The funnel should be wider at the top and harder at the gate.

The old model often forced us to concentrate before we had enough information. The AI-native model can postpone concentration until the marginal cost of learning begins to rise materially or the next test requires something scarce: customer trust, physical infrastructure, regulated data, organizational change, capital or executive sponsorship.

That boundary matters because AI capacity is not the same as enterprise capacity. A model may be able to generate thirty concept variants overnight. The organization may still only be able to pilot two responsibly. A team may be able to simulate a market argument. It may still need real customers to establish willingness to pay.

Human judgment therefore moves rather than disappears. It shifts from producing every candidate manually toward deciding which evidence matters, which assumptions are dangerous and when an idea has earned access to the next scarce resource.

The human gate becomes more important when AI does more of the work

This is where I think people can become lazy in a very specific way. We get accustomed to AI doing heavy cognitive lifting for us. It retrieves, compares, synthesizes, drafts and challenges. The output looks finished enough that evaluation starts to feel like the smaller task.

It is not.

At some point, we still need to get our butts off the couch and be diligent about evaluating what AI created. We need to ask whether the problem is worth solving, whether the evidence is real, whether the model has misunderstood the context, whether the customer cares and whether we are falling in love with the sophistication of the artifact instead of the strength of the case.

The enterprise was built in part to ration cognition and this is the individual version of the same transition. If AI makes more cognition available, human contribution should move toward the things scarcity previously crowded out: judgment, challenge, imagination, relationship, accountability and the willingness to say no.

That is also why the exoskeleton metaphor remains useful. An exoskeleton can let you carry more. It does not decide which direction you should walk.

Before the €500k, ask what facts make the return defensible

The ultimate test is not how many experiments an organization can run. It is whether experimentation improves the quality of consequential investment decisions.

Imagine an executive is being asked to approve €500,000 for an AI initiative. A polished prototype is not enough. Enthusiastic user feedback is not enough. A vendor benchmark is not enough. Even a credible use case is not enough if nobody can explain how the intervention creates economic value.

The evidence gate before material investment

  • What return are we expecting? Revenue, avoided cost, capacity, risk reduction, cycle-time improvement, quality or another measurable outcome.
  • What mechanism creates that return? The causal chain between the AI-enabled change and the business outcome must be explicit.
  • Which facts already support the mechanism? Real behavior, tests, operating data, customer evidence or comparable cases; not only model-generated reasoning.
  • What remains uncertain enough to kill the case? The next experiment should target that uncertainty before the commitment grows.

The standard should become stricter as experimentation becomes cheaper. If an organization can pressure-test ten versions of a proposition before requesting €500,000, the executive should expect a better answer to the ROI question than they did when every meaningful test required weeks of specialist work.

This is the paradox I find most interesting. AI reduces the cost of being wrong early. That should make us less tolerant of being wrong expensively.

The Capacity Dividend should buy better human cognition

The real Capacity Dividend is not that I can do the same work faster. It is that I can rebalance my effort toward the things that require the unique characteristics of human cognition.

In my own work, that means I rarely reject an idea simply because I do not have time to explore it. I vary the intensity. Some ideas get substantial direct work from me. Some receive structured prompting and AI-supported investigation. Some begin as conversations with AI while I am in the car or moving between other commitments.

The point is not that every thought becomes a project. Most should not. The point is that the cost of giving a thought enough attention to become intelligible has fallen dramatically.

That changes the role of the human bottleneck. My scarce contribution should increasingly be deciding where to care deeply, where to challenge the machine, where to involve another person, where to gather real evidence and where to stop.

For organizations, the same principle applies. AI-powered experimentation should widen the evidence funnel, not the investment funnel. Let more ideas reach reality. Let more assumptions be challenged. Let more prototypes fail before they become programs.

Then concentrate the scarce things: capital, trust, organizational disruption, executive attention and human accountability, on what survived.

That is a better use of the cognitive exoskeleton. Not to avoid thinking. Not to make every idea look good. To make it cheaper to discover which ideas were never good enough to deserve our scarce human effort in the first place.

Continue Exploring

More from Open Source Consulting

Continue with the latest thinking across Micro Experiments, Macro Frameworks and the Executive Briefing.

Business outcomes with AI: two industrial service professionals inspect a machine component in a manufacturing workshop

Briefings

Open Source Executive Briefing CW43: When Faster Execution Misses the Business Outcome

Agentic capability and implementation capacity are expanding. The management advantage is not producing more deliverables but shortening the path from a business hypothesis to credible buyer evidence and a human go/no-go decision.

AI-native operating system connecting busy office operations with a distant strategic objective across applications

Macro Frameworks

The Operating System Is Moving Above the Applications

AI can make teams exceptionally productive while they drift from their objectives. An executive case for objective-led orchestration, evidence and human judgment.

AI-native workflow concept: a professional reviewing information as work records and documents move between teams

Micro Experiments

A Workflow Is Not a Collection of App Clicks

An AI-native workflow carries reliable knowledge into human decisions and preserves what happens afterward. A practical approach to organizational handoffs.