The Consensus Machine

Nine AI judges can look like nine independent checks. Their mistakes may tell a different story.

A 2026 preprint featured by Apple’s machine-learning research group examined nine models from seven model families. Across three natural-language-inference datasets, the panel supplied roughly two independent votes’ worth of information. Its apparent breadth concealed substantial overlap in its errors. 1

This was a study of model evaluation, with reference answers drawn from human annotations. Its finding is not a universal conversion rate between model count and independent judgment. A separate, peer-reviewed ICML study evaluated more than 350 models across its datasets and also found substantial correlation in errors, including among capable models from different providers and architectures. 2

The question is whether organisations will mistake the proliferation of analysis for the proliferation of independent knowledge. An investment committee can have several analysts, several models and several reports—and still rely on one untested premise.

The same claim, wearing different citations

Consider a hypothetical manufacturer claiming that its new process substantially lowers production costs.

The claim appears in its presentation. An investor repeats it in a portfolio update. A trade publication includes it in an article. A research platform incorporates that article into a company profile.

Three analysts then use different AI systems to investigate the business. Each returns a detailed assessment, with multiple citations supporting the cost advantage.

Nothing needs to have been fabricated. The sources can be real and the summaries accurate. Yet all three analyses may ultimately depend on the manufacturer’s original statement. The committee has acquired more documentation, but not necessarily more evidence.

There are two distinct problems here. Models can share reasoning errors even when their providers differ. Researchers can also count the same observation several times because it has travelled through different documents. The first concerns correlated judgment; the second concerns the origin of information. Both can make confidence rise faster than understanding.

Now replace one of those reports with a customer’s production records: performance across several batches, under actual operating conditions, including downtime and rejected output. That evidence may support the claim or undermine it. Either way, it can change the decision in a way another summary cannot.

Independence is specific to the question. A financing demonstrates that someone committed capital. Without knowing what that investor examined, it is not direct proof that a manufacturing process works at scale. A successful pilot establishes performance in the pilot; it does not automatically establish routine deployment economics.

The useful question is, “Which observation supports this particular claim—and who actually made it?”

A second opinion is not necessarily a second observation.

Cheaper consensus, different economics

In their 1980 work on informationally efficient markets, Sanford Grossman and Joseph Stiglitz explored a basic tension: if prices perfectly revealed all relevant information, investors would have little reason to incur the cost of acquiring it. Their model allowed prices to be informative while preserving an incentive to produce information. 3

AI changes the cost of some of the activities within that tension.

Suppose competent analysis of public information becomes widely available at low cost. That should help more people understand businesses, identify inconsistencies and evaluate alternatives. But an activity can become more useful to everyone while becoming less differentiating for any particular participant.

If competing investors can produce similar market maps, technical explanations and financial scenarios, possessing those analyses may cease to be much of an advantage. The difference must come from better questions, better interpretation, different observations, superior access or a greater ability to act.

Two processes could occur together. Existing information could enter decisions more efficiently, while questions requiring new evidence remain unresolved. The market becomes faster at processing what is available without necessarily becoming equally good at discovering what is missing.

This is a hypothesis about where advantage may migrate, not a prediction that AI will make markets less efficient. Better reasoning can uncover insights in information everyone already possesses. Cheaper research can also make neglected questions economical to investigate. The outcome depends partly on whether organisations use the savings to produce more summaries or to explore further.

A contrarian opinion is cheap to generate. Evidence that justifies it is not.

Who captures the learning?

There is a further step between useful evidence and an investable company. The organisation generating information does not necessarily retain its economic value.

A laboratory may produce valuable results while customers own the data and intellectual property. An industrial application may record operating outcomes while the equipment manufacturer controls access. A verification product may be useful but easy for a larger platform to bundle.

The strongest business is not necessarily the one performing the hardest work. It is the one that can retain an advantage from the learning that work produces.

Does additional deployment improve the product for subsequent customers? Does the company have the rights and technical ability to reuse what it learns? Can it select what to test, or does it merely execute instructions? Does accumulated experience reduce costs, improve reliability or enable capabilities competitors cannot easily match?

These questions distinguish a learning advantage from an expensive data-collection operation.

They also leave room for different winners. An integrated company might combine intelligence and experimentation. A neutral infrastructure provider might serve competing models and research teams. An incumbent might already control the instruments, workflow or customer relationship needed to collect useful outcomes.

Our thesis is that, as capable reasoning becomes more accessible, control over a productive learning process deserves greater attention. Its investment value depends on who retains the benefit.

The scarce asset is a way to learn

A valuable system can check an answer and help identify which observation would most improve the next decision.

Consider two hypothetical industrial software companies. Both recommend interventions using equipment manuals and historical operating data. The first does not observe what happens after its recommendations are implemented. The second tracks whether the fault returned, whether a different intervention worked and which operating conditions changed the result.

Both can use the same underlying model. The second additionally connects its own recommendations to their consequences.

That does not automatically make it a better business. Its measurements may be noisy, customers may withhold data, or integrations may be expensive. The additional observations must improve decisions enough to justify their cost.

Still, this points toward a more precise advantage than “proprietary data.” A million private records can repeat the same narrow experience. A smaller collection of well-chosen outcomes may reveal something the larger dataset cannot.

The asset is a process that identifies consequential uncertainty, obtains useful evidence and incorporates the result into what happens next. In manufacturing, that could mean discovering which conditions determine yield. In enterprise software, it could mean connecting a recommendation to an observable customer outcome. The economic test is whether the system gets better at solving a problem customers value.

More data is an input. Less consequential uncertainty is the objective.

Reality can answer without a human referee

Useful correction need not remain slow, expensive or dependent on a human judgment.

Google DeepMind’s AlphaEvolve connects models that propose programs with automated evaluators that run, verify and score them. Results guide subsequent search. DeepMind has reported applications to algorithm discovery, computing efficiency and mathematical problems. Proposed solutions encounter tests, rather than another opinion about whether they sound convincing. 4

Feedback can therefore be computational as well as physical. Where correctness or performance can be checked reliably, generating and testing alternatives can produce new knowledge. The limitation is the test’s scope: passing it establishes what it covers, not every condition under which the output might be used.

Lila describes a physical version, connecting scientific reasoning models, automated laboratories and experimental results. Its stated architecture illustrates the approach; it does not by itself establish the resulting advantage. As we explored in AI Needs Somewhere to Learn, experimental capacity matters when reliable, informative results improve subsequent decisions. 5

Even physical tests can reproduce a shared blind spot. Repeating a measurement under narrow conditions does not answer a different question. The capacity to test alternatives matters as much as the capacity to generate results.

The advantage lies in a reliable way to discover why an existing view is wrong.

Independence is partly an organisational choice

Research also argues against treating homogenisation as inevitable. A Columbia Business School study found that structured reasoning combined with varied, ordinary personas increased diversity in model-generated ideas, exceeding the human comparison in the tasks studied. A separate 2026 preregistered creative-writing experiment by Nathanael Jo and Manish Raghavan found that participants rewarded for originality produced more diverse work than those rewarded for quality alone. They used AI more selectively and incorporated fewer suggestions verbatim. 6 7

The latter remains a preprint. Neither study establishes independent factual validation: idea diversity is a different outcome. But both give organisations a reason to examine how they structure AI-assisted work.

A team rewarded for polished answers delivered quickly may behave differently from one rewarded for finding the assumption that changes a decision. Asking a researcher to confirm a thesis creates a different task from asking which observation would overturn it.

This is not a case for disagreement as a performance metric. Novelty without regard to truth can produce confident nonsense. The objective is to preserve the possibility of correction.

That could mean collecting initial judgments before sharing a synthesised recommendation, asking experts questions that do not reveal the preferred answer, or tracing important claims back to their original observations.

Uncertainty must remain visible. A system should distinguish “several sources repeat this” from “several parties independently established this.” A well-written explanation should not silently upgrade the status of its evidence.

The information advantage after abundant analysis

For investors, this adds another dimension to diversification. A portfolio can span sectors while depending on the same underlying assumption. A research process can span people, models and information services while relying on the same evidence.

Neither concentration is necessarily wrong. The problem is failing to recognise it.

Independent conviction does not require disagreeing with the market. It requires understanding why a conclusion should survive examination beyond the market’s enthusiasm. Sometimes that comes from a new observation; sometimes from a better interpretation of shared information. Better models can help with both.

At Aldea, we would judge an AI research system less by its report count than by whether it helps answer three questions: What do we know? What remains assumed? Which next observation could change the decision?

The next information advantage may belong to whoever has built the best way to find out whether a sophisticated answer is wrong.

The risk is not that everyone agrees. It is that everyone mistakes agreement for independent confirmation.

Disclosure: Aldea is an investor in Lila. 8