Your AI isn't Stuck on Technology, it's Stuck on You

 

One of the largest banks in the US ran 47 AI pilots last year. I asked a senior exec there how many had changed the real number on the P&L. He thought about it, then said: "One. Maybe."

Forty-seven experiments. One result. That gap is the whole story of AI in 2026, and almost nobody is naming it correctly.

For three years, the questions were about technology. Which model? Build or buy? How do we do RAG? Are we behind? Fair questions in 2023. They've quietly stopped mattering. The models are good enough, cheap enough, and available to everyone, including your competitors. The barrier to building something with AI has fallen to roughly zero.

So why is the promised return still stuck in pilot purgatory?

Because deploying AI is easy, changing how a company works is hard. And the second one is the actual job.

AI doesn't fix your company. It exposes it

The part executives don't want to hear: AI amplifies whatever was already there. Run it on top of a sharp, fast-deciding organization, and it compounds the speed. Run it on top of unclear ownership and slow approvals, and all you've done is generate confusion faster.

The pilot that summarizes contracts works fine in the demo. Then it dies in the org. Legal doesn't trust the output, nobody decides who's accountable when it's wrong, and the old manual process still runs in parallel "just to be safe." None of that is a data-science problem. You can't prompt your way out of an org chart.

That bank's 47 pilots weren't a technology achievement. They were a museum. Lots of impressive exhibits, nothing in production.

Three places the work actually breaks

Decisions move at committee speed while information moves at machine speed. AI now hands a team a forecast in minutes. Then that forecast waits two weeks for a Thursday steering meeting. When insight is instant and the decision is slow, the speed you paid for evaporates inside your own approval chain.

You're measuring effort in a world that no longer rewards it. Hours worked. Headcount. Tickets closed. Number of pilots launched. These tell you a team is busy. If one person now does what five used to, "busy" is the wrong thing to count. Cycle time, decision velocity, cost avoided, a customer kept. Those are the numbers that moved, and most dashboards don't track them.

The work sits between your departments, but your org chart doesn't. The useful AI workflows cut across marketing and analytics, finance and forecasting, product and support. Your structure still has walls exactly where the value wants to flow. So it doesn't flow.

The one comparison worth holding onto

When factories first wired up to electricity, productivity barely moved. The owners had swapped the steam engine for an electric motor and changed nothing else. Same layout, same workflow, same building designed around one central power source.

The gains came decades later, when a new generation of managers redesigned the whole floor around what electricity made possible: machines anywhere, smaller flexible lines, a different shape of work entirely. The technology had been ready for years. The management caught up late.

We are at the "wired up but unchanged" stage with AI. The motor is bolted in. The factory floor hasn't been touched.

What this asks of you

This is the uncomfortable shift. Your job stops being "what can AI do for us" and becomes "what has to change in how we run for any of that to land."

That means deciding, on purpose, which decisions stay human, which become AI-assisted, and which you'll let a system make on its own - and who is accountable for each. It means killing work, not just speeding it up; half your reports and approvals exist only because automation didn't exist. It means promoting the operator who can redesign a process, not only the engineer who can fine-tune a model.

None of that is exciting. It's slower and more political than buying another tool. Which is exactly why it's the real moat. Anyone can buy the model. Few people are willing to take on the management work around it.

The biggest risk to most executives right now isn't getting out-innovated. It's getting out-managed by a competitor running the same models you have - just inside a company built to actually use them.

So the question I'd put to you: where is your AI actually stuck - the technology, or the way your organization decides, measures, and owns the work? Be honest about which one you've been funding.

Habsburg Jaw in making: Model Collapse in AI

 

 AI model collapse is a degenerative process in machine learning where a model's future generations degrade in quality when trained on synthetic (AI-generated) data rather than original human-created data.

On November 1st, 1700, an entire dynasty of kings came to a crashing end with the death of Charles II of Spain. He was physically & mentally disabled and disfigured. A large tongue made his speech difficult to understand; he was bald by the age of 35, and he died senile and wracked by epileptic seizures. He had two wives, but being impotent, he had no children and thus, no heirs. What else do you expect after 16 generations of inbreeding!

How is it happening

To understand model collapse, you have to understand how AI models process probability.

When a generative AI (like an LLM or an image generator) creates content, it calculates the most statistically probable next word or pixel based on its training. Because it favors the "most likely" answer, its outputs naturally gravitate toward the average or the median of what it has learned.

Generation 0 Training Data

Training Data set originates from human-written books, articles, websites, images, audio, video, code, and human conversations.

The model learns the full spectrum of human expression, including rare, weird, and highly diverse "edge cases" (the tails of the statistical distribution).

Generation 1 Training Data

Training Data set mostly originates from human-generated sources and a few incidents of machine-generated data. The machine-generated data is entering the training data set knowingly as well as unknowingly.

Still good, but begins inheriting biases and omissions.

Generation 2+ Training Data

Most of the training datasets originate from machine generation.  The proportion of human-generated data reduces significantly with each generation.

Training data sets become generic, and different models feed each other. This is resulting in the disappearance of rare information, and mistakes get reinforced.

Every real-world human dataset has a distribution - a spread of outputs across many possible styles, topics, phrasings, edge cases, and rare examples. When a model trains on this and generates new content, it approximates that distribution but doesn't reproduce it perfectly. The tails - the unusual, the rare, the surprising - get underweighted. When the next model trains on those outputs, it further smooths out the tails. Over enough generations, you converge on an overly smooth, narrowly peaked distribution. The model becomes increasingly "average" and loses the ability to represent rare-but-real things.

Types of Model Collapse

Distribution Collapse – Loss of Tail Distribution

The model forgets rare but important patterns.

Example: Original data contains 1,000 bird species, but AI-generated data mostly discusses common birds.  This will result in future models forgetting uncommon species.

Error Amplification

Small errors become accepted facts.

Example: Model A hallucinates a historical date, so AI-generated articles repeat it.  Model B is consuming data from Model A’s output. Model B learns it as truth.

Diversity Collapse

As each model will be trained on other models’ output, outputs across Models become increasingly similar. This will result in the same writing style, explanations, examples, and reduced creativity.

Capability Collapse

The model loses nuanced reasoning abilities.

Examples:

• Less robust coding
• Poor edge-case handling
• Weak scientific reasoning

Symptoms of Model Collapse

The symptoms include:

• Loss of Variance: The model output becomes incredibly repetitive. In image generation, for example, all faces might start to look like the same generic, heavily averaged face.
• Disappearance of the "Tails": The model forgets about rare concepts, subcultures, or complex vocabulary.
• Compounding Hallucinations: If Generation 1 makes a slight factual error, Generation 2 treats that error as a fact and amplifies it. By Generation 5, the model is completely detached from reality.
• Perceptual Blindness: The model loses the ability to understand the original distribution of the data, making it impossible for it to "recover" even if human data is reintroduced later.

Is Synthetic Data Always Bad?

The simple answer is No, but….

High-quality synthetic data can be extremely valuable in rare edge cases, safety training, mathematical proofs, coding exercises, and data augmentation

The danger arises when synthetic data dominates the training data set, no grounding in real-world human data, and quality controls are weak.

How the AI Industry is Fighting Back

Because high-quality, original human data is a finite resource (often referred to as "peak data"), the industry is actively developing defenses against model collapse:

• Data Provenance and Watermarking: Developers are working on invisible watermarks for AI text, code, and images. This allows future web-scrapers to automatically filter out synthetic data and only train on verified human data.
• Strict Data Curation: AI labs are moving away from "scrape everything" approaches. They are employing massive teams of humans to curate small, incredibly high-quality datasets of original human work.
• Synthetic Data Filtering: While training exclusively on synthetic data causes collapse, researchers have found that training on a carefully curated mix of human data and high-quality synthetic data can actually be beneficial. The key is ensuring the synthetic data is strictly verified for accuracy.
• Alternative Modalities: Since text and image data are easily polluted, some researchers are looking toward training models on physical world data (like video, robotics, and sensor data), which is much harder for an AI to fake.
• Architectural or training innovations: Techniques that better preserve tails or use reinforcement to avoid plateaus.

Summary

Model collapse is the ultimate bottleneck for AI scaling. It proves that AI cannot replace the need for human creativity and original data. To continue advancing, AI models will always rely on a steady diet of genuine, messy, diverse human input. If the internet becomes an echo chamber of AI talking to AI, the technology will stagnate and degrade.