The internet is undergoing one of the most consequential transitions in its history, from a world dominated by search engines to one increasingly ruled by AI-powered answer engines.
It’s an economic earthquake that threatens to obliterate the business model that’s underpinned the web for decades.
While most tech companies have jumped on the AI bandwagon, one prominent CEO is sounding the alarm and actively proposing a fix: Cloudflare’s Matthew Prince.
“These aren’t search engines anymore, they’re answer engines. The economics and rules are very different,” he told me in a recent interview. “We have to strike a new deal.”
The problem
For the last 25 years, Google’s search engine has generated a digital map that sent users on a treasure hunt across the web to find the information they wanted. This generated traffic, which supported the grand bargain of the internet where sites let Google copy their data in return for referrals and the related value that brought. Money from ads and subscriptions paid for more content creation, which, in turn, helped Google show better search results.
In today’s era of AI-powered “answer engines,” the map is no longer the product — the answer is. With tools such as Google’s AI Overviews and AI Mode, OpenAI’s ChatGPT, and Perplexity, users are fed synthesized responses that often eliminate the need to visit the original sources of information.
“Answer engines don’t drive traffic,” Prince said. “Search engines were the engine that drove revenue on the web. If there’s no traffic, then the existing ecosystem, based on the current business model, falls apart.”
A precipitous drop
The data proves this is happening. Barclays analysts recently shared charts showing a precipitous drop in referrals and visits this year to websites in categories such as publishing, e-commerce, travel, fashion, finance, and food and drink.
This is happening while AI bots from tech giants scrape websites much more often, scooping up their data for free without permission and spiking traffic-related costs.
With AI summarization taking off, Google now crawls 18 web pages for every 1 user it sends to a site. That’s up from a ratio of 2 to 1 a decade ago, according to data compiled by Barclays.
“A year ago, it was about five times harder to get traffic from Google,” Prince said. “Now it’s 10 times harder.”
OpenAI’s crawl-to-send ratio is nearly 100 times worse than Google, while Anthropic is even worse, data compiled by Barclays shows.
“We are at a turning point,” Prince said.
Why Prince cares
Prince seems to be the only major tech CEO trying to address this growing crisis, or even caring about it.
There’s a reason. Most Big Tech and AI companies have an incentive to downplay the value of data in models, chatbots, and related products. They’re spending billions of dollars on GPUs, data centers, and talented researchers — the last thing they want to do is pay for data, too.
Cloudflare is in a different position. It’s an infrastructure, security, and software company that helps run about 20% of the internet. The company does well when the web thrives, and vice versa.
“This is an existential threat to the internet,” Prince said. “If the internet’s business model breaks, that’s not great for Cloudflare.”
Controversial moves
Earlier this year, Cloudflare took a bold step: it began blocking AI bots by default and created a system that pushes AI companies to pay websites for access to their content. In essence, it’s transforming what used to be a one-sided relationship — where tech companies took data for free — into a market transaction.
These moves have been controversial. Guillermo Rauch, founder and CEO of AI startup Vercel, called it “blocking progress” this week.
Perplexity, a startup that’s building an AI answer engine, tried to evade the new digital blockade, according to Cloudflare. Perplexity said Cloudflare is “overblocking,” which undermines user choice and access to innovative services competing with “established giants.”
“Cloudflare’s leadership is either dangerously misinformed on the basics of AI, or simply more flair than cloud,” the startup wrote in a spicy blog post.
Google, the reluctant AI kingmaker
Still, according to Prince, most tech companies have been surprisingly receptive to Cloudflare’s proposal that they pay for the data that powers their AI answer engines.
“Every AI company that’s a long-term thinker understands that at some point they will have to pay for original content,” the CEO said. “Google is a long-term thinker. OpenAI is. Want to bet which company still exists in 10 or 20 years? Google will be here, and OpenAI will be. Perplexity? I doubt it.”
Google is at the heart of this crisis, and potentially its solution. As the dominant gatekeeper of the internet, its move from search engine to AI-powered answers has had a dramatic, chilling effect on web traffic this year.
Despite this, Prince remains cautiously optimistic. “I do think Google will pay for content,” he said. “The only question is whether they do it voluntarily or are forced to.”
Either way, Google “understands that something needs to change,” Prince added, while noting the company has been mostly a huge force of good in the world.
Indeed, Google recently said it’s exploring and experimenting with new types of partnerships, after Bloomberg reported that the internet giant has been recruiting news organizations for a new licensing project related to AI. Google didn’t respond to a request for comment.
Prince said most of the tech companies Cloudflare has engaged with say they will pay for content, as long as there’s a level playing field.
“What they mean by that is that they will do it as long as Google does it, too,” he added. “If Google goes, everyone else will follow.”
A vision of three AI futures
Right now, the tech industry is in a stand-off over these questions. The future could play out in three ways, according to Prince.
1. Content Collapse: No sustainable business model emerges, original content withers, and the web becomes a wasteland of AI slop generated by robots and drones.
2. Oligarchic Control: All content creators work for a handful of tech giants. Like the Medicis of the Renaissance, these companies become the sole patrons of knowledge, controlling what gets created, what gets distributed, and to whom. “There will be a conservative US AI, a liberal AI, a Chinese AI, and probably an Indian AI. Europe will try too, but it will probably just use the liberal US AI,” Prince said.
3. The Swiss Cheese Model: A more hopeful future where AI companies compensate creators to fill the holes in their giant knowledge bases. “The big AI companies have slurped up a huge amount of data, which, today, is the best approximation of human knowledge. Think of this as a giant block of Swiss cheese,” Prince explained. “If we can create a new business model that rewards creators for filling in the holes in this cheese, that’s a great outcome.”
Spotify > BuzzFeed
Prince finds inspiration in Spotify’s business model. The streaming giant often shares user-generated demand signals, and independent musicians create content to meet this. The system has generated billions of dollars for creators and has supported the continued production of high-quality original work.
That’s where AI content economics might go, with creators paid to supply bespoke answers to fill gaps in the AI knowledge universe, not just attention-grabbing headlines.
This would be better than the digital world that Google inadvertently created, according to Prince.
“At some level, everything that’s wrong with the world is Google’s fault,” the CEO said. “Google taught us to worship the traffic deity, which begot Facebook, TikTok, and the entire attention-economy black hole that we’ve fallen into.”
Prince recalls talking with a BuzzFeed executive who described how the online publisher would run A/B tests for different headlines to figure out which one generated the biggest cortisol response in readers.
“That’s bad. We don’t need that. And journalists don’t like writing this stuff, they want to write original, thoughtful work,” said Prince, who owns a local newspaper in Park City, Utah.
I didn’t have to guts to tell him in the interview that BI does these headline tests, too. (I’m telling you now, Matthew).
“Instead, what if we create a new business model that incentivizes creators to help fill gaps in knowledge and pay them for that,” Prince said. “That’s a better world — a less enraged and less violent world.”
Internally, Cloudflare refers to this initiative as “Act 4,” a new era for the company following its work in security, networking, and developer tools.
Prince is betting that this next act will define not only the company’s future but the internet’s, too.
Sign up for BI’s Tech Memo newsletter here. Reach out to me via email at [email protected].
Read the full article here
