When The Bots Outnumbered Us

Here's what's on our plate today:

• 🤖 The web's main visitor just flipped from people to bots, and nobody sent an invoice.

• 📱 SpaceX's mystery AI handset, Wall Street learning tokenomics, and OpenAI's 5% offer to Washington.

• 🧰 Weekend To-Do: audit your referral traffic, read the Cloudflare data, and test the AI-summary click drain.

• 🗳️ Poll: If machines are the web's main audience now, who pays for the content?

Let’s dive in. No floaties needed.

The Laboratory

TL;DR

The internet's main visitor just switched from people to bots, and nobody sent an invoice.

• The crossover happened early: Cloudflare's Matthew Prince revealed that automated requests now make up 57.5% of web traffic, against 42.5% from humans, a shift he expected to arrive in 2027, not 2026.

• AI crawlers take without giving back: Anthropic's crawler scraped Cloudflare's network roughly 73k times for every referral sent back, OpenAI's about 1,700 times, and Google's 14 times, per June 2025 data.

• AI answers are killing clicks: Pew Research found users click a traditional link just 8% of the time when Google shows an AI summary, down from 15%, and click a cited source only 1% of the time.

• The fix is a paywall for bots: Cloudflare's Pay-per-Crawl lets publishers charge AI crawlers per visit, reviving the web's dormant "402 Payment Required" code.

• The stakes: if machines become the web's primary audience, the ad-and-attention economy that funded publishing for three decades has no obvious replacement.

When the web's main audience became machines

Since the internet's earliest days, the web has operated on a simple assumption: that at the end of every page request was a person. Every click represented someone who might read an article, watch a video, discover a product, remember a brand, or return the next day. That expectation shaped not only how websites were built but also how the internet's economy evolved.

As the web became the backbone of modern communication, this assumption turned it into one of the most valuable channels for businesses to reach potential customers. Advertising rates were determined by human traffic, search engines struck revenue-sharing agreements with publishers on the promise of sending human visitors, and the entire digital publishing industry came to measure success through page views, unique visitors, and engagement metrics. Publishers created content for readers, search engines delivered those readers, and advertising financed the entire ecosystem.

For decades, almost every part of the internet's business model rested on the idea that a website visit meant a human being had arrived. Then, on June 3, 2026, the company hosting a substantial share of the world's websites released a statistic suggesting that this foundational assumption had quietly begun to break down.

A milestone that arrived early

Matthew Prince, the chief executive of Cloudflare, the internet infrastructure firm that routes and protects traffic for millions of sites, shared data from the company's measurement dashboard showing that automated systems now generate 57.5% of requests for web content, against 42.5% from people. "Welp, that happened faster than I predicted," he wrote, noting that he had told an audience only months earlier that the crossover would not arrive until 2027.

The figure comes with an important caveat that Prince himself acknowledged: the underlying data are “a bit messy.” Cloudflare measures HTTP requests, the individual calls a browser or program makes to load a webpage, rather than the amount of time people spend online or the attention they give to content. By those measures, humans still dominate the internet. People continue to spend far more time watching videos, scrolling through social media, and using apps than bots do. The milestone, therefore, reflects the composition of web traffic, not the number of people online.

What the data does reveal, however, is a fundamental change like that traffic. A decade ago, most automated requests came from search engine crawlers quietly indexing websites so they could appear in search results. Today's growth is increasingly being driven by agentic AI, software that browses the web on behalf of assistants such as ChatGPT and Gemini. The difference between the two lies in scale. A person researching a new camera might visit five or six websites before making a purchase. An AI agent given the same task could visit thousands of pages, compare specifications, read reviews, and gather information before returning a single answer. A single human request can now generate thousands of machine requests, creating a volume of web traffic the internet was never designed to handle.

When a visit stops paying for itself

The deeper significance of this shift extends well beyond web traffic itself. It signals a change in the economic model that has sustained the internet for decades. A human visit was the fundamental unit around which the web's economy was built, because a person might see the advertisement alongside an article, click through to the source, subscribe, or return later. A machine visit consumes the same content but generates little or no value.

The clearest evidence of this growing disconnect comes from Cloudflare's own measurements of how often AI companies retrieve content compared with how often they send readers back. According to the company's June 2025 data, reported by TechCrunch, Google's crawler scraped Cloudflare's network 14 times for every referral it returned. OpenAI's crawler did so roughly 1,700 times per referral, while Anthropic's crawler fetched content about 73k times per visitor it sent back. The content moves out; the visitor, and with it the opportunity to monetize that visitor, rarely comes back.

Publishers have given this fear a name: “Google Zero.” The term describes a future in which referral traffic from search dwindles toward nothing because AI-generated answers increasingly resolve users’ questions without requiring them to visit the websites that produced the information. It remains a scenario rather than a settled outcome, and Google disputes the premise, but the early evidence offers little comfort to businesses that depend on clicks.

A study by the Pew Research Center, which analyzed the browsing behavior of 900 U.S. adults, found that when Google displayed an AI-generated summary at the top of search results, users clicked a traditional web link only 8% of the time, compared with 15% when no summary appeared. They clicked on a source cited within the AI summary just 1% of the time. Browsing sessions were also more likely to end immediately, occurring on 26% of pages with an AI summary versus 16% of those without. In other words, the AI summary answers the question, while the website that supplied the answer often never sees the visitor.

Google, however, maintains that AI is not harming the broader web, arguing that more content is being published than ever before. But that response measures something different. Google points to the growing supply of online content, while publishers are concerned with the shrinking flow of traffic that has historically funded their creation.

Re-architecting the web for readers who never arrive

If AI agents are now visiting more webpages than humans, the obvious question becomes how those visits should be paid for. The web's economy was designed around human visitors who generated value by viewing advertisements, subscribing to publications, or making purchases. Machine visitors consume the same content but rarely create those downstream opportunities. As AI traffic grows, the infrastructure beneath the web is beginning to adapt to that new reality.

Cloudflare's response is a system called Pay-per-Crawl, which allows a website owner to charge an AI crawler for access to its content. The system revives an old and largely unused part of the web's plumbing: HTTP status code 402, labeled “Payment Required” and reserved for exactly this purpose since the protocol's early days. When a crawler requests a page, the publisher can choose to allow it, block it, or return a price. Cloudflare then acts as the merchant of record, collecting payment from the AI company and passing it on to the publisher.

The company sees this as the foundation of a more agentic web, where an AI research assistant is given a budget and negotiates access to the best sources on behalf of its user, paying its way one page at a time. That vision transforms a webpage from a freely accessible document into a metered resource. It also reflects a broader shift in how the web is being rebuilt, with machine-readable formats designed for AI agents rather than human readers, and crawler-blocking increasingly becoming the default, so that access is something websites explicitly grant rather than passively surrender.

For publishers, however, neither path is especially attractive. Opting out of AI crawling protects their content but risks making them invisible as more people rely on AI-generated answers instead of search results. Allowing crawlers in, meanwhile, helps feed the very systems that are reducing referral traffic. Licensing agreements have emerged, but they remain largely limited to major publishers with enough bargaining power to negotiate them. At the same time, lawsuits against AI companies for using copyrighted material for training, including the case brought by The New York Times, underscore how unsettled the question of compensation remains. Charging a small fee for every crawl is an elegant idea, but the web's long and unsuccessful history with micropayments suggests that persuading AI companies to reliably pay tiny amounts at internet scale will be far from straightforward.

Who the web is being written for

The open question underneath all of this is what a webpage is for once its primary reader is a machine. The web grew into a vast, freely linked library on the premise that publishing meant reaching people, and that reaching people could be turned into money through attention. A web whose majority audience is automated pulls at both halves of that premise at once, because machines do not see advertisements and do not form the brand loyalty that direct subscriptions depend on. Some of the strongest publishers are already responding by building direct relationships, newsletters, paid memberships, and audiences that arrive without a search engine in between, which is a rational hedge and also a quiet concession that the open, ad-supported web may be narrowing into something more enclosed.

The internet was built on the expectation that every page request represented a person, and that, somewhere behind every click, lay the possibility of attention, influence, or commerce. That expectation became so fundamental that it was almost invisible. The figures Cloudflare published in June do not mark the end of the human internet, but they do suggest the end of that assumption. If the web's most frequent visitors are increasingly machines rather than people, then the question is no longer simply who is reading the internet. It is what the internet is ultimately built for.

Headlines You Actually Need

• SpaceX's phone-ish AI gadget: The WSJ reports SpaceX showed investors a handset-like AI device before its IPO, fueling talk it wants to go into wireless, though Musk called the report "utterly false."

• Wall Street learns tokenomics: Analysts are now tracking AI token spend like a commodity, with Citadel arguing adoption is driven by cost over capability as users flock to cheaper models.

• OpenAI offers Washington a cut: OpenAI has proposed handing the U.S. government a 5% stake, worth roughly $42.6 billion, to ease mounting political pressure, per the Financial Times.

Friday Poll

Weekend To-Do

• Check your own Google Zero exposure: Open your analytics and look at how much traffic still comes from search referrals versus direct and email; the gap tells you how vulnerable you are.

• Read the Cloudflare crawl-to-referral data: The June 2025 numbers on how often each AI crawler scrapes versus sends a visitor back are the clearest picture of the imbalance, and are worth ten minutes.

• Run the AI-summary test yourself: Ask a question you'd normally Google and watch whether the AI answer makes you click through at all; you'll feel the click-loss the Pew study measured firsthand.

Meme Of The Day

The Toolkit

• Regie.ai: AI sales agent that researches prospects, writes personalized outbound at scale, and handles follow-ups so reps can focus on closing.

• Replit AI: A browser-based coding environment with an AI agent that builds, debugs, and deploys apps from prompts, with no local setup required.

• Sourcegraph: Code intelligence platform with Cody, an AI assistant that understands your entire codebase so it can answer questions and write code that actually fits.

Rate This Edition