Reddit's Paywall Doesn't Control Distribution. It Controls Price.

What Reddit actually did

On July 1, 2024, Reddit rewrote its robots.txt to shut out nearly everything. Not a tightening, a wall. Search engines, AI crawlers, commercial scrapers: blocked by default. Reddit carved out an exception for what it called good faith actors, naming the Internet Archive, for non-commercial use.

Microsoft was not amused. Bing's crawler was among the blocked, and the two companies argued publicly about why. Reddit CEO Steve Huffman was unambiguous about the terms, telling Microsoft and AI search engines to pay if they wanted access.

This was, on its face, the boldest publisher move of the AI era. Every other site was arguing about whether to block GPTBot. Reddit blocked essentially everyone and put a price list on the door.

It is also the single best real-world test of the strategy half the publishing industry has been debating since, and it comes with unusually good conditions. Reddit had scale, a corpus nobody else owned, a public listing that forced it to report the consequences quarterly, and enough legal budget to sue anyone who went around it. If pay-to-crawl works anywhere, it works here. That is exactly why the results matter to everyone who does not own Reddit.

The exception that pays

Google kept crawling because Google had already signed. The February 2024 agreement, reported at around 60 million dollars annually, gave Google continued access and the right to train on Reddit content.

The commercial logic was clean. Reddit converts an unpriced input into a licensed one. Google removes uncertainty about a corpus it considers valuable. Both sides get something. And for two years the arrangement looked like proof that content licensing could work at scale.

It also created an asymmetry worth naming. One company crawls; everyone else is locked out unless they pay. That is a competitive structure as much as a commercial one, and it puts Reddit in the unusual position of having made a single search engine the sole legitimate gateway to its content. Reddit and Google have since reportedly discussed deepening the partnership, which would weave Reddit discussions further into Google's AI products.

Notice what that does to Reddit's negotiating position over time. The more thoroughly Reddit's content is embedded in Google's AI surfaces, the more Reddit depends on the counterparty it is trying to charge. Leverage that comes from being indispensable to one buyer is leverage right up until the moment you need to walk away.

The content escapes anyway

Here is where the strategy meets reality.

In October 2025, Reddit filed suit in the Southern District of New York against four companies. The allegation was not that they crawled Reddit. It was that they took Reddit content out of Google's search results, at what Reddit called industrial scale, then resold or reused it. As CNBC framed it at the time, the case expanded Reddit's data rights battle from the companies training on its content to the intermediaries supplying them.

That distinction is the whole story in miniature. Reddit had already solved the crawling problem. What it had not solved, and arguably cannot solve, is the resale problem.

The detail that makes the complaint stick is the trap. Reddit says it created a test post configured to be visible only to Google's crawler. Within hours, according to the filing, that post appeared in Perplexity's results. If accurate, that is not an inference about scraping. It is a tracer dye.

Perplexity rejected the characterisation, calling the suit "a show of force in Reddit's training data negotiations with Google and OpenAI" and arguing that Reddit's position is "the opposite of an open internet." Reddit had already sued Anthropic in June 2025, alleging its bots kept scraping more than 100,000 times after Anthropic said it had blocked them.

The plumbing is visible from the other end too. Search Engine Land has reported that SerpApi was or is an OpenAI customer, which helped explain how Google search results turned up inside ChatGPT. Separately, researchers reading ChatGPT's network traffic identified retrieval pipelines labelled bright and oxylabs and described them as scraped Google results purchased live. We went through that retrieval stack piece by piece in ChatGPT built its own search index.

Be careful with the inference here, because it matters. OpenAI holds its own Reddit licence, so its position is not Perplexity's. And a pipeline name in a network capture is not proof of what any given result contained. What the evidence does establish is the shape of the market: Google's index is a distribution layer, third parties sell access to it, and content inside it travels well beyond the parties who paid the original owner.

The accident that proved it

The strongest evidence is not the lawsuit. It is something nobody engineered.

Around September 10, 2025, Google removed the num=100 search parameter, which many SEO tools and data providers used to pull up to 100 results per request. It was a housekeeping change to Google's own product.

Reddit's citation share in ChatGPT collapsed within weeks, and Reddit's stock moved with the coverage. G2 growth advisor Kevin Indig argued, in hedged terms, that the num=100 removal made it harder for outside data providers to reach the deeper results where Reddit threads typically sit, and that this rather than any decision by OpenAI or Reddit was the likely cause.

Sit with that. Google changed a URL parameter. Reddit's visibility changed inside OpenAI's product. Neither Reddit nor OpenAI did anything.

That is only possible if Reddit's presence in ChatGPT was flowing through Google's results. Which means Reddit's robots.txt, the most aggressive on the open web, was not governing where Reddit content ended up. Google's index was.

A second decline followed in August 2026. The GEO analytics firm Promptwatch measured Reddit's ChatGPT Search citation share dropping from an average of 3.83 percent between July 18 and August 7 to 0.52 percent between August 14 and 17, an 86 percent relative fall.

Search Engine Journal's coverage is worth reading for its restraint. The popular explanation was a change on August 8 in how ChatGPT builds its background searches, when queries scoped to a single domain jumped from 0.37 percent to 16.8 percent of all fan-out queries in a day and the average number of fan-outs per response nearly doubled. But Reddit's share did not collapse until six days later, and Promptwatch itself called the magnitude provisional and said it could not rule out a data-collection issue on its own end.

So the second episode is genuinely unexplained, which is its own kind of finding. The honest summary is that Reddit's AI visibility has now swung violently twice, that on neither occasion did anyone at Reddit touch anything, and that on the one occasion where a cause was identified it was a housekeeping change at a company Reddit does not control.

The asset is worth less than it was

While all this was happening, the value of the thing Reddit sells has been quietly eroding.

Reddit's second quarter of 2026 was, on paper, excellent.

The cause was one line in Huffman's shareholder letter, reported by TechCrunch:

Search referrals were choppy in the quarter, and traffic was more volatile later in the quarter, but the bigger picture is unchanged: the commercial business is strong.

On the earnings call an analyst put it bluntly, saying investors sense "a user problem, especially in the U.S.," and spelling out the mechanism: logged-out traffic comes under pressure as search shifts to AI, which makes it harder to convert logged-out visitors into logged-in members. Then came the question that matters for this whole argument, whether Huffman saw "any world where you're not licensing data to Google and OpenAI next year." His answer was not reassuring to anyone modelling that revenue line: "I don't think there's a binary outcome."

It is worth being fair to Reddit's counterargument, because it is not weak. Huffman's position is that people come to Reddit for something a summary cannot deliver, telling CNBC that users "want the human perspective, the multiple perspectives, the lived experience that we provide" and that "people want Reddit, they don't necessarily want a summary of Reddit." The company has been pushing to convert web visitors into direct traffic for exactly this reason. If that works, the licensing revenue becomes a bonus rather than a dependency. The Q2 numbers say the commercial engine is genuinely strong. The US user line says the conversion is not finished.

He has been consistent about the underlying complaint. AI Overviews, he told CNBC, has "yet to make a similar level of positive impact" to the ten blue links, and he has said flatly that chatbots are not a traffic driver.

The industry data backs him. Cloudflare's crawl-to-refer ratios, reported by Search Engine Land, and TollBit's referral finding, also via Search Engine Land, tell the same story.

TollBit's version of the same point: Google sends 831 times more visitors than AI systems do.

Then in July 2026 came the report that Reddit might not renew the Google deal at all. The stock sank.

So Reddit is in an awkward position. Its leverage over Google was always partly the threat of withdrawal. But withdrawal costs Reddit the referral traffic it still depends on, from the one crawler with a defensible ratio, at a moment when its US user growth has stalled. And if it walks, the content does not become unreachable. It becomes unpriced, reachable by the same indirect routes the lawsuits are about.

What this means if you are not Reddit

Most sites will never be offered a licensing deal. That does not make this irrelevant, because Reddit ran the experiment everyone else is theorising about, with more leverage than any of us will ever have. We argued a year ago that Reddit is to GEO what DMOZ was to SEO, a proxy signal with an expiry date. This is what the expiry looks like from the inside.

Three findings transfer.

There is a fourth lesson, and it is the uncomfortable one. Reddit's visibility in AI answers has now moved twice for reasons that originated at Google, not at Reddit and not at the AI companies. If the most valuable user-generated corpus on the internet cannot hold its position in a third party's product against a routine parameter change, then nobody's AI visibility is as durable as their reporting implies. That is not an argument for giving up. It is an argument for building demand that survives a bad quarter on any single surface.

What to actually do

The useful question is not whether Reddit wins this negotiation. It is what your own plan looks like if the surface you rely on most changes its terms without asking you. Reddit is finding out that a wall only works when the thing behind it cannot be reached another way. Most of us do not have a wall to begin with, which makes the second half of that sentence the part to plan around. If you want that plan pressure-tested, it is the shape of our SEO and GEO work.

A note on sourcing: where a figure originates from a commercial analytics vendor, we credit the vendor by name in the text and link to the independent trade publication that reported it, rather than to the vendor's own page. Reddit's robots.txt is described here from that trade reporting rather than from a direct fetch.