Googlebot once again generated more traffic than any other crawler in 2025, according to a new Cloudflare report. It outpaced every search and AI bot as Google continued crawling the web for search indexing and AI training.
By the numbers. Googlebot accounted for more than 25% of all Verified Bot traffic observed by Cloudflare.
- Googlebot alone generated 4.5% of all HTML request traffic – more than all other AI bots combined (4.2%).
- AI “user action” crawling surged more than 15x year over year, showing a sharp rise in bots that simulate human behavior.
- Googlebot’s crawl volume dwarfed every other AI crawler, including OpenAI, Anthropic, and Meta.
AI crawling surges. AI crawlers were the most frequently fully disallowed user agents in robots.txt files.
- Anthropic showed the highest crawl-to-refer ratio among major AI and search platforms, meaning it crawled far more content than it sent back as traffic. The ratio peaked near ~500,000:1 early in the year, then settled between ~25,000:1 and ~100,000:1 after May. For comparison:
- OpenAI spiked to ~3,700:1 in March.
- Perplexity was the lowest among major AI platforms. It started below 100:1, briefly jumped above 700:1 in late March during a PerplexityBot crawl spike, then stayed mostly below 400:1 and under 200:1 from September onward.
Search platforms looked very different:
- Microsoft hovered between ~50:1 and ~70:1 with a weekly cycle.
- Google rose from just over ~3:1 to ~30:1 by April, fell back to ~3:1 by mid-July, then gradually increased again.
- DuckDuckGo stayed below 1:1 for the first three quarters, then jumped to ~1.5:1 in mid-October and remained elevated.
Google still monopolizes search. Traditional search dominance barely changed.
- Google remained the top search engine by a wide margin, delivering nearly 90% of search engine referral traffic.
- Bing (3.1%), Yandex (2.0%), Baidu (1.4%), and DuckDuckGo (1.2%) rounded out the top five.
- Cloudflare saw minimal movement during the year.
- Google stayed dominant throughout.
- Yandex slipped from 2.5% in May to 1.5% in July.
- Baidu rose from 0.9% in April to 1.6% in June.
The report. The 2025 Cloudflare Radar Year in Review: The rise of AI, post-quantum, and record-breaking DDoS attacks
Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.
About the Author
