AI vs search engine: crawl to referral ratio

[Brad]

https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar/

[rcjordan]

Just an hour ago, I told Louise that I strongly suspected that since retail searchers are a lazy bunch and would go for the low-hanging serp, they would just take the AI overview as the answer to their query.

[buckworks]

The AI overviews often contain errors. They are not to be trusted without further checking.

=========

A small way to fight back: beef up your robots.txt with this list to exclude AI crawlers:

https://perishablepress.com/ultimate-ai-block-list/#block-ai-via-robots

[ergophobe]

Nice that he has Apache and nginx rules that return 403 instead of assuming that his robots.txt directives will be honored.

Periodically I get high CPU warnings and based on the amount of money being poured into AI crawling and what a huge presence they have in logs last I looked, I wonder if that could explain it.

The AI overviews often contain errors. They are not to be trusted without further checking.

Human-generated overviews, assertions and "facts" often contain errors. As Aristotle wrote, "Don't trust everything you find on Instagram."