Post Syndicated from corbet original https://lwn.net/Articles/1080822/

Our article “Fighting the AI scraper bot
scourge“, published in early 2025, discussed the problem of widespread
scraping of web sites in search of training data for large language models
and related projects. This activity overwhelms sites with traffic. Over a
year after that article is published, the problem is still growing. The
hammering of sites by shadowy actors has reached new heights, and the open
web is becoming increasingly difficult to maintain. Where is this traffic
coming from, and what can be done about it?