minus-squaredraughtcyclist@lemmy.worldtoTechnology@lemmy.zip•Cloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.linkfedilinkEnglisharrow-up5arrow-down1·1 month agoThis is amazing. The number of crawlers that don’t respect crawling directives is out of hand. Feed them garbage till they can’t pay their bills. linkfedilink
This is amazing. The number of crawlers that don’t respect crawling directives is out of hand. Feed them garbage till they can’t pay their bills.