Table of contents

Crawler tarpits

%3 cluster_7f19bd94_0a57_4944_b722_e35897c4e942 Crawler tarpits _55c22b96_83c9_492b_9667_f16caceebac8 Nephentes _b8c09743_8f30_4276_8835_63993b733090 LLMs __0:cluster_7f19bd94_0a57_4944_b722_e35897c4e942->_b8c09743_8f30_4276_8835_63993b733090

Some programs used to trap bad behaving crawlers, suspected to be used to train LLMs. These programs create sprawling web mazes used to distract these crawlers away so they don't request more compute-intensive sites.