Kernel.org reports that AI-powered web crawlers are hammering its Git infrastructure, generating roughly six million daily requests for individual commit pages and consuming about 20 percent of its total CPU capacity. Many bots request HTML views of each commit instead of cloning the freely available repositories, multiplying traffic across millions of commits, numerous forks, and multiple commit, patch, diff, and plain-text URL variants. After IP blocking and proof-of-work defenses were undermined by proxy networks and adaptive crawlers, Kernel.org is weighing cuts to crawlable URLs and restrictions on resource-heavy actions for anonymous users while keeping repositories publicly accessible.
This update represents a notable development in the Ai sector. Organizations and founders tracking this space should evaluate potential strategic and technical implications on their operations.