3 ms·
Noob question, I recall https://news.ycombinator.com/item?id=49491791 https://news.ycombinator.com/item?id=49491791 mentioned how the kernel source code and dif
by ghm2199 22d ago
Noob question, I recall https://news.ycombinator.com/item?id=49491791 https://news.ycombinator.com/item?id=49491791 mentioned how the kernel source code and diffs were being crawled excessively. All else being equal, won't all the AI crawling agents mean traffic to docs should at least be stable(presuming changes happen all the time)?
- ghm2199 22d agoAnd also the fact that agents are solving anubis level 5 challenges Tailwind would not be able to figure out how much traffic is from people vs bots. So that would mean the traffic should be up...
- samtheprogram 22d agoTailwind is so pervasive it's 100% in pretraining _and_ RLHF. I've done several Tailwind projects and rarely did models from various providers need to reference Tailwind documentation.
- simonw 22d agoA crawler visiting your docs won't help a human decide to pay for your commercial service advertised on those docs.