4 ms·
Sure, overly spamming websites is shitty behaviour. But blocking AI crawlers hurts you in the end. Guess what will replace SEO in the long run?
by RamblingCTO 2y ago
Sure, overly spamming websites is shitty behaviour. But blocking AI crawlers hurts you in the end. Guess what will replace SEO in the long run?
- entropi 2y ago>But blocking AI crawlers hurts you in the end. Guess what will replace SEO in the long run? Maybe. But even if that turns out to be true, what good is it for the source website? The "AI" will surely not share any money (or anything else that may help the source website) with the source anyways. Why would they, they already got the content and trained on it.
- RamblingCTO 2y agoWhat good is it? If "AI" doesn't know about you down the line, you won't be discovered. Be it in LLM weights or via crawling (perplexity, jina reader etc.), you won't get any organic traffic. It's not about sharing profits.
- entropi 2y agoAgain, the "AI" doesn't care about the website. It doesn't even link to it in vast majority of the cases. Even if it did, the "AI" derives a lot of its business value from the fact that it is providing what the client requests while removing the need to visit potentially dozens of these pages. So the clients, in most cases, would not even click them (as they already got what they wanted).
- xena 2y agoSo you're willing to pay my hosting bills?
- RamblingCTO 2y agoRe-read what I posted and stop projecting.
- xena 2y agoBut if the AI crawlers are taking the website down and money buys more server time, are you willing to do your part and use money to make sure your training data sources are solvent until you can replace them?
- RamblingCTO 2y agoI am not training shit dude ... So stop projecting. And again: spamming is unfair, as I said.
- CyberDildonics 2y agoWhat a bizarre response. They are just saying that some of these crawlers are being used for search engines in one way or another.