4 ms·
It would indeed. Note the warning: "There is not currently a way to differentiate between web crawlers that are indexing sites for search purposes, vs crawlers
by BryantD 2y ago
It would indeed. Note the warning: "There is not currently a way to differentiate between web crawlers that are indexing sites for search purposes, vs crawlers that are training AI models. ANY SITE THIS SOFTWARE IS APPLIED TO WILL LIKELY DISAPPEAR FROM ALL SEARCH RESULTS."
- rvnx 2y agoIt's actually a great idea to spread malware without leaving traces too, it makes content inspection to be very difficult, view-source: to be broken and most of debugging tools, saving to .har, etc.
- bugtodiffer 2y agohow is view source broken
- rvnx 2y agoIt waits for the whole page to load
- jsheard 2y agoReal search engines respect robots.txt so you could just tell them not to enter Markov Chain Hell.
- throwaway744678 2y agoI suspect AI crawler would also (quickly learn to) respect it also?
- deleted 2y ago[deleted]
- jsheard 2y agoIn that case, mission accomplished.
- deleted 2y ago[deleted]