Perplexity certainly was ignoring robots.txt [0] Anthropic... Their robots.txt requires a delay to be defined, even though its an optional extension. But whate

Perplexity certainly was ignoring robots.txt [0] Anthropic... Their robots.txt requires a delay to be defined, even though its an optional extension. But whatever. [0] https://www.wired.com/story/perplexity-is-a-bullshit-machine/ https://www.wired.com/story/perplexity-is-a-bullshit-machine...