Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
ghostlyy
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
2 ms
·
1.
▲
by
ghostlyy
5mo ago
partial answer: the major labs (Anthropic, OpenAI) do respect robots.txt for their named crawlers, so blocking ClaudeBot/GPTBot in robots.txt works for those specific bots. What you can't easily opt out of is the indirect ingestio
2.
▲
by
ghostlyy
5mo ago
Timing's also worth nothing. the investments piece has been reported on for over a year. It becomes a probe right before liquidity, which makes both sides look opportunistic rather than principled.