Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
olivia-l
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
4 ms
·
1.
▲
by
olivia-l
2y ago
> Multiple IPs makes some sense since requests are being made through random proxies, and I don't think on its own demonstrates intention of bad behavior. Agreed, just using multiple IPs isn't malicious on it's own. I thou
2.
▲
by
olivia-l
2y ago
Not only do they not respect robots.txt, but they publish an entire page[1] in their docs dedicated to circumventing scraping countermeasures. I pointed their scraper at a url on my server to test it's behavior. It made four separate r