Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
taptaptapimin
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
3 ms
·
1.
▲
by
taptaptapimin
7y ago
I am no longer with this company, but it was a small web dev company that had a problem with insecure legacy code on just about all 200 of its clients. I wrote a software firewall to prevent the exploitation of these insecure websites as we
2.
▲
by
taptaptapimin
7y ago
This is true, but only technically. Google won't actively scrape anything disallowed in robots.txt, but those resources can still be indexed if found in the many other ways Google aggregates data, all of which is automated. Robots.txt