2 ms·
Frankly, I don't understand why someone would even try to crawl Hacker News. There is an official dump which doesn't even require parsing HTML at all: https://
by maxloh 10mo ago
Frankly, I don't understand why someone would even try to crawl Hacker News.
There is an official dump which doesn't even require parsing HTML at all: https://console.cloud.google.com/marketplace/details/y-combinator/hacker-news?pli=1 https://console.cloud.google.com/marketplace/details/y-combi...
- dang 10mo agoThese are not, er, experienced crawlers. https://www.youtube.com/watch?v=Sbpl3ywNlpA#t=56s https://www.youtube.com/watch?v=Sbpl3ywNlpA#t=56s