3 ms·
Hmm, I wondered a while back if anyone was keeping track of these. It seems to be a manual list so far, so added my main server, as that's been transmitting the
by Jaruzel 11y ago
Hmm, I wondered a while back if anyone was keeping track of these. It seems to be a manual list so far, so added my main server, as that's been transmitting the clacks overhead since he died. Can't wait for the crawler to go live and then we'll see how many people did add it, including any big name sites.
- jonathanmh 11y agoHi Jaruzel, the crawler is live :D you can see in the bottom which page it is crawling right now. It's basically scraping all links on every page it hits and tests the headers if they are containing the clacks value. I added the form so people can submit pages to speed up the development of the list, even though I believe eventually the crawler would get to their pages :D
- Jaruzel 11y agoCool! What software are you using for the crawler?
- jonathanmh 11y agoI built a not very advanced one with node.js and mongodb :) Mainly in use in the crawler: * request * cheerio