3 ms·
> In reality, at home, we have some serious CPU horsepower just spinning billions of empty cycles per second. Consumer hardware is insane. Yeah. I've got my se
by marginalia_nu 5y ago
> In reality, at home, we have some serious CPU horsepower just spinning billions of empty cycles per second. Consumer hardware is insane.
Yeah. I've got my search engine basically a beefy gaming PC with no graphics card and 128 Gb ram. I've not only gotten supposed HN death hugs multiple times, I've had my search engine see multiple queries per second without as much as flinching. It took Elon Musk tweeting a link to one of my blog entries before I started getting more traffic than my computer could handle and it started dropping a few connections.
Modern consumer PCs are ridiculously powerful if you make good use of the hardware.
- mcovalt 5y agoVery cool! Running the not-well-maintained https://hndex.org https://hndex.org search engine (and other memory hungry linear algebra based services) was also my original motivation for tunneling to my home as opposed to hosting on a VPS. Are you hosted via a residential ISP? It's my hunch that peering agreements favor routes of consumer -> data center -> consumer as opposed to consumer -> consumer. That's mainly why I tunnel. Has that been your experience?
- agencies 5y agoNice! Have you considered publishing your crawl data?
- marginalia_nu 5y agoYeah it's on residential broadband. Haven't really had much trouble to be honest. Though I'm based in Sweden and we have fairly well robust networking infrastructure all around, I guess that may be a factor. I was hit by a botnet after my first big HN hug, so right now at least the search engine goes visitor->cloudflare->server, but if anything that's just added a bunch of ping. I'm also doing crawling and so on on the same network and it's really not bad for that either. Granted my crawls are fairly slow and very far from saturating any connections.