3 ms·
> A search engine that forces people to "sign-in". I honestly think this may be what the future of search looks like. The only reason Google doesn't force you
by marginalia_nu 4y ago
> A search engine that forces people to "sign-in".
I honestly think this may be what the future of search looks like. The only reason Google doesn't force you to sign in is they already know who you are.
Operating a truly public and anonymous search engine is a serious uphill battle with the background noise of bot traffic. Looking at what Cloudflare turns down, I'm currently on the receiving end of about 1.5 million fake queries per day from a botnet[1]. It's more traffic than you typically get from lingering on the HN front page for a day or so, and these aren't just favicon.ico requests, but actual search queries.
Websites submitted to HN regularly keel over from this amount of traffic. In any other business this would be a low grade DDoS attack. In search we call it Tuesday. Now I run my own hardware and the worst that happens is my site becomes slow or drops out. If I was cloud backed, I'd be destitute from paying the bills.
[1] https://www.marginalia.nu/junk/cloudflare.webp https://www.marginalia.nu/junk/cloudflare.webp