6 ms·

Wait, what, one of the best text search engines for the whole internet runs on a residential broadband connection on a home server? My mind is blown.

by _xnmw 5y ago

nicbou 5y ago
Sometimes I wonder what kind of scaling startups are scared of. I keep reading about popular websites running on a single server, and how cheap bare metal is compared to an equivalent AWS setup. Am I missing something?
- speed_spread 5y ago
  Serving static HTTP GETs is easy. The moment you start trying to make money off the generated traffic its no longer a read only affair. Your system is not stateless anymore and tracking costs spiral up quickly.
  - marginalia_nu 5y ago
    I largely agree with the analysis, with the caveat that a search engine is a bit more involved than serving static GETs.
    - speed_spread 5y ago
      Sorry, I lost context and was not thinking of the search engine but of a statically generated blog with small pages and little media. Also, thanks for confirming my hunch about the bigger factor being statefulness rather than dynamicity. CPU, RAM storage are incredibly cheap nowadays. But so much can be going on at a time that _certainty_ is now what's pricey.
- marginalia_nu 5y ago
  I don't think most startups really understand how their systems scale, so assume they'll scale like a social media platform (which do not scale well at all). I attribute a lot of my performance to initially targeting very low powered hardware. I ran it on a raspberry pi cluster until the index was around 1 million documents (I think). Having those limitations early on excluded a lot of design mistakes that would have been a lot harder to correct later. Serving search requests is a embarrassingly parallel operation that could be very easily load balanced to different instances. If so Max Schrems hectored every other search engine out of Europe, and Ursula von der Leyen, taking pity on my living room search operation, sent me a bunch of money, I'd probably be able to deal with 1000x the traffic about as fast as I'd be able to lease space in a datacenter. I am a bit choked in terms of how big I can grow my index, though. I could probably double, maybe even a couple of times, but not too many times if I can't find serious optimizations.
  - nicbou 5y ago
    Hey, I just checked your search engine and it's really cool! I googled my own website and ended up finding a lot of non-samey content.
- jamesmishra 5y ago
  A lot of startups' products are write-intensive applications with the following requirements: - strong read-after-write consistency (or their users could receive inaccurate information) - as-high-as-possible availability (because they lose money whenever the site goes down) - low-latency operations that are difficult to optimize--such as recommendations, complex full-text search, or large graph traversal And engineering teams don't notice that their solutions are suboptimal until their product hits scale.
  - Nextgrid 5y ago
    > And engineering teams don't notice that their solutions are suboptimal until their product hits scale. A lot of engineering teams scale prematurely. Maybe their complicated engineering playground will indeed scale to 100M users, at the expense of being extremely complex and costly from an infrastructure point of view, where as a more "boring" stack may not scale to 100M but will handle up to 10M in a much simpler & cheaper way. The more complex solution makes sense if you're sure you'll hit 100M, but megalomaniac founders' dreams aside, this rarely happens and they end up dealing with the unnecessary complexity & expense of a stack whose scalability isn't actually required.
  - deleted 5y ago
    [deleted]
- Nextgrid 5y ago
  Startups don't always want the most cost-effective solution. A lot of startups' intentions is to look "serious". If everyone has an engineering playground full of microservices with 5 different programming languages all hosted on AWS, you have to be the same if you want to join the "big boys" club. Sometimes startups don't even care about solving the business problem. The objective is instead to get the "startup CEO" lifestyle for the connections and reputation regardless of whether you end up solving your business problem in the end. Same for a lot of engineers - they'll happily join and build the engineering playground while enjoying their salary regardless of whether it's the right option from an engineering point of view. If anything, an engineering playground gives you the opportunity to manage lots of people and solve (self-inflicted) problems that will look good on your resume in a way that a boring solution with a handful of engineers won't. On the other hand, if you live and die by solving the business problem the most cost-effective way, all the "scalability" goes out the window and a simple stack on top of bare-metal makes much more sense, because the only thing that pays you is solving the business problem (in the cheapest way possible) as opposed to bragging about your self-inflicted technical complexity at AWS conferences.
  - marginalia_nu 5y ago
    I wouldn't discount Hanlon's razor. There seems to be many people who simply fall for the marketing and think that kubernetes and hundreds of microservices are somehow necessary to run a small to medium sized web service. They can't comprehend how a technology stack with a 20 year track record could be even nearly as stable and reliable as a grab bag of 2 weeks old javascript microframeworks. It's absolutely inconceivable how something that performed well a decade ago on hardware that was a hundred times slower could keep up on modern servers.
    - anderspitman 5y ago
      And unfortunately the effect flows both ways. I'm having a heck of a time finding a simple, modern log aggregator not designed for kubernetes and infinite scale. I have 4 load balanced VMs and literally just need to pull the logs into a central location sorted by time. There's a shocking amount of complexity around all the popular tools in this space.
marginalia_nu 5y ago
Yup. Not even server hardware. If it wasn't for the unusual number of hard drives, 128 Gb RAM, $20 graphics card, and UPS, you could almost confuse it for a gaming rig.

Wait, what, one of the best text search engines for the whole internet runs on a residential broadband connection on a home server? My mind is blown.

by _xnmw 5y ago

Wait, what, one of the best text search engines for the whole internet runs on a residential broadband connection on a home server? My mind is blown.

nicbou 5y ago

Sometimes I wonder what kind of scaling startups are scared of. I keep reading about popular websites running on a single server, and how cheap bare metal is compared to an equivalent AWS setup. Am I missing something?

speed_spread 5y ago
Serving static HTTP GETs is easy. The moment you start trying to make money off the generated traffic its no longer a read only affair. Your system is not stateless anymore and tracking costs spiral up quickly.
- marginalia_nu 5y ago
  I largely agree with the analysis, with the caveat that a search engine is a bit more involved than serving static GETs.
  - speed_spread 5y ago
    Sorry, I lost context and was not thinking of the search engine but of a statically generated blog with small pages and little media. Also, thanks for confirming my hunch about the bigger factor being statefulness rather than dynamicity. CPU, RAM storage are incredibly cheap nowadays. But so much can be going on at a time that _certainty_ is now what's pricey.
marginalia_nu 5y ago
I don't think most startups really understand how their systems scale, so assume they'll scale like a social media platform (which do not scale well at all). I attribute a lot of my performance to initially targeting very low powered hardware. I ran it on a raspberry pi cluster until the index was around 1 million documents (I think). Having those limitations early on excluded a lot of design mistakes that would have been a lot harder to correct later. Serving search requests is a embarrassingly parallel operation that could be very easily load balanced to different instances. If so Max Schrems hectored every other search engine out of Europe, and Ursula von der Leyen, taking pity on my living room search operation, sent me a bunch of money, I'd probably be able to deal with 1000x the traffic about as fast as I'd be able to lease space in a datacenter. I am a bit choked in terms of how big I can grow my index, though. I could probably double, maybe even a couple of times, but not too many times if I can't find serious optimizations.
- nicbou 5y ago
  Hey, I just checked your search engine and it's really cool! I googled my own website and ended up finding a lot of non-samey content.
jamesmishra 5y ago
A lot of startups' products are write-intensive applications with the following requirements: - strong read-after-write consistency (or their users could receive inaccurate information) - as-high-as-possible availability (because they lose money whenever the site goes down) - low-latency operations that are difficult to optimize--such as recommendations, complex full-text search, or large graph traversal And engineering teams don't notice that their solutions are suboptimal until their product hits scale.
- Nextgrid 5y ago
  > And engineering teams don't notice that their solutions are suboptimal until their product hits scale. A lot of engineering teams scale prematurely. Maybe their complicated engineering playground will indeed scale to 100M users, at the expense of being extremely complex and costly from an infrastructure point of view, where as a more "boring" stack may not scale to 100M but will handle up to 10M in a much simpler & cheaper way. The more complex solution makes sense if you're sure you'll hit 100M, but megalomaniac founders' dreams aside, this rarely happens and they end up dealing with the unnecessary complexity & expense of a stack whose scalability isn't actually required.
- deleted 5y ago
  [deleted]
Nextgrid 5y ago
Startups don't always want the most cost-effective solution. A lot of startups' intentions is to look "serious". If everyone has an engineering playground full of microservices with 5 different programming languages all hosted on AWS, you have to be the same if you want to join the "big boys" club. Sometimes startups don't even care about solving the business problem. The objective is instead to get the "startup CEO" lifestyle for the connections and reputation regardless of whether you end up solving your business problem in the end. Same for a lot of engineers - they'll happily join and build the engineering playground while enjoying their salary regardless of whether it's the right option from an engineering point of view. If anything, an engineering playground gives you the opportunity to manage lots of people and solve (self-inflicted) problems that will look good on your resume in a way that a boring solution with a handful of engineers won't. On the other hand, if you live and die by solving the business problem the most cost-effective way, all the "scalability" goes out the window and a simple stack on top of bare-metal makes much more sense, because the only thing that pays you is solving the business problem (in the cheapest way possible) as opposed to bragging about your self-inflicted technical complexity at AWS conferences.
- marginalia_nu 5y ago
  I wouldn't discount Hanlon's razor. There seems to be many people who simply fall for the marketing and think that kubernetes and hundreds of microservices are somehow necessary to run a small to medium sized web service. They can't comprehend how a technology stack with a 20 year track record could be even nearly as stable and reliable as a grab bag of 2 weeks old javascript microframeworks. It's absolutely inconceivable how something that performed well a decade ago on hardware that was a hundred times slower could keep up on modern servers.
  - anderspitman 5y ago
    And unfortunately the effect flows both ways. I'm having a heck of a time finding a simple, modern log aggregator not designed for kubernetes and infinite scale. I have 4 load balanced VMs and literally just need to pull the logs into a central location sorted by time. There's a shocking amount of complexity around all the popular tools in this space.

marginalia_nu 5y ago

Yup. Not even server hardware. If it wasn't for the unusual number of hard drives, 128 Gb RAM, $20 graphics card, and UPS, you could almost confuse it for a gaming rig.