5 ms·

It's way too complicated. But if this is all you have ever seen and if you've been designing such systems for a decade, this seems like normal to you. Here's a

by hsn915 4y ago

It's way too complicated. But if this is all you have ever seen and if you've been designing such systems for a decade, this seems like normal to you. Here's an alternative stack that can handle over 99% of websites: - Self contained executable - One-file database - Cache is memory - Text search is a library function - Indexing is a library function - Serving http is a library function Such a stack can handle > 20k concurent connections (per second). The code doesn't need to be "optimized"; just non-pessimized. You can scale "vertically" for a very long time, specially in 2020 and beyond, where you have machines with over 64 CPU cores. That's almost a cluster of 64 machines, except in one single machine. If you _must_ scale horizontally for some reason - maybe you are Twitter/Facebook/Google - then you can still retain the basic architecture of a single executable but allow it to have "peers" and allow data to be sharded/replicated across peers. Again all the coordination you need is just a library that you embed in the program; not some external monster tool like k8s.

kgeist 4y ago
There are several reliability issues: 1) a single panic/exception/segfault in the executable brings down the whole website and so it will be unavailable until the executable restarts 2) entropy *always* increases (RAM usage, memory corruption, hardware issues, OS misconfiguration etc.) so eventually the application will break and stop serving traffic until it's repaired/restarted (which can take time if it's a hardware issue) 3) deployments are tricky if there's nothing before the executable (stop, update, restart => downtime) 4) if cache is in-process, on a restart it will have to be repopulated from scratch, leading to temporary slowdowns (+ and maybe a thundering herd problem) which will happen *every time* you deploy an update I think much of it is ignoreable if the site is just a personal blog or a static site. But if the site is a real time "web application" which people rely on for work, then you still need: 1) some kind of containerization, to deal with inevitable entropy (when a container is restarted, everything is back to the initial clean state) 2) at least two instances of the application: one instance crashes => the second one picks up traffic; or during rolling updates: while one instance is being killed and replaced with a new version, traffic is routed to another instance 3) persistent data (and sometimes caches) need to be replicated (and backed up) -- we've had many hardware issues corrupting DBs 4) automatic failover to a different machine in case the machine is dead beyond repair >not some external monster tool like k8s What can you use instead of k8s for this kind of scenario? (an ultra reliable setup which doesn't need a whole cluster)
- hsn915 4y ago
  Re: Crashes. If there's a bug that brings the server down, it will happen in all instances and repeatedly no matter how many times you restart. Specially when the users keep repeating the action that triggered the crash. Re: Entropy. Entropy increases with complex setup. The whole point of not having a complex setup is to reduce entropy and make the system as whole more predictable. Re: caches. There are two types of caches: indicies that are persisted with the database, and LRU caches in memory. LRU caches are always built on demand so this is not even a problem. Plus modern CPUs are incredibly fast and can process several GBs of data per second. Even in the worst cases, you should be able to rebuild all your caches in a second.
  - kgeist 4y ago
    >If there's a bug that brings the server down, it will happen in all instances and repeatedly no matter how many times you restart. Not necessarily so. Many bugs are pretty rare bugs which are triggered only under specific conditions (a user, or the system, must do X, Y, Z at the right moment). So it doesn't happen all the time. But when it happens, the whole server crashes or starts behaving in a funky way and other users are affected. Sure you may say if it's a rare bug, then users will be rarely affected. But we don't have a single bug like that, there's always N such bugs lurking around (we never know how many of them in a large application); multiply it by N bugs and you have server crashes for different reasons quite often, making your paying customers dissatisfied. It also assumes you can fix such a bug immediately while it's not always true, there's often Heisenbugs it takes weeks to root out and fix, while your customers are affected (sure the application will restart but ALL users (not just the one who triggered the bug) can loose work, get random errors when the app is not available -- not a good experience). So having several app instances for backup allows to soften such blows, because there will always be at least one app instance which is available. >Entropy increases with complex setup. The whole point of not having a complex setup is to reduce entropy and make the system as whole more predictable I agree that entropy increases with complex setup, but there's also base entropy which accumulates simply because of time (which I think is more dangerous). Like make a sufficient number of changes to the setup of your application (which you often need if you release often) and eventually someone or something somewhere will make a mistake or expose a bug somewhere, and you will need to repair it and you won't be able do it easily because your setup is not containerized which would allow to return to the clean state quite easily with no effort. We've had issues like that with our non-containerized deployments and it's a very complex and error-prone undertaking to do it flawlessly (no downtime or regressions) compared to containerized deployments. >Plus modern CPUs are incredibly fast and can process several GBs of data per second. Even in the worst cases, you should be able to rebuild all your caches in a second Hm, usually caches are placed in front of disk-based DB's to speed up I/O, i.e. it's not a matter of slow CPU's, it's a matter of slow I/O. Rebuilding everything which is in the caches from DB sources is not super fast.
porker 4y ago
> - Text search is a library function What text search will provide me with the same features as Elasticsearch? Index time analysis, stemming, synonyms; search time expansions, prefix matching, filtering and (as a separate feature) type ahead autocomplete? I would love to never touch another Elasticsearch cluster so this is a genuine question.
- hsn915 4y ago
  What about any of this prohibits it from being a library? https://lucene.apache.org/core/ https://lucene.apache.org/core/ This is the Java library that ES is based on. Without even having to look at it I can make the following judgement: It should be easy to port to any language. It's open source, and it's Java. Java has no special features that makes it impossible or particularly difficult to replicate this functionality in any other compiled language, like C, Rust, Go, or any other language that is not 100x wasteful of system resources.
  - tm-guimaraes 4y ago
    Go is not less wasteful than java, both are garbage collected and their memory pressure depend highly on the given workload, and the runtime of the program. But java allow more GC tuning and even different GCs for different use cases (ie: shenadoah and ZGC favor very low latency workloads, while the default G1GC favors throughout (not that simple, but you get the point)) Regardless, Java/Go tier of performance is good enough for this kind of thing.
    - hsn915 4y ago
      I was referring to Ruby/Python when I said 100x wasteful languages.
  - porker 4y ago
    > This is the Java library that ES is based on. Based on, but Elasticsearch is not just a server wrapped around the library. Features ES has are not in Lucene, otherwise anyone could release a competitor by wrapping the library. > It should be easy to port to any language. You win the "Most Hacker News comment of March 2023" award. This thread is talking about less effort, and you bring up porting Lucene to another programming language.
hardware2win 4y ago
>- One-file database SQLite? If yes, then I dont really believe that you can have 20k concurrent users where significant part goes to db, not cache. But Ive been messing with just 1 vCore, so.
- hsn915 4y ago
  You can always put an LRU cache between you and SQLite. I personally moved from SQLite to a B-Tree based key-value store, and most requests can be serviced in ~500us (that is microseconds). I don't mean a request to the database: I mean a request from the client side that queries the database and fetches multiple objects from it. In my SQLite setup, the same kind of query would take 10ms (that is 20x the time) even _with_ accelerator structures that minimize hits to the DB. But you can always scale up vertically. You can pay $240/mo for 8 vCPUs with 32GB of RAM. Much cheaper than you would pay for an elastic cloud cluster.
  - osigurdson 4y ago
    >> ~500us (that is microseconds). 500us is slow. This kind of performance does not remotely obsolete an LRU cache (main memory access is ~5000X faster). 500us is essentially intra-datacenter latency. Obviously your data is in memory on the B-Tree server as there is no room in this budget for disk IO. Postgres will perform just as well if data is in memory hitting a hash index (even B-Tree probably). I don't think the B-Tree key-value store you mention is adding much. Use Redis or even just Postgres.
malborodog 4y ago
When you say text indexing and serving http are library functions, what do you mean? Also, is the language here go or what? Since you said python is too slow and then necessitates all the infra to manage it.
- hsn915 4y ago
  Go or any language that actually gets compiled down to machine code to get executed directly on the hardware, and where libraries are compiled into the final product. When I say something is a library function, I mean you just compile it into your code. In your code, you just call the function. This is in contrast to the current defactor practice of making an http request to ask another program (potentially on a different machine) to do the work.
  - malborodog 4y ago
    Beautiful. Got it; thank you.

by hsn915 4y ago

kgeist 4y ago

There are several reliability issues: 1) a single panic/exception/segfault in the executable brings down the whole website and so it will be unavailable until the executable restarts 2) entropy *always* increases (RAM usage, memory corruption, hardware issues, OS misconfiguration etc.) so eventually the application will break and stop serving traffic until it's repaired/restarted (which can take time if it's a hardware issue) 3) deployments are tricky if there's nothing before the executable (stop, update, restart => downtime) 4) if cache is in-process, on a restart it will have to be repopulated from scratch, leading to temporary slowdowns (+ and maybe a thundering herd problem) which will happen *every time* you deploy an update I think much of it is ignoreable if the site is just a personal blog or a static site. But if the site is a real time "web application" which people rely on for work, then you still need: 1) some kind of containerization, to deal with inevitable entropy (when a container is restarted, everything is back to the initial clean state) 2) at least two instances of the application: one instance crashes => the second one picks up traffic; or during rolling updates: while one instance is being killed and replaced with a new version, traffic is routed to another instance 3) persistent data (and sometimes caches) need to be replicated (and backed up) -- we've had many hardware issues corrupting DBs 4) automatic failover to a different machine in case the machine is dead beyond repair >not some external monster tool like k8s What can you use instead of k8s for this kind of scenario? (an ultra reliable setup which doesn't need a whole cluster)

hsn915 4y ago
Re: Crashes. If there's a bug that brings the server down, it will happen in all instances and repeatedly no matter how many times you restart. Specially when the users keep repeating the action that triggered the crash. Re: Entropy. Entropy increases with complex setup. The whole point of not having a complex setup is to reduce entropy and make the system as whole more predictable. Re: caches. There are two types of caches: indicies that are persisted with the database, and LRU caches in memory. LRU caches are always built on demand so this is not even a problem. Plus modern CPUs are incredibly fast and can process several GBs of data per second. Even in the worst cases, you should be able to rebuild all your caches in a second.
- kgeist 4y ago
  >If there's a bug that brings the server down, it will happen in all instances and repeatedly no matter how many times you restart. Not necessarily so. Many bugs are pretty rare bugs which are triggered only under specific conditions (a user, or the system, must do X, Y, Z at the right moment). So it doesn't happen all the time. But when it happens, the whole server crashes or starts behaving in a funky way and other users are affected. Sure you may say if it's a rare bug, then users will be rarely affected. But we don't have a single bug like that, there's always N such bugs lurking around (we never know how many of them in a large application); multiply it by N bugs and you have server crashes for different reasons quite often, making your paying customers dissatisfied. It also assumes you can fix such a bug immediately while it's not always true, there's often Heisenbugs it takes weeks to root out and fix, while your customers are affected (sure the application will restart but ALL users (not just the one who triggered the bug) can loose work, get random errors when the app is not available -- not a good experience). So having several app instances for backup allows to soften such blows, because there will always be at least one app instance which is available. >Entropy increases with complex setup. The whole point of not having a complex setup is to reduce entropy and make the system as whole more predictable I agree that entropy increases with complex setup, but there's also base entropy which accumulates simply because of time (which I think is more dangerous). Like make a sufficient number of changes to the setup of your application (which you often need if you release often) and eventually someone or something somewhere will make a mistake or expose a bug somewhere, and you will need to repair it and you won't be able do it easily because your setup is not containerized which would allow to return to the clean state quite easily with no effort. We've had issues like that with our non-containerized deployments and it's a very complex and error-prone undertaking to do it flawlessly (no downtime or regressions) compared to containerized deployments. >Plus modern CPUs are incredibly fast and can process several GBs of data per second. Even in the worst cases, you should be able to rebuild all your caches in a second Hm, usually caches are placed in front of disk-based DB's to speed up I/O, i.e. it's not a matter of slow CPU's, it's a matter of slow I/O. Rebuilding everything which is in the caches from DB sources is not super fast.

porker 4y ago

> - Text search is a library function What text search will provide me with the same features as Elasticsearch? Index time analysis, stemming, synonyms; search time expansions, prefix matching, filtering and (as a separate feature) type ahead autocomplete? I would love to never touch another Elasticsearch cluster so this is a genuine question.

hsn915 4y ago
What about any of this prohibits it from being a library? https://lucene.apache.org/core/ https://lucene.apache.org/core/ This is the Java library that ES is based on. Without even having to look at it I can make the following judgement: It should be easy to port to any language. It's open source, and it's Java. Java has no special features that makes it impossible or particularly difficult to replicate this functionality in any other compiled language, like C, Rust, Go, or any other language that is not 100x wasteful of system resources.
- tm-guimaraes 4y ago
  Go is not less wasteful than java, both are garbage collected and their memory pressure depend highly on the given workload, and the runtime of the program. But java allow more GC tuning and even different GCs for different use cases (ie: shenadoah and ZGC favor very low latency workloads, while the default G1GC favors throughout (not that simple, but you get the point)) Regardless, Java/Go tier of performance is good enough for this kind of thing.
  - hsn915 4y ago
    I was referring to Ruby/Python when I said 100x wasteful languages.
- porker 4y ago
  > This is the Java library that ES is based on. Based on, but Elasticsearch is not just a server wrapped around the library. Features ES has are not in Lucene, otherwise anyone could release a competitor by wrapping the library. > It should be easy to port to any language. You win the "Most Hacker News comment of March 2023" award. This thread is talking about less effort, and you bring up porting Lucene to another programming language.

hardware2win 4y ago

>- One-file database SQLite? If yes, then I dont really believe that you can have 20k concurrent users where significant part goes to db, not cache. But Ive been messing with just 1 vCore, so.

hsn915 4y ago
You can always put an LRU cache between you and SQLite. I personally moved from SQLite to a B-Tree based key-value store, and most requests can be serviced in ~500us (that is microseconds). I don't mean a request to the database: I mean a request from the client side that queries the database and fetches multiple objects from it. In my SQLite setup, the same kind of query would take 10ms (that is 20x the time) even _with_ accelerator structures that minimize hits to the DB. But you can always scale up vertically. You can pay $240/mo for 8 vCPUs with 32GB of RAM. Much cheaper than you would pay for an elastic cloud cluster.
- osigurdson 4y ago
  >> ~500us (that is microseconds). 500us is slow. This kind of performance does not remotely obsolete an LRU cache (main memory access is ~5000X faster). 500us is essentially intra-datacenter latency. Obviously your data is in memory on the B-Tree server as there is no room in this budget for disk IO. Postgres will perform just as well if data is in memory hitting a hash index (even B-Tree probably). I don't think the B-Tree key-value store you mention is adding much. Use Redis or even just Postgres.

malborodog 4y ago

When you say text indexing and serving http are library functions, what do you mean? Also, is the language here go or what? Since you said python is too slow and then necessitates all the infra to manage it.

hsn915 4y ago
Go or any language that actually gets compiled down to machine code to get executed directly on the hardware, and where libraries are compiled into the final product. When I say something is a library function, I mean you just compile it into your code. In your code, you just call the function. This is in contrast to the current defactor practice of making an http request to ask another program (potentially on a different machine) to do the work.
- malborodog 4y ago
  Beautiful. Got it; thank you.