15 ms·

Many developers severely underestimate how much workload can be served by a single modern server and high-quality C++ systems code. I've scaled distributed work

Many developers severely underestimate how much workload can be served by a single modern server and high-quality C++ systems code. I've scaled distributed workloads 10x by moving them to a single server and a different software architecture more suited for scale-up, dramatically reducing system complexity as a bonus. The number of compute workloads I see that actually need scale-out is vanishingly small even in industries known for their data intensity. You can often serve millions of requests per second from a single server even when most of your data model resides on disk. We've become so accustomed to extremely inefficient software systems that we've lost all perspective on what is possible.

pleasecalllater 7y ago
Two. The number is two. You always need a backup server :)
- twic 7y ago
  I used to think that. But there are a couple of notable exceptions at either end of the latency spectrum. If your latency requirements are slack, then you can get away with one machine, because you can reboot or reprovision it and carry on processing without meet your requirements. If your latency requirements are tight, you don't have time to fail over anyway, so you might as well run one machine and make sure you can deal with failing to meet your requirements.
  - chucky_z 7y ago
    This is why I ran Redis on a single server with no failover strategy. Our team spent maybe 15 hours workshopping a handful of other things (redis cluster, sentinels + replicas) before realizing we could spend an hour to have everything sit in a degraded state working around it while Redis itself was fixed. Redis only failed once ever and it took all of an hour to fix it, all during non-peak hours.
- dboreham 7y ago
  Three. So you can do maintenance on one while still having HA.
  - fizwhiz 7y ago
    This guy SRE-s!
  - naniwaduni 7y ago
    No, that's still two. You don't do maintenance when you're not awake.
  - paulddraper 7y ago
    *Sometimes three. Go back to two when maintenance is done.
  - anon73044 7y ago
    I was expecting a "two is one and one is none" reference. Three keeps the investors happy in my case.
hnews_account_1 7y ago
Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point). Are you saying this can be optimised to fit inside a single 10 core server in terms of compute loads?
- sfifs 7y ago
  Don't know why you're being downvoted but I'll assume your question is genuine. You use a cluster when your data and compute requirements are large and parallel enough that the tax paid on network latency trumps the 10-20X speedup you get on SSD and 1000X speedup you get from just keeping data in RAM. 250 Gigs is tiny enough that you could probably much get better performance running on high memory instance in AWS or GCP. You'll generally have to write your own multiprocessing code though which is fairly simple - your existing library may also be able to support it. I once actually ran this kind of workload on just my laptop using a compiled language that performed better than pyspark on a cluster.
  - hnews_account_1 7y ago
    I'd love to keep it in RAM if I could. The problem is, the library I'm familiar with (pandas) typically seems to take more memory than the original csv file once it loads it onto memory. I know this is due to bad data types but in certain cases, I cannot get around those. However, even if I could load it all into memory at once, and assuming it takes 200 gb, I'm still using a master's student access to a cluster. So I get preempted like it's nobody's business. Hence why I prefer a smaller memory footprint even if I take up cpus at variable rates through a single execution. I did try to write my own multiprocessing code for this, but the operations are sometimes too complicated (like groupby) for me to rewrite everything from the ground up. If I'm not reliant on serial data communication between processes (like you'd need to sort a column), I can get it done pretty easily. In fact, I wrote my data cleaning code with this and cleaned up the entire file in half an hour because single chunks didn't rely on others. However, if you have some idea of how to run these computational loads in parallel in python or any other language on single compute instances (like the size of a laptop's memory of 16 gb), I'd really love to see it. Thanks.
remeq 7y ago
But isn't the containerization trend leading us to a completely opposite direction ie. scale-out by default and do that not only because of performance but mainly because of how you want to manage your production environment?

pleasecalllater 7y ago

Two. The number is two. You always need a backup server :)

twic 7y ago
I used to think that. But there are a couple of notable exceptions at either end of the latency spectrum. If your latency requirements are slack, then you can get away with one machine, because you can reboot or reprovision it and carry on processing without meet your requirements. If your latency requirements are tight, you don't have time to fail over anyway, so you might as well run one machine and make sure you can deal with failing to meet your requirements.
- chucky_z 7y ago
  This is why I ran Redis on a single server with no failover strategy. Our team spent maybe 15 hours workshopping a handful of other things (redis cluster, sentinels + replicas) before realizing we could spend an hour to have everything sit in a degraded state working around it while Redis itself was fixed. Redis only failed once ever and it took all of an hour to fix it, all during non-peak hours.
dboreham 7y ago
Three. So you can do maintenance on one while still having HA.
- fizwhiz 7y ago
  This guy SRE-s!
- naniwaduni 7y ago
  No, that's still two. You don't do maintenance when you're not awake.
- paulddraper 7y ago
  *Sometimes three. Go back to two when maintenance is done.
- anon73044 7y ago
  I was expecting a "two is one and one is none" reference. Three keeps the investors happy in my case.

hnews_account_1 7y ago

Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point). Are you saying this can be optimised to fit inside a single 10 core server in terms of compute loads?

sfifs 7y ago
Don't know why you're being downvoted but I'll assume your question is genuine. You use a cluster when your data and compute requirements are large and parallel enough that the tax paid on network latency trumps the 10-20X speedup you get on SSD and 1000X speedup you get from just keeping data in RAM. 250 Gigs is tiny enough that you could probably much get better performance running on high memory instance in AWS or GCP. You'll generally have to write your own multiprocessing code though which is fairly simple - your existing library may also be able to support it. I once actually ran this kind of workload on just my laptop using a compiled language that performed better than pyspark on a cluster.
- hnews_account_1 7y ago
  I'd love to keep it in RAM if I could. The problem is, the library I'm familiar with (pandas) typically seems to take more memory than the original csv file once it loads it onto memory. I know this is due to bad data types but in certain cases, I cannot get around those. However, even if I could load it all into memory at once, and assuming it takes 200 gb, I'm still using a master's student access to a cluster. So I get preempted like it's nobody's business. Hence why I prefer a smaller memory footprint even if I take up cpus at variable rates through a single execution. I did try to write my own multiprocessing code for this, but the operations are sometimes too complicated (like groupby) for me to rewrite everything from the ground up. If I'm not reliant on serial data communication between processes (like you'd need to sort a column), I can get it done pretty easily. In fact, I wrote my data cleaning code with this and cleaned up the entire file in half an hour because single chunks didn't rely on others. However, if you have some idea of how to run these computational loads in parallel in python or any other language on single compute instances (like the size of a laptop's memory of 16 gb), I'd really love to see it. Thanks.

remeq 7y ago

But isn't the containerization trend leading us to a completely opposite direction ie. scale-out by default and do that not only because of performance but mainly because of how you want to manage your production environment?