4 ms·

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

echelon 3y ago
The reasons to switch away from cloud keep piling up. We're doing some amount of on-prem, and I'm eager to do more.
- aarmenaa 3y ago
  I've previously worked for a place that ran most of their production network "on-prem". They had a few thousand physical machines spread across 6 or so colocation sites on three continents. I enjoyed that job immensely; I'd jump at the chance to build something like it from the ground up. I'm not sure if that actually makes sense for very many businesses though.
  - kuchenbecker 3y ago
    I'm getting that opportunity, however I expect it will be the last as most have migrated to the cloud and smaller companies are appealing to me; smallest company (of 5) I've worked for had 4.4k employees and large companies have the resources to roll their own. Unless there is an onprem movement I expect cloud to be the future as maintaining the tech stack onprem is difficult and we need to nake decisions down to the hardware we order.
malfist 3y ago
I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache. Samsung 990 Pro 2TB has a latency of 40 μs DDR4-2133 with a CAS 15 has a latency of 14 nano seconds. DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD. L1 cache is typically accessible in 4 clock cycles, in 4.8 ghz cpu like the i7-10700, L1 cache latency is sub 1ns.
- avg_dev 3y ago
  pretty cool comparisons. quite some differences there. tangent, I remember reading some post called something like "Latency numbers every programmer should know" and being slightly ashamed when I could not internalize it.
  - malfist 3y ago
    Oh don't feel bad. I had to look up every one of those numbers
  - nzgrover 3y ago
    You might enjoy Grace Hopper's leacture which includes this snippet: https://youtu.be/ZR0ujwlvbkQ?si=vjEQHIGmffjqfHBN&t=2706 https://youtu.be/ZR0ujwlvbkQ?si=vjEQHIGmffjqfHBN&t=2706
  - darzu 3y ago
    probably this one: https://gist.github.com/hellerbarde/2843375 https://gist.github.com/hellerbarde/2843375
- LeifCarrotson 3y ago
  I wonder how many people have built failed businesses that never had enough customer data to exceed the DDR4 in the average developer laptop, and never had so many simultaneous queries it couldn't be handled by a single core running SQLite, but built the software architecture on a distributed cloud system just in case it eventually scaled to hundreds of terabytes and billions of simultaneous queries.
  - Repulsion9513 3y ago
    A LOT... especially here.
  - malfist 3y ago
    I totally hear you about that. I work for FAANG, and I'm working on a service that has to be capable of sending 1.6m text messages in less than 10 minutes. The amount of complexity the architecture has because of those constraints is insane. When I worked at my previous job, management kept asking for that scale of designs for less than 1/1000 of the throughput and I was constantly pushing back. There's real costs to building for more scale than you need. It's not as simple as just tweaking a few things. To me there's a couple of big breakpoints in scale: * When you can run on a single server * When you need to run on a single server, but with HA redundancies * When you have to scale beyond a single server * When you have to adapt your scale to deal with the limits of a distributed system, i.e. designing for DyanmoDB's partition limits. Each step in that chain add irrevocable complexity, adds to OE, adds to cost to run and cost to build. Be sure you have to take those steps before you decide too.
jeffbee 3y ago
"I will simply have another box for redundancy" is already a system so complex that having it in or out of the cloud won't make a difference.
- Nextgrid 3y ago
  It really depends on business requirements. Real-time redundancy is hard. Taking backups at 15-min intervals and having the standby box merely pull down the last backup when starting up is much easier, and this may actually be fine for a lot of applications. Unfortunately very few actually think about failure modes, set realistic targets, and actually test the process. Everyone thinks they need 100% uptime and consistency, few actually achieve it in practice (many think they do, but when shit hits the fan it uncovers an edge-case they haven't thought of), but it turns out that in most cases it doesn't matter and they could've saved themselves a lot of trouble and complexity.
  - littlestymaar 3y ago
    So much this. I'd github can afford the amount of downtime they do, it's likely that your business can afford 15 minutes of downtime every once in a while due to a failing server. Also, the less servers you have overall, the least common a failure will be. Backups and cold failover server are mandatory, but anything past that should be weighted on a rational cost/benefit analysis, and for most people the cost/benefit ratio just isn't enough to justify infrastructure complexity.

echelon 3y ago

The reasons to switch away from cloud keep piling up. We're doing some amount of on-prem, and I'm eager to do more.

aarmenaa 3y ago
I've previously worked for a place that ran most of their production network "on-prem". They had a few thousand physical machines spread across 6 or so colocation sites on three continents. I enjoyed that job immensely; I'd jump at the chance to build something like it from the ground up. I'm not sure if that actually makes sense for very many businesses though.
- kuchenbecker 3y ago
  I'm getting that opportunity, however I expect it will be the last as most have migrated to the cloud and smaller companies are appealing to me; smallest company (of 5) I've worked for had 4.4k employees and large companies have the resources to roll their own. Unless there is an onprem movement I expect cloud to be the future as maintaining the tech stack onprem is difficult and we need to nake decisions down to the hardware we order.

malfist 3y ago

I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache. Samsung 990 Pro 2TB has a latency of 40 μs DDR4-2133 with a CAS 15 has a latency of 14 nano seconds. DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD. L1 cache is typically accessible in 4 clock cycles, in 4.8 ghz cpu like the i7-10700, L1 cache latency is sub 1ns.

avg_dev 3y ago
pretty cool comparisons. quite some differences there. tangent, I remember reading some post called something like "Latency numbers every programmer should know" and being slightly ashamed when I could not internalize it.
- malfist 3y ago
  Oh don't feel bad. I had to look up every one of those numbers
- nzgrover 3y ago
  You might enjoy Grace Hopper's leacture which includes this snippet: https://youtu.be/ZR0ujwlvbkQ?si=vjEQHIGmffjqfHBN&t=2706 https://youtu.be/ZR0ujwlvbkQ?si=vjEQHIGmffjqfHBN&t=2706
- darzu 3y ago
  probably this one: https://gist.github.com/hellerbarde/2843375 https://gist.github.com/hellerbarde/2843375
LeifCarrotson 3y ago
I wonder how many people have built failed businesses that never had enough customer data to exceed the DDR4 in the average developer laptop, and never had so many simultaneous queries it couldn't be handled by a single core running SQLite, but built the software architecture on a distributed cloud system just in case it eventually scaled to hundreds of terabytes and billions of simultaneous queries.
- Repulsion9513 3y ago
  A LOT... especially here.
- malfist 3y ago
  I totally hear you about that. I work for FAANG, and I'm working on a service that has to be capable of sending 1.6m text messages in less than 10 minutes. The amount of complexity the architecture has because of those constraints is insane. When I worked at my previous job, management kept asking for that scale of designs for less than 1/1000 of the throughput and I was constantly pushing back. There's real costs to building for more scale than you need. It's not as simple as just tweaking a few things. To me there's a couple of big breakpoints in scale: * When you can run on a single server * When you need to run on a single server, but with HA redundancies * When you have to scale beyond a single server * When you have to adapt your scale to deal with the limits of a distributed system, i.e. designing for DyanmoDB's partition limits. Each step in that chain add irrevocable complexity, adds to OE, adds to cost to run and cost to build. Be sure you have to take those steps before you decide too.

jeffbee 3y ago

"I will simply have another box for redundancy" is already a system so complex that having it in or out of the cloud won't make a difference.

Nextgrid 3y ago
It really depends on business requirements. Real-time redundancy is hard. Taking backups at 15-min intervals and having the standby box merely pull down the last backup when starting up is much easier, and this may actually be fine for a lot of applications. Unfortunately very few actually think about failure modes, set realistic targets, and actually test the process. Everyone thinks they need 100% uptime and consistency, few actually achieve it in practice (many think they do, but when shit hits the fan it uncovers an edge-case they haven't thought of), but it turns out that in most cases it doesn't matter and they could've saved themselves a lot of trouble and complexity.
- littlestymaar 3y ago
  So much this. I'd github can afford the amount of downtime they do, it's likely that your business can afford 15 minutes of downtime every once in a while due to a failing server. Also, the less servers you have overall, the least common a failure will be. Backups and cold failover server are mandatory, but anything past that should be weighted on a rational cost/benefit analysis, and for most people the cost/benefit ratio just isn't enough to justify infrastructure complexity.