4 ms·

Citations please? That's a pretty bold statement to make in the face of observed reality.

by doug_durham 4y ago

fho 4y ago
https://yourdatafitsinram.net/ https://yourdatafitsinram.net/
- threeseed 4y ago
  This is such a lazy response. I/O performance is just one of many characteristics that impact performance and from experience the one you least need to worry about. RAID 0 across multiple high-end NVME drives with OS file caching is going to be more than fast enough for most use cases. The issue is running out of CPU performance and being able to seamlessly scale up/down compute with live running workloads.
  - beckingz 4y ago
    A large computer is radically CPU overprovisioned for most workloads.
    - threeseed 4y ago
      But we aren't talking about most workloads.
      - fho 4y ago
        But ... we are ... basically by definition. Vanishing little projects actually need cloud scale infrastructure. And, to address your previous statement: one beefy server is actually pretty scalable. Soft threads spin up in microseconds to serve incoming requests, communication between threads is blazing fast, caching is simpler on one machine, etc. You don't even have to worry to much about scaling, the CPU just throttles itself when there is no load. And every once in a while you just upgrade to the next gen beefy machine.
beckingz 4y ago
Even if this is off by two orders of magnitude and it's only 100,000 companies that need distributed compute, that means that almost all companies just need a single large computer. Looking at the distribution of companies by employee count and assuming that data scales with employee count (dangerous assumption, but probably true enough on average), that means that companies don't need distributed compute until they get several hundred employees. [0] [0] https://www.statista.com/statistics/487741/number-of-firms-in-the-us-by-employment-size/ https://www.statista.com/statistics/487741/number-of-firms-i...

Citations please? That's a pretty bold statement to make in the face of observed reality.

fho 4y ago

https://yourdatafitsinram.net/ https://yourdatafitsinram.net/

threeseed 4y ago
This is such a lazy response. I/O performance is just one of many characteristics that impact performance and from experience the one you least need to worry about. RAID 0 across multiple high-end NVME drives with OS file caching is going to be more than fast enough for most use cases. The issue is running out of CPU performance and being able to seamlessly scale up/down compute with live running workloads.
- beckingz 4y ago
  A large computer is radically CPU overprovisioned for most workloads.
  - threeseed 4y ago
    But we aren't talking about most workloads.
    - fho 4y ago
      But ... we are ... basically by definition. Vanishing little projects actually need cloud scale infrastructure. And, to address your previous statement: one beefy server is actually pretty scalable. Soft threads spin up in microseconds to serve incoming requests, communication between threads is blazing fast, caching is simpler on one machine, etc. You don't even have to worry to much about scaling, the CPU just throttles itself when there is no load. And every once in a while you just upgrade to the next gen beefy machine.

beckingz 4y ago

Even if this is off by two orders of magnitude and it's only 100,000 companies that need distributed compute, that means that almost all companies just need a single large computer. Looking at the distribution of companies by employee count and assuming that data scales with employee count (dangerous assumption, but probably true enough on average), that means that companies don't need distributed compute until they get several hundred employees. [0] [0] https://www.statista.com/statistics/487741/number-of-firms-in-the-us-by-employment-size/ https://www.statista.com/statistics/487741/number-of-firms-i...