3 ms·

Always worth giving a shoutout to the "Scalability! But at what COST?" paper (pdf https://www.frankmcsherry.org/assets/COST.pdf https://www.frankmcsherry.org/as

Always worth giving a shoutout to the "Scalability! But at what COST?" paper (pdf https://www.frankmcsherry.org/assets/COST.pdf https://www.frankmcsherry.org/assets/COST.pdf) We offer a new metric for big data platforms, COST, or the Configuration that Outperforms a Single Thread. The COST of a given platform for a given problem is the hardware configuration required before the platform outperforms a competent single-threaded implementation. COST weighs a system’s scalability against the overheads introduced by the system, and indicates the actual performance gains of the system, without rewarding systems that bring substantial but parallelizable overheads.

Joel_Mckay 2y ago
After numerous trials with CUDA enabled machines, I found the "cost function" models are better at determining efficiency. For example: * A single GPU machine is often more efficient than multiple cards due to bus i/o bandwidth constraints, lack of software support, and obscure driver failure modes * A single CPU machine is often more efficient than multi-chip solutions due to memory access latency and caching issues * A sequential drive data access machine is often more efficient than arrays due to pipelined memory cache layout and baked access latency * A single CPU core bound process is often more efficient due to avoiding threading and or mailbox overheads Thus, if a problem is truly separable, than it is sometimes wiser to bind n-many slow jobs to n-many cores rather than parallelize 1 job to try to get it done more quickly. I found this rather surprising in processing large video media data sets. Some may disagree, but they are mostly 15% to 30% more wrong for truly separable tasks. YMMV =3
fifilura 2y ago
What immediately makes it worth it is when the cloud provider you are (supposedly) already invested in provides the CPU cluster without extra set up cost. Examples Google BigQuery, AWS Athena or AWS serverless EMR. These cloud providers always have a bunch of CPUs idle, so they even provide the CPU cost for free, it is just the loading of data that costs.
- datadrivenangel 2y ago
  I love BigQuery, but they do charge you for CPU slots/processing. It's a great deal to not have to manage all that stuff though.
  - fifilura 2y ago
    Oh, that's new then? Or maybe it was a part of the monthly bill I never saw. AWS (still) costs per data loaded though.

Joel_Mckay 2y ago

After numerous trials with CUDA enabled machines, I found the "cost function" models are better at determining efficiency. For example: * A single GPU machine is often more efficient than multiple cards due to bus i/o bandwidth constraints, lack of software support, and obscure driver failure modes * A single CPU machine is often more efficient than multi-chip solutions due to memory access latency and caching issues * A sequential drive data access machine is often more efficient than arrays due to pipelined memory cache layout and baked access latency * A single CPU core bound process is often more efficient due to avoiding threading and or mailbox overheads Thus, if a problem is truly separable, than it is sometimes wiser to bind n-many slow jobs to n-many cores rather than parallelize 1 job to try to get it done more quickly. I found this rather surprising in processing large video media data sets. Some may disagree, but they are mostly 15% to 30% more wrong for truly separable tasks. YMMV =3

fifilura 2y ago

What immediately makes it worth it is when the cloud provider you are (supposedly) already invested in provides the CPU cluster without extra set up cost. Examples Google BigQuery, AWS Athena or AWS serverless EMR. These cloud providers always have a bunch of CPUs idle, so they even provide the CPU cost for free, it is just the loading of data that costs.

datadrivenangel 2y ago
I love BigQuery, but they do charge you for CPU slots/processing. It's a great deal to not have to manage all that stuff though.
- fifilura 2y ago
  Oh, that's new then? Or maybe it was a part of the monthly bill I never saw. AWS (still) costs per data loaded though.