3 ms·
See also: Scalability! But at what COST? [1] [2] > The COST of a given platform for a given problem is the hardware configuration required before the platform
by 0xcde4c3db 8y ago
See also: Scalability! But at what COST? [1] [2]
> The COST of a given platform for a given problem is the
hardware configuration required before the platform outperforms a competent single-threaded implementation.
COST weighs a system’s scalability against the overheads introduced by the system, and indicates the actual
performance gains of the system, without rewarding systems that bring substantial but parallelizable overheads.
[1] http://www.frankmcsherry.org/assets/COST.pdf http://www.frankmcsherry.org/assets/COST.pdf
[2] https://news.ycombinator.com/item?id=11855594 https://news.ycombinator.com/item?id=11855594
- nisa 8y agoKeep in mind it's for graph processing - Hadoop/HDFS still shines for data-intensive streaming workloads like indexing a few hundred terabytes of data - where you can exploit the parallel disk io of all disks in the cluster - if you have 20 machines with 8 disks in a cluster that's 20 * 8 * 100mbyte/s = 16gbyte/s throughput - for 200 machines it's 160gbyte/s. However for iterative calculations like pagerank the overhead for distributing the problems is often not worth it.