7 ms·

It's probably worth extending "Your data fits in RAM" to "Your data doesn't fit in RAM, but it does fit on an SSD". So many problems will still work with quite

by Smerity 11y ago

It's probably worth extending "Your data fits in RAM" to "Your data doesn't fit in RAM, but it does fit on an SSD". So many problems will still work with quite reasonable performance when using an SSD instead. By using a single machine with an array of SSDs, you also avoid the complexity and overhead of distributed systems. My favourite realization of this: Frank McSherry shows how simplicity and a few optimisations can win out on graph analysis in his COST work. In his first post[1], he shows how to beat a cluster of machines with a laptop. In his second post[2], he applies even more optimizations, both space and speed, to process the largest publicly available graph dataset - terabyte sized with over a hundred billion edges - all on his laptop's SSD. [1]: http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html http://www.frankmcsherry.org/graph/scalability/cost/2015/01/... [2]: http://www.frankmcsherry.org/graph/scalability/cost/2015/02/04/COST2.html http://www.frankmcsherry.org/graph/scalability/cost/2015/02/...

cornellphds 11y ago
This is classic case of "Algorithm/Problem Selection" if your algorithm/problem is tailored to a task such as PageRank, surely a single threaded highly optimized code will beat a cluster designed for ETL tasks. In real organizations where there are multiple workflows/algorithms, distributed systems always win out. Systems like Hadoop take care of Administration, Redundancy, Monitoring and Scheduling in a manner that a single machine cannot. Sure you can "Grep" faster on a laptop than AWS EMR with 4 Medium instances, but in reality where you have 12 types of jobs which are run by team of 6 people, you are much better off with a distributed system.
- acqq 11y ago
  Ditto for computationally intensive work: if it is CPU dominated, more CPU's calculating in parallel will be of advantage, even if the data could fit some RAM. There's no a single simple answer, but sure, whenever less computers are enough, less should be used. The recent problem is, some people love "clouds" so much today that they push there the work that could really be done locally.
  - vidarh 11y ago
    Part of the problem is that a lot of problems that are CPU dominated on a single system becomes IO dominated once you start distributing it at very low node counts without very careful attention to detail.
    - acqq 11y ago
      Yes, and starting from the distributed version first, one can even miss the fact that everything can be immensely faster once the communication is out of the way. There's no a single simple answer, except: don't decide about the implementation first, instead competently and without prejudice evaluate your actual possibilities. Also don't decide the language like "Python" or "Ruby" first. If you expect that the calculations are going to take a month, then code them in Ruby and wait the month for the result, you can miss the fact that you could have had the results in one day by just using another language, most probably without sacrifying the readability of the code much. Only if the task is really CPU-bound, of course. On another side, if you have a ready solution in Python, and you'd need a month to develop the solution for other language, the first thing you have to consider is how often you plan to repeat the calculations afterwards. Etc.
      - acdha 11y ago
        I'd also toss in ease of correctness if you don't have a really well-known problem with a good test suite: I've seen a number of cases where people developed “tight” C code which gave results which appeared reasonable and then ran it for months before noticing logic errors which would either have been impossible or easier to find in a higher-level language or, particularly, one which had better numeric error handling characteristics. There is a lot to be said for confirming that you have the right logic and then looking into something like PyPy/Cython/etc. or porting to a lower-level language.
        acqq 11y ago
        > one which had better numeric error handling characteristics Which language do you suggest which matches this? I don't know many people who use the language "numeric error handling characteristic" but I know the benefit of already having good written high-level primitives (or libraries).
- Retric 11y ago
  Some things take 1 second on 1 machine beefy machine or 6 hours on a cluster due to latency issues. And yes I do mean 20,000 times slower, though 2,000 is far more common due to latency inside a datacenter being around ~500,000 ns vs ~100 ns for main memory vs 0.5 nm from L1 cache. PS: Not that most systems are built around these kinds of edge cases, but 'just use a cluster' is often not a good option unless each node is sufficiently beefy.
threeseed 11y ago
Scalable graph systems have been around for many years and have never taken off. Most businesses doing big data (like ours) often have multiple disparate data sources that at the start of the pipelines are ETLing into some EDW. Trying to consolidate them into a single integrated view is very difficult and time/resource intensive. Having billions of disconnected nodes in the graph would be very hard to reason with.
stephengillie 11y ago
I feel compelled to point out that the average SSD has an order of magnitude (or maybe 2 orders) more IOPS than a 6-disk 15k RAID6 or RAID10 array. And that's a single, standalone, non-RAIDed SSD. When you get a 6-SSD RAID10, magic starts to happen. And if you RAID enough SSDs (10-20?), you can theoretically start to get more bandwidth than you do with RAM.
- xjia 11y ago
  sounds good to me, but why people not doing that? ssd price too high?
  - s_baby 11y ago
    RAID SSD has the same latency as a single SSD. This setup will help where the bottleneck is I/O throughput.
  - fweespeech 11y ago
    We run 8 SSD RAID 10 for all our database nodes. However, you still need the same metric ton of RAM you'd need before. However, if you have an I/O heavy load [which almost all databases do] you can cut down on total number of machines compared to SATA.
  - vidarh 11y ago
    People are doing that. If you see those PCIe "SSDs", most of them are effectively multiple SSD's + controller on a card, combined with either software or hardware RAID-0... E.g. some of the OCZ cards at least show up as 4 individual units under Linux, while the Windows drivers at least by default will show it as one device. But yes, it is expensive if you measure per GB. If your dataset is small, though, and you are more interested in cost per IO operation, they can be very cheap.
    - toomuchtodo 11y ago
      I've built boxes for computationally intensive workloads that use these PCIe SSD cards. The IO is STUPID fast, considering you're bypassing the SATA chipset and are limited only by PCIe speeds. You don't get a large amount of space (I believe the cards I installed were only 64 or 128GB, but this was ~4 years ago), but for small datasets that you need extremely fast access to, they get the job done.
vardump 11y ago
A step between RAM and SSD could be "Your data fits in RAM in compressed form". LZ4 compression is takes 3-4x longer than memcpy. LZ4 decompression is only 50% [1] slower. 2-3 GB/s per core. [1]: Your mileage may vary.
- bpicolo 11y ago
  If it's slower both in and out what's the benefit? To guy below me: Ah, thanks. I thought the guy above was trying to say it's slower than paging to disk. : )
  - jhgg 11y ago
    Because it's so much faster than having to go to disk.
  - peapicker 11y ago
    Basically, parsing the percentages, using compressed data in RAM is ~1 order of magnitude slower to get into memory, and less than 1 order of magnitude slower to read from RAM. That is still around 2 (or more depending on setup) orders of magnitude faster than the hit to go to SSD / SSD RAID. Useful knowledge on latency: https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832

by Smerity 11y ago

cornellphds 11y ago

This is classic case of "Algorithm/Problem Selection" if your algorithm/problem is tailored to a task such as PageRank, surely a single threaded highly optimized code will beat a cluster designed for ETL tasks. In real organizations where there are multiple workflows/algorithms, distributed systems always win out. Systems like Hadoop take care of Administration, Redundancy, Monitoring and Scheduling in a manner that a single machine cannot. Sure you can "Grep" faster on a laptop than AWS EMR with 4 Medium instances, but in reality where you have 12 types of jobs which are run by team of 6 people, you are much better off with a distributed system.

acqq 11y ago
Ditto for computationally intensive work: if it is CPU dominated, more CPU's calculating in parallel will be of advantage, even if the data could fit some RAM. There's no a single simple answer, but sure, whenever less computers are enough, less should be used. The recent problem is, some people love "clouds" so much today that they push there the work that could really be done locally.
- vidarh 11y ago
  Part of the problem is that a lot of problems that are CPU dominated on a single system becomes IO dominated once you start distributing it at very low node counts without very careful attention to detail.
  - acqq 11y ago
    Yes, and starting from the distributed version first, one can even miss the fact that everything can be immensely faster once the communication is out of the way. There's no a single simple answer, except: don't decide about the implementation first, instead competently and without prejudice evaluate your actual possibilities. Also don't decide the language like "Python" or "Ruby" first. If you expect that the calculations are going to take a month, then code them in Ruby and wait the month for the result, you can miss the fact that you could have had the results in one day by just using another language, most probably without sacrifying the readability of the code much. Only if the task is really CPU-bound, of course. On another side, if you have a ready solution in Python, and you'd need a month to develop the solution for other language, the first thing you have to consider is how often you plan to repeat the calculations afterwards. Etc.
    - acdha 11y ago
      I'd also toss in ease of correctness if you don't have a really well-known problem with a good test suite: I've seen a number of cases where people developed “tight” C code which gave results which appeared reasonable and then ran it for months before noticing logic errors which would either have been impossible or easier to find in a higher-level language or, particularly, one which had better numeric error handling characteristics. There is a lot to be said for confirming that you have the right logic and then looking into something like PyPy/Cython/etc. or porting to a lower-level language.
      - acqq 11y ago
        > one which had better numeric error handling characteristics Which language do you suggest which matches this? I don't know many people who use the language "numeric error handling characteristic" but I know the benefit of already having good written high-level primitives (or libraries).
Retric 11y ago
Some things take 1 second on 1 machine beefy machine or 6 hours on a cluster due to latency issues. And yes I do mean 20,000 times slower, though 2,000 is far more common due to latency inside a datacenter being around ~500,000 ns vs ~100 ns for main memory vs 0.5 nm from L1 cache. PS: Not that most systems are built around these kinds of edge cases, but 'just use a cluster' is often not a good option unless each node is sufficiently beefy.

threeseed 11y ago

Scalable graph systems have been around for many years and have never taken off. Most businesses doing big data (like ours) often have multiple disparate data sources that at the start of the pipelines are ETLing into some EDW. Trying to consolidate them into a single integrated view is very difficult and time/resource intensive. Having billions of disconnected nodes in the graph would be very hard to reason with.

stephengillie 11y ago

I feel compelled to point out that the average SSD has an order of magnitude (or maybe 2 orders) more IOPS than a 6-disk 15k RAID6 or RAID10 array. And that's a single, standalone, non-RAIDed SSD. When you get a 6-SSD RAID10, magic starts to happen. And if you RAID enough SSDs (10-20?), you can theoretically start to get more bandwidth than you do with RAM.

xjia 11y ago
sounds good to me, but why people not doing that? ssd price too high?
- s_baby 11y ago
  RAID SSD has the same latency as a single SSD. This setup will help where the bottleneck is I/O throughput.
- fweespeech 11y ago
  We run 8 SSD RAID 10 for all our database nodes. However, you still need the same metric ton of RAM you'd need before. However, if you have an I/O heavy load [which almost all databases do] you can cut down on total number of machines compared to SATA.
- vidarh 11y ago
  People are doing that. If you see those PCIe "SSDs", most of them are effectively multiple SSD's + controller on a card, combined with either software or hardware RAID-0... E.g. some of the OCZ cards at least show up as 4 individual units under Linux, while the Windows drivers at least by default will show it as one device. But yes, it is expensive if you measure per GB. If your dataset is small, though, and you are more interested in cost per IO operation, they can be very cheap.
  - toomuchtodo 11y ago
    I've built boxes for computationally intensive workloads that use these PCIe SSD cards. The IO is STUPID fast, considering you're bypassing the SATA chipset and are limited only by PCIe speeds. You don't get a large amount of space (I believe the cards I installed were only 64 or 128GB, but this was ~4 years ago), but for small datasets that you need extremely fast access to, they get the job done.

vardump 11y ago

A step between RAM and SSD could be "Your data fits in RAM in compressed form". LZ4 compression is takes 3-4x longer than memcpy. LZ4 decompression is only 50% [1] slower. 2-3 GB/s per core. [1]: Your mileage may vary.

bpicolo 11y ago
If it's slower both in and out what's the benefit? To guy below me: Ah, thanks. I thought the guy above was trying to say it's slower than paging to disk. : )
- jhgg 11y ago
  Because it's so much faster than having to go to disk.
- peapicker 11y ago
  Basically, parsing the percentages, using compressed data in RAM is ~1 order of magnitude slower to get into memory, and less than 1 order of magnitude slower to read from RAM. That is still around 2 (or more depending on setup) orders of magnitude faster than the hit to go to SSD / SSD RAID. Useful knowledge on latency: https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832