5 ms·
Can a 'perfect' algorithm actually give end-users the consistent feel of flash performance with 80% of data stuck on platters? I doubt it. Access patterns havi
by zaroth 12y ago
Can a 'perfect' algorithm actually give end-users the consistent feel of flash performance with 80% of data stuck on platters? I doubt it.
Access patterns having some statistical distribution is one thing. Writing the allocator -- and always guessing right in where you allocate a file is another thing altogether.
Flash is massively higher performance, so every time you are wrong the user will definitely know. All we hear about these drives is that they aren't consistently fast. Or they're decent until you actually push them too hard.
SSDs themselves used to have poor consistency. But drives are manufactured to ever increasing standards. The Samsung 10 year warranty for example is striking.
80% of your data on a medium which is less reliable over time?! More complex moving parts increases failure rates. There are downsides here to consider I think?
- avem2 12y agoThat's the goal right? To make the software do the heavy lifting so your storage is better utilized without requiring the end user to do the constant change between mediums of storage.
- nl 12y agoCan a 'perfect' algorithm actually give end-users the consistent feel of flash performance with 80% of data stuck on platters? I doubt it. Of course not. That's not the point! What may be possible is to give the "consistent feel of flash performance" some very high percent of the time for a dramatically lower price. If it's possible to get flash-like performance 95%+ of the time for <50% of the price then I'd find that pretty compelling.
- zaroth 12y agoBut flash will have higher performance and higher capacity, at least that's the projections. I don't know why flash shouldn't be able to win at price/capacity, longevity, total capacity, robustness, performance, etc. At some point, there just shouldn't be any reason to manufacture storage in any other form. In a few generations, isn't that the level of dominance of flash?
- rasz_pl 12y ago>I don't know why flash shouldn't be able to win at price/capacity, longevity, total capacity, robustness, performance, etc yes, you dont, and this is why you think this: >But flash will have higher performance and higher capacity longevity, robustness, endurance, it all goes down with every new generation of flash. Performance at silicon level is already as good as it gets, we only win by making wider busses and assessing more data in parallel. Price per gigabyte comes at a price (haha) or endurance. Speed comes at a price of more chips per disk (or more wafers/structures inside chips) = money. Want that sweet new 1TB PCIE SSD able to write at 2GB/s? it will shit itself after ~24hours of continuous write. There is no free lunch with SSDs, they are a dead alley.
- hengheng 12y agoWhat the OP didn't show was the integral of response times for an average set of read accesses, e.g. The typical expected waiting time. With a latency difference of 1/50k second vs 1/100 second, even 1 on 500 queries missing the cache will double the latency. I'll consider the stats in the article, but rather as a means of fine-tuning between storage classes that are more similar, say high-end PCIe ssds vs cheap and large consumer ssds. Assuming the tiered storage controller is completely transparent and performs perfectly, that is.
- andywarfield 12y agoGetting things perfect is pretty hard: it basically required being able to see the future. There are still a bunch of examples where imperfect decisions end up being a really significant win. For instance: 1. The counter stacks that we build using the HLLs allow us to draw miss ratio curves. These are a model for how much a given application will benefit from getting additional fast memory (whether that fast memory is flash/ram/etc). A useful aspect of this isn't just trying to give it more memory but also identifying the workloads that you simply can't win on. For workloads with absolutely enormous working sets, you can decide to give them a lot less resources, and then end up speeding up all the other applications in the system. Alternatively, you can decide to give them the large amount of memory that they do need to be successful, possibly by having to buy additional fast memory. In either case, being able to get detailed working set analysis helps make appropriate decisions a lot. 2. 80/20 is completely anecdotal -- the appropriate split is always going to be entirely dependent on workload. That said, as you move out into the tail of the access frequency curve, cost per access increases. So even if you don't move data out to disk -- even with only a single type of storage media -- understanding these curves can allow you to decide to do things like compress the colder half of your storage, trading off some compute work and latency on access for more space. In the case of hybrid systems, strategies like this have the potential to really increase the effectiveness of caches. Totally agree with your other points. There are many factors to consider and balance in any system design. Note that for raw media, magnetic disks are still way more reliable in terms of archiving your bits without data errors over long periods of time than NAND. In either case, good system design needs redundancy and scrubbing to make sure that device errors are caught and recovered from before data is actually lost.