3 ms·
How bad is memory latency? If this is really a 64TB address space you are going to need an insane cache hierarchy to make this fast unless they've made some sc
by habosa 13y ago
How bad is memory latency? If this is really a 64TB address space you are going to need an insane cache hierarchy to make this fast unless they've made some scientific breakthrough and not shared it with the world.
- fiatmoney 13y agoThey mention it being composed as a cluster of blades with "Numalink 6 interconnect", so I'd guess "pretty bad" if you're hitting non-local RAM. You'll definitely be rewriting your programs to deal with the quasi-distributed aspect. There are actually a number of options if you're willing to do that kind of architecture, but I haven't seen anything with >2tb of RAM on a single motherboard. Even those are pretty strongly NUMA between sockets.
- rwallace 13y agoIt probably does have an insane cache hierarchy, but in any case think of it this way: latency is going to be dramatically better to main memory on this machine than to disk, which is what the alternative would be if you have a highly nonlocal workload.
- habosa 13y agoTrue, but gotta wonder how much better this latency is than a more reasonable amount of RAM backed by a super-fast SSD.
- rwallace 13y agoIt depends on locality I suppose. If your workload is spatially local, you're good on this machine and might also be good on a bunch of smaller boxes. If spatially nonlocal but temporally local, your solution might work well. If also temporally nonlocal, this machine might be your best bet; even if latency hurts bad, it might still be better than on any other hardware.
- frozenport 13y agoThere isn't a bus with the characteristics you mentioned. For example, SATA 3 saturates at 1 GBps. A few years ago I measured something like 50GBps on these guys. The real trick is to walk a graph with multiple processors so that you saturate the bus. That being said I liked the Yarc data architecture more.
- andruby 13y agoThat's why the fastest SSD's use PCI Express instead of SATA. [1] http://www.fusionio.com/products/iodrive-octal/ http://www.fusionio.com/products/iodrive-octal/ (6GB/s)
- frozenport 13y agoThere is an interesting story here, take a look at the benchmarks http://regmedia.co.uk/2011/04/07/ssd_write_drop_off.jpg http://regmedia.co.uk/2011/04/07/ssd_write_drop_off.jpg I did a bunch of work with SSDs and trying to get high throughput, at the end of the day I could touch ram at 10 megabytes per millisecond but only do IO at 2 megabyte per millisecond.