4 ms·
I just boggle at the idea of a server with 1TB of RAM. I'm sure the oil&gas folks are salivating as this allows them to to put an entire high resolution 'cube'
by ChuckMcM 10y ago
I just boggle at the idea of a server with 1TB of RAM. I'm sure the oil&gas folks are salivating as this allows them to to put an entire high resolution 'cube' into memory and analyze it but for us mere mortals at what point is there so much stuff in memory that you need a couple hours of hold up time just to flush it out to SSD.
- api 10y agoRun your whole system in RAM with periodic backup to "slow" SSD?
- Keyframe 10y ago1.4 gigabytes per second or more for 4K flat video work at 4 color components at 16 bits at 25fps (PAL land) each raw for each channel on the timeline saturates system real quick. For online finishing systems that would be sweet. 8 TB would fill an average 90 minutes movie if you had a finishing timeline (no additional ones over it). With over 12TB I wouldn't know what to do anymore, as of this point in time. Sure, you can build a RAID that fast already though.
- threeseed 10y agoThose of us in the Big Data space can easily chew through that. Few HDFS data sets stored in RAM or some more Data Science Spark workloads.
- flukus 10y ago> at what point is there so much stuff in memory that you need a couple hours of hold up time just to flush it out to SSD A few hundred megabytes for our "enterprise" financial software...
- gfody 10y agoWe've had database servers with 2-3TB of RAM in production for years. You can get servers today with 24TB RAM - they're 8-way so lots of NUMA going on but still tons o' RAM.
- apaprocki 10y agoThe bread-and-butter HP DL380 goes up to 3TB. On the other end of the spectrum you have beasts like the IBM E880 that goes up to 32TB.
- binarycrusader 10y agoThe SPARC M7-16 has up to 16 TB of ram, sixteen processors (512 cores, 4,096 threads), 48 PCIe 3.0 x16 slots: http://www.oracle.com/us/products/servers-storage/sparc-m7-16-ds-2687045.pdf http://www.oracle.com/us/products/servers-storage/sparc-m7-1... This is just the mainstream catching up.
- IntelMiner 10y agoHow does it benchmark against x86 systems available now though? I was under the impression SPARC was on life support
- olegkikin 10y agoYou can buy 256GB of DDR3 RAM for just $576, so 1TB is only $2304. Example: http://www.ebay.com/itm/182537878712 http://www.ebay.com/itm/182537878712 Of course that's just RAM. If you want a server with it, it's not much more, just over $1K shipped: http://www.ebay.com/itm/391697886461 http://www.ebay.com/itm/391697886461
- astrodust 10y agoIt's true, 1TB of memory is well within the realm of an over-eager enthusiast. Motherboards that support it aren't even exotic, they're just physically large server-grade ones.
- agumonkey 10y agoCompletely. 2000CUR is nothing for someone who is looking at this. It's an apple laptop. Nothing smashing. That said I still want my OS to fit on a fdd and have 64k demos generating the universe.
- yuhong 10y agoThey are often based on lower density DDR3 chips that consume more power per bit. That being said you can do 1TB on a normal dual socket server with 64GB DDR4 LR-DIMMs for a while now, though they do come at a cost per bit premium.
- ChuckMcM 10y agoI guess I should have been more clear and said a 'commodity server' with 1TB of RAM :-)
- tbrownaw 10y agoTry an analytics / reporting database server. Lots of data to keep readily available, but almost no dirty pages.
- chx 10y agoThe big headache is not 1TB but 64TB which is the maximum physical RAM limit of the Linux kernel / x86-64 architecture. Big NUMA systems could go higher but they don't. Look here: https://www.sgi.com/products/servers/uv/ https://www.sgi.com/products/servers/uv/ "SGI UV 300 scales from 4 to 64 sockets with up to 64TB of shared memory in a single system." vs "SGI UV 3000 scales from 4 to 256 CPU sockets with up to 64TB of shared memory as a single system." see how 256 and 64 sockets both only support 64TB? More ordinary servers typically stop at 6TB since one Xeon can support 1.5TB so an ordinary quad socket board typically with 96 DIMM slots can go up to 6TB with 64GB DIMMs. You can configure a machine like this at http://www.thinkmate.com/system/superserver-4048b-tr4ft http://www.thinkmate.com/system/superserver-4048b-tr4ft and see that for the relatively low price of $110K you can get a machine with 6TB of RAM.
- AaronFriel 10y ago> The big headache is not 1TB but 64TB which is the maximum physical RAM limit of the Linux kernel / x86-64 architecture. It sounds like you know what you're talking about, so I'm sure it was inadvertent that you wrote "physical RAM limit of the Linux kernel". It's primarily the x86-64 architecture and 5-level paging is coming which extends the linear address space to 57 bits (128 PiB) and the physical address space up to 52 bits (4 PiB). Still, one wonders how long it will take for that to be inadequate. https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf https://software.intel.com/sites/default/files/managed/2b/80...
- qb45 10y agoAnd Linux support for 5-level is being actively worked on since December, a quick git search shows this in 4.11: Merge 5-level page table prep from Kirill Shutemov: "Here's relatively low-risk part of 5-level paging patchset. Merging it now will make x86 5-level paging enabling in v4.12 easier.
- dom0 10y agoThere are still 8-socket systems, like the Supermicro 7088B (192 DIMMs), which support a theoretical 24 TB (with 128 GB modules). (It would be interesting to know whether this backplane-based system actually manages to achieve 4.8 GHz QPI speed (9.6 GHz symbol rate), or whether the physical aspects limit it to lower speeds. Given that processors only have four QPI links this would further increase communication overhead -- in a 8-way system some sockets are separated by two hops).
- qb45 10y ago> I just boggle at the idea of a server with 1TB of RAM. MooseFS (a scale-out storage system) keeps metadata in RAM for low latency and is known for pushing the limits of commodity hardware in big installations. I presume the same applies to any kind of low-latency database of similar size.
- krakensden 10y agoI've worked with companies running MySQL with a terabyte of ram. Fun times.
- StillBored 10y agomere mortals at what point is there so much stuff in memory that you need a couple hours of hold up time just to flush it out to SSD If your buying these machines with multiple TB of RAM, then buying flash arrays that can drive multiple GB/sec of IO bandwidth shouldn't be a problem either. At say 8GB/sec write bandwidth flushing 1TB is a little over two minutes. Although, why you have that much "dirty" data in RAM might be another question. A database machine using that amount of RAM is going to care more about the IOP rates of the disk, so that its flushing updates to disk at the same rates they are arriving. Meaning that the RAM won't need to be flushed to disk if the machine/power/whatever fails. Disk arrays with >1M IOP/s have been around for over a decade, and given a SAN can be wired together to increase aggregate performance.