6 ms·
80M I/O Per Second with a Standard 2U System
- codevark 5y agoWhere have you been? (hint: already done by someone else)
- chx 5y ago> The system has 2 CPUs with 64 PCIe Gen 4 lanes each, giving us plenty of PCIe lanes to connect NVMe SSDs Imagine if they used EPYC CPUs... 160 PCIe lanes.
- unglaublich 5y agoThey could have achieved this 'milestone', 1.5 yrs ago. Feels like an Intel ad to me.
- SkipperCat 5y agoAgreed. Intel Optane NVMe has been out for quite a while, we've been using them for at least 3 years now. Still, I'm love reading articles showcasing their performance. NVMe is such a game changer for storage I/O based systems.
- pacetherace 5y ago80M 512 byte I/O per second. That’s 10M 4KB I/O per second. That’s not groundbreaking.
- wmf 5y agoThey also did 25M 4K IOPS on the same system. Anyway, it's basically an ad for Intel® Xeon® and Optane®.
- toolslive 5y agoNo.Extending your reasoning, it would just be 1 40GB I/O per second. Each io request has a fixed cost, which is better measured with small requests. Bandwidth is not really relevant here.
- hutrdvnj 5y agoThis makes for a great database server.
- rapsey 5y agoSo what is the price for such a system
- ants_a 5y ago2x$8k for the CPUs + 16x$2.2k for the drives + rest of the system >= $50k
- SkipperCat 5y agoWe've been using these systems for about 2 years now. We can put 32TB in a 1U box with dual Xeon Gold processors for a little under $20K. Our business is storing incoming market data and this architecture has allowed us to write bursts of up to 80 Gbps with ease. NVMes rock!
- bullen 5y agoWhat is the point of that speed if the rest of your system is a bottle neck and the drives will fail in a year if you use that speed continously? I think we need to go back and look at cycles per bit (DWPD is confusing because you need to know the number of years and sometimes it's not mentioned on the product page). Complexity can increase some performance but there is always a cost to complexity. Optane seemed promising because they increased the speed (that I personally will never need) and preserved the resiliance. Time will tell, but I bought 30x new 50nm SLC 64GB X25-E drives from 2011 on ebay for $100 each that I will mount on 12x SATA motherboards (only got cable/power for 8+1 per board) instead of buying one 480GB Optane drive for $1500! Cheaper, slower and maybe more robust? Atleast it is simpler! Atleast SATA I can use in my laptops/desktops/consoles etc. Also nobody is looking at CPU usage of NVMe vs. SATA!
- consp 5y agoI did something similar on a smaller scale: "Caching" ssd drives which are not the best but have a huge amount write bytes left are widely available on ebay and I bought 8 200GB SAS3 drives for dirt cheap (under 200$ total, drives from 2017). Not the fastest drives but good enough for my purpose (overkill even).
- bullen 5y agoWhat drives did you get?
- consp 5y agoSun/oracle branded HGST SS300 drives (HUSMH4020ASS210), they have the specs in the ballpark of the SS800 drives in endurance (according to the scsi data pages from the disks) but read/write performance of the SS300.
- bullen 5y agoTrying to google it it seems unclear to me what the product is... SSD or hybrid? Also what tech (SLC, MLC, TLC, QLC) and how many nm? This is the real reason why I went all in on the X25-E; there is no clear winner since 10 years... either it's super expensive and might be good or cheap and guaranteed crap! Also the interfaces are all over the place, PCIe, SAS, SATA, NVMe (which is PCIe?), M.2 with double PCIe?! with all kinds of connectors... We have a serious problem with storage of data when magnetic tape is still the real solution!
- jgalt212 5y agoIs this the death of kubernetes? Can I now use one server instead of 20?
- SkipperCat 5y agoNVMe will allow your storage to be super fast, but it doesn't add any more compute resources. So unless your K8s cluster was all about IO, i'd say no...
- bashinator 5y agoI daydream about running our entire company k8s infra on a single 1U with 256 Epyc threads, a few terabytes of memory, and a bunch more of tiered storage. IMO this kind of miniaturization is amazing for container orchestration. Add a couple more, overprovision by 50%, let k8s handle the high availability, and you’ve got a datacenter in a small box.
- gravypod 5y agoI have nightmares of someone convincing my company to do something like that and then a having a hardware, network, or power failure.