6 ms·
You would if you had a storage appliance on a SAN, as the SAN is almost certainly faster than a single SSD. This is an extremely common setup for video product
by wayoutthere 7y ago
You would if you had a storage appliance on a SAN, as the SAN is almost certainly faster than a single SSD.
This is an extremely common setup for video production, so it’s not out of the target market at all.
- ianhowson 7y agoThere are plenty of cheap NVMe SSDs that can push 3.5Gbytes/sec (Samsung 970, Adata XPG). Even with dual 10Gbe NICs, you can't match that, nor the low access times of local storage. But I do agree that SAN storage is the norm here.
- chx 7y agoWe are past that now, new PCIe 4.0 SSDs just have been showcased along with the new AMD chips and they can do 5GBytes/s read and a bit above 4GB/s write (AMD is rumored to have invested in the R&D of the controller). You'd need 40 GbE to match one -- and EPYC Rome, also scheduled for this fall, will have 160 lanes allowing for dozens of them. You could very easily reach 100 GByte/s read which no network will match.
- deleted 7y ago[deleted]
- __d 7y agoTwin ConnectX6 adaptors, gives you 800Gbps, or ~1GB/s, at an absolute theoretical max. It's good to see that local storage has finally returned to the reasonable state of being faster than network storage. SATA / SAS was a long, slow period ...
- justinclift 7y agocough 1GB/s can by done by 10GbE. Maybe a slight typo there? Need to add a few zeros? :)
- __d 7y agoYou're right, doh. See above.
- tigershark 7y agoIf it’s 800 gbps then it’s 100GB/s, not 1...
- __d 7y agoYou're right. Brainfade. So, even with protocol overhead from all the stack layers chewing up maybe an order of magnitude, that'd still leave 10GBps. So .. I guess it's still possible, if impractical, to outperform a good PCIe SSD with the latest network interface.
- justinclift 7y ago... ~10GB/s can be done by a single 100Gb/s adapter. More 0's needed? :)
- shaklee3 7y agoPcie 3 was not the bottleneck for SSD. They typically are only 4 pcie lanes, when they could go up to 16 for 4x the bandwidth.
- Tepix 7y agoBut the standard M.2 NVMe interface happens to only have 4 lanes. PCIe4 will double the available bandwidth for these very common SSDs.
- justinclift 7y ago> You could very easily reach 100 GByte/s read which no network will match. High end networking gear already has higher throughput: http://www.mellanox.com/page/ethernet_cards_overview http://www.mellanox.com/page/ethernet_cards_overview https://www.servethehome.com/mellanox-connectx-6-brings-200gbe-and-hdr-infiniband-fabric-to-hpc/ https://www.servethehome.com/mellanox-connectx-6-brings-200g... A SAN/NAS using the same PCIe 4.0 SSD's you mention could probably fill the pipes too. ... and it would probably need a bunch of network stack tuning. ;)
- tigershark 7y agoThe 200 gb/s network adapter that you linked are 4 times slower than 100GB/s. The parent comment wrote explicitly 100 Gbyte/s.
- justinclift 7y agoOops. Didn't spot that, sorry. :) That being said, after re-reading the comment they're talking about adding multiple PCIe cards to a box to achieve 100GB/s of local total throughput. That would be achievable over a network by adding multiple PCIe 200Gb/s network cards too. :)
- chx 7y agoNah, a motherboard with enough M.2 connectors could easily exist. Or, U.2 or OCuLink. We have already seen 1P EPYC servers with six OCULink connectors...
- justinclift 7y agoSure. My point is just that whatever bandwidth you can do locally, you can also do over the network. As a sibling comment mentions though... the cost difference would be substantial. :(
- ti_ranger 7y ago>> You could very easily reach 100 GByte/s read which no network will match. > High end networking gear already has higher throughput: 100GB/s > 200Gbps You would need 4x 200Gbps ports to reach 100GB/s, so 2x MCX653105A-ECAT (each 2x 16-lanes) at > $700 each, and pay for 1/10th of a ~$30 000 switch, IOW 100GB/s would cost you ~ $4400, before paying for the storage. Sure, it could be done, but it wouldn't be cheap, and you'll have used most of the PCIe lanes.
- mseidl 7y agoThe new x570 motherboards will have them soon the pci 4.0 soon
- thinkmassive 7y agoThunderbolt 3 can beat that though
- Unklejoe 7y agoThunderbolt 3 basically just provides 4 lanes of PCIe 3.0
- programd 7y agoA typical product aimed at mid-range video producers, the G-Speed Shuttle SSD, can do up to 2800MB/s. That's 32TB of local Thunderbolt 3 attached SSD storage. Mind you, you'll pay $15K for it, but if you're in that business you can well afford it even if you're not a top tier Hollywood production shop. Given that your storage array costs that much, the fully loaded Mac Pro price (somewhere in the $20K range?) is not that outrageous. The people who use Red cameras and G-Tech storage arrays are the Mac Pro demographic Apple is going for here. Disclaimer: I used to work with G-Tech but no longer there.
- imagetic 7y agoPrices have dropped then, cause the G-Speed shuttle I use is 96TB and doesn't cost half that much. I've also used almost every model and in the real world and you don't ever get close to advertised speeds for R/W on those. Plus when the volume gets full it will drop to <100MB/s write. They are popular though. I see them a lot, but I have had very little success with them over the years.
- manigandham 7y agoIt's usually direct-attached storage, not network-attached. They use USB-C or Thunderbolt cables to join to a RAID storage device, and then backup/archive to a network based storage pool later.
- imagetic 7y agoI rarely see direct attached storage anymore. It's too cost effective from a media management standpoint to just to right to 10Gbe RJ45 or fiber network storage. The only direct storage I see is if speed is absolutely critical and that's very rare, mostly just high end 3D stuff.
- manigandham 7y agoGot it. I'm a few years removed but fast DAS raid boxes for each workstation were common with work product being synced to a network share. Looks like the NICs and SANs are fast enough now to run everything off the network.
- justinclift 7y agoLots of animation / multimedia houses use workstations with 40Gbps+ adapters (40GbE or Infiniband) connecting to network storage. It's done this way so people can connect to the storage that's appropriate for the task at hand. Different projects can be stored on different SAN/NAS arrays , each specc'ed out according to the size/needs of the project. eg a short run animation doesn't need the same storage capacity of a full length feature film. They may have similar throughput needs though. (Summarising here, but the general concept is ok) So, lets say someone is a Flame editor (Flame generally has high bandwidth needs). They're working on project A for today. So they connect to the storage being used for that project from their workstation. The next day, they might be working on a different project, so will be connecting to a different set of storage. Other people using different software will connect to the same storage for their tasks, but can have different latency/throughput needs. Obviously, this isn't the approach taken by single person multimedia er... "houses". ;)
- TomVDB 7y agoI know nothing about SAN, but I'm still surprised that it would be faster than a PCIe connected local SSD?
- leoc 7y agoI'm just speculating here, but I assume that online storage isn't faster, but it may be bigger, online, easier for IT to manage, and fast enough. It also seems likely that some employers are worried about engineers or creatives just walking out with commercially sensitive data.
- philjohn 7y agoThis. Have a look at the various Linus Tech Tips videos on their ingestion and editing workflow. There's a reason they upgraded their network to 10Gige and it wasn't for fun.
- jlarocco 7y agoLatency will be higher, but throughput can be nearly the same with a fast enough network.
- justinclift 7y agoDecent network cards (40GbE, etc) with reasonable network NAS/SAN storage can return data in similar time frames to a local PCIe SSD. That's for storage physically located nearby though. eg same building, no more than 1 or two network hops (roughly). Higher end networking from the last decade or so has become really fast. :)
- TomVDB 7y ago40GbE is indeed crazy fast. Didn’t realize that you can have PCIe cards that do this for less than $500.
- justinclift 7y agoFor a home lab or dev/test gear, where getting stuff off Ebay is an ok thing to do, older Mellanox cards can commonly be found for under $100. Sometimes much cheaper. depending on what the seller is doing. :) Looking quickly at the US Ebay just now, here are some older single port ConnectX-2 cards: https://www.ebay.com/itm/Mellanox-MHQH19B-XTR-PCI-E-Single-Port-Connecter-40GBS-Network-Adapter-Card-FP/132546880757 https://www.ebay.com/itm/Mellanox-MHQH19B-XTR-PCI-E-Single-P... US $23.00 each, free shipping. Note - I don't know the seller at all, this is just from a quick look on Ebay. There's a tonne of stuff on there. ConnectX-2 is no longer supported by Mellanox, though the cards themselves are generally pretty bullet proof. Drivers for Linux and FreeBSD come with the OS. :) Drivers for Windows are a bit more complicated. Mellanox has an archive section with the ConnectX-2 drivers, which work for most people. Cabling directly between two cards - instead of going via a switch - is pretty common for people just trying out the tech. It lets them plug one card into (say) a FreeNAS server, with the other card in their desktop or workstation. Removes the ~120MB/s limit of 1GbE, assuming any kind of reasonable disks in the connected NAS. :) If this kind of thing is of interest, probably the best place on the whole internet (not joking) for this stuff is Serve The Homes' Networking forum: https://forums.servethehome.com/index.php?forums/networking.20/ https://forums.servethehome.com/index.php?forums/networking.... Plenty of advice and stuff to read there, and a good place to ask questions from people that have a clue. :)