3 ms·

The Bandwidth of Sneakernet to the Cloud

8 points by lmacvittie 17y ago

gdp 17y ago
It brings a whole new danger to the idea of a "disk crash" though, don't you think?
- kevindication 17y ago
  Consider also that the weight of just the hard drives is about 9,300 pounds (7.2oz * 20704). Expect a spectacular impact if that van is even capable of carrying such a load. ;-) But this does beg the question, why choose 160GB drives? Even doubling it to 320GB shouldn't hurt the bottom line much. Density per dollar is going to be very important for the Van-based Data Packet.
  - masklinn 17y ago
    > Density per dollar is going to be very important for the Van-based Data Packet. The dollar part actually isn't that important, given you can reuse your drives for later transfers (TeraScale's boxes zip around, AWS sends back your storage devices when the import is done) so unless you're hitting a density/dollar cliff, you're probably going to go all density with TB drives and such. FWIW, in 2002 Terascale was already packing its sneakernet boxes with 300GB drives (7/box)
yellowbkpk 17y ago
This calculation only includes getting the data physically near to the cloud. What about uploading it into the cloud?Surely transferring 3 million GB at around 40 MB/sec would lower the average? Edit: My calculations say it'll take 2.7 years to read all that data if you go at 40MBps. Hopefully they can parallelize that across a few dozen machines...
- fhars 17y ago
  You'd actualy put the disks into a Sun X4500 which you can just put into a rack at the destination site: http://www.c0t0d0s0.org/archives/4260-Less-known-Solaris-Features-Remote-Mirror-with-AVS-Part-7-Truck-based-synchronisation.html http://www.c0t0d0s0.org/archives/4260-Less-known-Solaris-Fea...
- masklinn 17y ago
  > My calculations say it'll take 2.7 years to read all that data if you go at 40MBps. 40MBps is very low, especially given the kind of files transferred over a sneakernets (it's basically the best-case for purely sequential transfers): even fairly slow drives (e.g. WD Caviar Green) are going to yield ~70MBps average transfer speeds (and velociraptors are going to be around 100MBps). And it's pretty trivial to parallelize disks over fast (e.g. 10gig or infiniband) connections. Furthermore, 3000 TB is quite "out there" with the current datasets: Google's transfers are still at a few hundred TB (and they do use sneakernets for that). Finally, modern sneakernets tend not to use raw drives: AWS Import/Export suggests full devices (maximum size is a 8U rack), Terascale Sneakernet boxes are a full PC with a bunch of drives and a Gig or 10Gig network card.
burke 17y ago
The way I see it, the bandwidth is astronomically high, with 4.6 hour latency. Maybe they're using TCP/HiP (Highway Protocol).

8 points by lmacvittie 17y ago

gdp 17y ago

It brings a whole new danger to the idea of a "disk crash" though, don't you think?

kevindication 17y ago
Consider also that the weight of just the hard drives is about 9,300 pounds (7.2oz * 20704). Expect a spectacular impact if that van is even capable of carrying such a load. ;-) But this does beg the question, why choose 160GB drives? Even doubling it to 320GB shouldn't hurt the bottom line much. Density per dollar is going to be very important for the Van-based Data Packet.
- masklinn 17y ago
  > Density per dollar is going to be very important for the Van-based Data Packet. The dollar part actually isn't that important, given you can reuse your drives for later transfers (TeraScale's boxes zip around, AWS sends back your storage devices when the import is done) so unless you're hitting a density/dollar cliff, you're probably going to go all density with TB drives and such. FWIW, in 2002 Terascale was already packing its sneakernet boxes with 300GB drives (7/box)

yellowbkpk 17y ago

This calculation only includes getting the data physically near to the cloud. What about uploading it into the cloud?Surely transferring 3 million GB at around 40 MB/sec would lower the average? Edit: My calculations say it'll take 2.7 years to read all that data if you go at 40MBps. Hopefully they can parallelize that across a few dozen machines...

fhars 17y ago
You'd actualy put the disks into a Sun X4500 which you can just put into a rack at the destination site: http://www.c0t0d0s0.org/archives/4260-Less-known-Solaris-Features-Remote-Mirror-with-AVS-Part-7-Truck-based-synchronisation.html http://www.c0t0d0s0.org/archives/4260-Less-known-Solaris-Fea...
masklinn 17y ago
> My calculations say it'll take 2.7 years to read all that data if you go at 40MBps. 40MBps is very low, especially given the kind of files transferred over a sneakernets (it's basically the best-case for purely sequential transfers): even fairly slow drives (e.g. WD Caviar Green) are going to yield ~70MBps average transfer speeds (and velociraptors are going to be around 100MBps). And it's pretty trivial to parallelize disks over fast (e.g. 10gig or infiniband) connections. Furthermore, 3000 TB is quite "out there" with the current datasets: Google's transfers are still at a few hundred TB (and they do use sneakernets for that). Finally, modern sneakernets tend not to use raw drives: AWS Import/Export suggests full devices (maximum size is a 8U rack), Terascale Sneakernet boxes are a full PC with a bunch of drives and a Gig or 10Gig network card.

burke 17y ago

The way I see it, the bandwidth is astronomically high, with 4.6 hour latency. Maybe they're using TCP/HiP (Highway Protocol).