3 ms·
Even getting the data to that machine presents a problem that seems to undermine the value of it entirely: As you mentioned, the real value in this is linear pr
by corresation 14y ago
Even getting the data to that machine presents a problem that seems to undermine the value of it entirely: As you mentioned, the real value in this is linear processing of large sets of data, but the storage is ephemeral so your process has to be some variation of firing this instance up, copy TBs of data to the machine, and then do linear processing. Given that you have to get the data there, the value of the high aggregated gross-throughput seems secondary -- just stream process it, etc.
I'm having a tough time seeing where this type of instance fits.
- edvinasbartkus 14y agoThat's why AWS Data Pipeline came along. http://aws.amazon.com/datapipeline/ http://aws.amazon.com/datapipeline/
- wmf 14y agoData Pipeline looks like a fine orchestration service, but it's not going to ingest 48 TB of data any faster than you can do it yourself. Which is probably not that fast.
- throw_away 14y agomaybe + http://aws.amazon.com/importexport/ http://aws.amazon.com/importexport/