4 ms·
It would be interesting to know how long it takes to run locally on a single instance
by joebo 15y ago
It would be interesting to know how long it takes to run locally on a single instance
- hogu 15y agoagreed, actually, I want to know how long it takes to run on say... an i7, I've gotten pretty poor performance out of those amazon small instances when doing alot of cpu oriented tasks.
- thirdhaf 15y agoLet's pessimistically assume you have to read all information associated with each file from disk. I suspect this sort of query is IO bound since processing is minimal and assuming we can mostly avoid fragmentation on disk we could feasibly read data at 100MB/s with a consumer level magnetic drive. The article says the data set is 300GB so we could feasibly get this done in an hour.
- joshu 15y agoYeah. That can't possibly be the right thing to do...