3 ms·
My favorite term is "annoying sized" data, not enough to warrant clusters and HPC, but enough to make a decent laptop crawl to a halt. It's that uncomfortable i
by anon1253 8y ago
My favorite term is "annoying sized" data, not enough to warrant clusters and HPC, but enough to make a decent laptop crawl to a halt. It's that uncomfortable in-between that makes up the bulk of the data I usually encounter.
- icelancer 8y agoThis is pretty much where we are at, especially with a lot of transforms and processing on high-speed video captures.
- zokier 8y agoThe one nice thing about AWS and its ilk is the ability to spin up big chunky VM for few hours/days for ad-hoc data processing for pretty small amount of money. That can in many cases shift the boundary of annoying sized enough, with the added bonus that your own workstation is not fully occupied by the processing. Of course such approach is not applicable to all scenarios, but it is a useful trick to keep in your sleeve.
- marmaduke 8y agoA while back on RDS we did our migrations by standing up a super fat instance, fail over, migrate, fail back and shutdown the fat instance. This was a great asset, and AWS made this easy.