3 ms·
> Redshift falls over near the 100TB level. This is based on what? They have stated in various presentations that they have customers using 1pb clusters.
by squirrelly 7y ago
> Redshift falls over near the 100TB level.
This is based on what? They have stated in various presentations that they have customers using 1pb clusters.
- chucky_z 7y agoIn my personal experience. There's a re:invent presentation of them having an in-house 1pb+ cluster. What they don't tell you is that 101 8xl nodes is half a million dollars a year in reserved instances, before you include any other costs associated with it; and that particular workflow (log scanning) is very nicely suited for any columnar store. Operations are also a disaster with Redshift; anything where you have to touch the cluster itself at any scale past a handful of nodes typically requires a support ticket with "hey, when this breaks, please work your magic and fix it?" There's also the issue of tuning your queues, which is a whole extra layer that you, the customer, must tune. Their suggestion tools are getting better on that front though. Just use something else if you have more than a few TB, or you have a ton of time and money to just throw around.
- squirrelly 7y agoHa, $500k/year is the amount a salesperson can knock off the cost of Teradata 100TB setup without calling their boss. As for ops stuff, I guess YMMV. We’ve never had any significant issues. Tuning is the same or less than on-prem systems IME. Perf is still pretty much unmatched in the cloud, especially perf/$. Try to do a join on 100tb in a competing system (e.g. BQ) and it won’t finish.