2 ms·
Hadoop is free for 400 (or n) computers. Let's say you need 100 computers (full time) to process the equivalent clustered query in Oracle, given the time needed
by ebiester 15y ago
Hadoop is free for 400 (or n) computers. Let's say you need 100 computers (full time) to process the equivalent clustered query in Oracle, given the time needed to get the formatted data. How much do the licenses for that cost? (And if it's a free system, you're still going to have to do an ETL job, so why not ETL it to a free solution?)
You need to process a couple terabytes of data, but you only need to do it once a month. However, even the highly tuned SQL will bring the system to its knees. So, 90% of the time really expensive servers are idling because of these peak processing needs. Once a month. (And Oracle isn't going to be available on a cloud service.)
So, yeah, if someone had a free/cheap SQL solution for these kinds of ETL datasets that was able to handle these embarrassingly parallel problems, people might be interested. Perhaps you are aware of solutions I am not.