4 ms·
OLTP. I personally believe Hadoop has got the OLAP problem-space covered.
by justinsb 15y ago
OLTP. I personally believe Hadoop has got the OLAP problem-space covered.
- jacques_chester 15y agoWhat are your most important non-functional requirements? In particular, how do you see the tradeoff between durability and performance?
- justinsb 15y agoYou've totally hit the nail on the head there. Up until a month ago, it was a fairly obsessive compulsion around performance. Durability and reliability are features of the design, so it was about getting performance under the design. However, the AWS outage has (we hope) put reliability back on the map. So maybe now we don't have to be the fastest database, if we're the fastest reliable database. That's a liberating change of perspective, which has been behind many of the recent advances.
- jacques_chester 15y agoDoes durability absolutely trump performance, or are you prepared to trade off? For example, if durability is an absolute must (as it is for current databases), then your main bottlenecks would be a) spinning rust and b) network traffic to distribute copies of the data throughout the network. SSDs will help this a lot, but already you're back where almost everyone has already started. If you're prepared to accept highly replicated in-memory copies as "durable", you can already make your overall system hundreds or even thousands of times faster. So how do you define durability? 1 copy on 1 disk? x copies on y disks in the same computer? x copies in y disks in different computers? x copies in RAM of y different computers?