5 ms·

There is a lot of use case optimization going on here. The sorting inside the columnar store is in and of itself a huge performance boost. And saying that it sc

There is a lot of use case optimization going on here. The sorting inside the columnar store is in and of itself a huge performance boost. And saying that it scans 138Billion rows per second is completely wrong because it uses a lot of short cuts to get the result and it doesnt even use rows to store the data. There is nothing stopping Oracle using the same optimization for their use case, although they would be better suited if someone from the crowd queried something that MemSQL didnt sort/optimize for.

nikita 12y ago
It's hard to define what a "fair" columnar store scan is. We use a technique that is called "segment elimination", which allows to skip over compressed segments of data as you scan. However every solid columnar database uses such techniques. It just happens that Oracle 12c doesn't have the ability to sort columnar indexes, and neither does it have the ability to store columnar indexes on flash or disk, which bloats the overall cost.
- mbesto 12y ago
  How does your tech differ from SAP HANA's columnar DB?
- jma24 12y ago
  The much bigger problem with Oracle In-Memory is that a lot of operations like joins and sorts often happen in the SGA, in row form. So you end up needing a truckload of SGA for temporary calculations. And when the SGA runs out, which it invariably does, you end up using TEMPDB. And everything comes to a crawl.
zdw 12y ago
Isn't this tanamount to saying: "You used an O(log(n)) solution, but I mandated a O(n) solution, and therefore your solution is wrong even though it's faster" At some point, how you solved the problem becomes moot if you get the same answer and your way is faster in most cases. That's the entire point of most algorithms texts.
- acbart 12y ago
  Assuming the correctness of the algorithm is maintained!
yellow_and_gray 12y ago
> There is nothing stopping Oracle using the same optimization There is. Skill, bureaucracy, grit, and the right motive. The meat here is MemSQL's sophisticated indexing with splay trees. http://en.wikipedia.org/wiki/Splay_tree http://en.wikipedia.org/wiki/Splay_tree
- deleted 12y ago
  [deleted]
t1m 12y ago
Then again, MemSQL didn't design the benchmark. They beat Oracle at their own benchmark on massively cheaper nodes.
- pyre 12y ago
  That doesn't mean a whole lot. Back in the early '00s, ATI and Nvidia were caught optimizing for certain benchmarks that were commonly used to compare video card performance (IIRC one of them was caught with code that specifically cut corners if it detected being run by the benchmark program). I'm not saying MemSQL is doing anything nefarious, but the fact that the benchmark tests are controlled by someone else doesn't carry a lot of weight.
  - t1m 12y ago
    I would say that doing the other guy's benchmark is the most objective thing you can do. You should take it one step further and include other competitive products. In this case, it's pretty easy to beat the crap out of Oracle, it's an elephant. I would challenge the MemDB guys to run the same benchmark against VoltDB and report on that. Also, let's get down to brass tacks - I don't see the actual queries that are being graphed in the MemDB post, and what is that graph actually measuring? I think the LMDB team has done a really good job with this type of comparative benchmark. They took the LevelDB benches, then ran them against a lot of competitive products, both new and legacy, and gave a really good view of the entire field, in terms of performance on a number of dimensions: http://symas.com/mdb/inmem/ http://symas.com/mdb/inmem/ http://symas.com/mdb/inmem/scaling.html http://symas.com/mdb/inmem/scaling.html

nikita 12y ago

It's hard to define what a "fair" columnar store scan is. We use a technique that is called "segment elimination", which allows to skip over compressed segments of data as you scan. However every solid columnar database uses such techniques. It just happens that Oracle 12c doesn't have the ability to sort columnar indexes, and neither does it have the ability to store columnar indexes on flash or disk, which bloats the overall cost.

mbesto 12y ago
How does your tech differ from SAP HANA's columnar DB?
jma24 12y ago
The much bigger problem with Oracle In-Memory is that a lot of operations like joins and sorts often happen in the SGA, in row form. So you end up needing a truckload of SGA for temporary calculations. And when the SGA runs out, which it invariably does, you end up using TEMPDB. And everything comes to a crawl.

zdw 12y ago

Isn't this tanamount to saying: "You used an O(log(n)) solution, but I mandated a O(n) solution, and therefore your solution is wrong even though it's faster" At some point, how you solved the problem becomes moot if you get the same answer and your way is faster in most cases. That's the entire point of most algorithms texts.

acbart 12y ago
Assuming the correctness of the algorithm is maintained!

yellow_and_gray 12y ago

> There is nothing stopping Oracle using the same optimization There is. Skill, bureaucracy, grit, and the right motive. The meat here is MemSQL's sophisticated indexing with splay trees. http://en.wikipedia.org/wiki/Splay_tree http://en.wikipedia.org/wiki/Splay_tree

deleted 12y ago
[deleted]

t1m 12y ago

Then again, MemSQL didn't design the benchmark. They beat Oracle at their own benchmark on massively cheaper nodes.

pyre 12y ago
That doesn't mean a whole lot. Back in the early '00s, ATI and Nvidia were caught optimizing for certain benchmarks that were commonly used to compare video card performance (IIRC one of them was caught with code that specifically cut corners if it detected being run by the benchmark program). I'm not saying MemSQL is doing anything nefarious, but the fact that the benchmark tests are controlled by someone else doesn't carry a lot of weight.
- t1m 12y ago
  I would say that doing the other guy's benchmark is the most objective thing you can do. You should take it one step further and include other competitive products. In this case, it's pretty easy to beat the crap out of Oracle, it's an elephant. I would challenge the MemDB guys to run the same benchmark against VoltDB and report on that. Also, let's get down to brass tacks - I don't see the actual queries that are being graphed in the MemDB post, and what is that graph actually measuring? I think the LMDB team has done a really good job with this type of comparative benchmark. They took the LevelDB benches, then ran them against a lot of competitive products, both new and legacy, and gave a really good view of the entire field, in terms of performance on a number of dimensions: http://symas.com/mdb/inmem/ http://symas.com/mdb/inmem/ http://symas.com/mdb/inmem/scaling.html http://symas.com/mdb/inmem/scaling.html