9 ms·

MemSQL Does Oracle’s Own Demo Ten Times as Fast, Sixty Times Cheaper

156 points by frostmatthew 12y ago

SEJeff 12y ago
The name of this makes me think of that MongoDB is web scale video[1]'s comment, 'Is /dev/null web scale?'. [1] https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs
threeseed 12y ago
Utterly stupid post from a company who should know better. The biggest reasons that companies buy expensive appliances is (a) data sovereignty and (b) ongoing support. The cloud e.g. AWS does have decent answers for this yet. And the companies that are buying these types of appliances like we do couldn't care less about a few hundred thousand dollars.
- koenigdavidmj 12y ago
  > The biggest reasons that companies buy expensive appliances is (a) data sovereignty and (b) ongoing support Don't underestimate reason (c): because some C-levels went golfing together.
  - etherael 12y ago
    That's actually more likely to be reason (a).
- jsolson 12y ago
  In terms of (a), why not build/host the cluster on your own gear? For (b), how do you earn that trust other than continuing to try taking on customers and... well, supporting them?
- tracker1 12y ago
  From TFA: ... $5/hr on Amazon or less than $20k if you wanted to buy 6 servers outright. I actually read the article...
- bhouston 12y ago
  > The biggest reasons that companies buy expensive appliances is (a) data sovereignty and (b) ongoing support. The cloud e.g. AWS does have decent answers for this yet. But they state that clients could buy 6 machines outright for just $20K if you didn't want to trust your data to AWS cloud. So while you can be negative on the cloud, but their solution seems pretty reasonable (cost and speed wise) for the use case presented -- I don't think it is "utterly stupid."
- hyperbovine 12y ago
  Larry, I thought you stepped down?
- t1m 12y ago
  I think you are missing the point. They chose to deploy on AWS because the 'modestly priced' instances they are talking about are gigantically less powerful than Oracle's 'appliances'. The article is about software, algorithms and modern database technology. Oracle has basically been flogging the same code base for twenty odd years, and claiming they are competitive by running their ancient, single node system on more powerful nodes. It is pure BS. Using a modern, distributed, in-memory store like MemSQL on crap nodes (AWS), still beats the pants off Oracle's ancient single node POS running on a super-computer.
deleted 12y ago
[deleted]
timetraveller 12y ago
I wonder if it's a good idea to list comcast as one of you clients...
- dguaraglia 12y ago
  Sure, why not? It's a big company with a lot of brand recognition. The people most likely to be interested in this kind of product will first think 'oh, they have a big client!' before they think 'oh, they are used by a company I dislike because of anecdotal data or their stance on so called net neutrality.' Heck, I'd be even more interested if they told me they serve the NSA or Palantir, as much as I dislike everything they stand for.
  - ericfrenkiel 12y ago
    This would be of interest to you then: http://siliconangle.com/blog/2014/09/29/heres-why-the-intelligence-community-bought-a-stake-in-memsql/ http://siliconangle.com/blog/2014/09/29/heres-why-the-intell...
    - dguaraglia 12y ago
      Ohhh... interesting. Thanks for the link!
- philip1209 12y ago
  "Big companies choose to give us their money instead of to other companies." What's the issue there?
currysausage 12y ago
A font with strokes 1px thick when you scale it to 200 percent? What a great design! I'm sure it looks so cool on the designer's retina display. You don't want me to read your page? Fine, then I won't. Edit: Hehe, now the contrast of my comment is about as bad as the contrast of the blog. I guess criticizing Silicon Valley's favorite design fad of the 2010s is too much for the HN crowd to stomach.
- function_seven 12y ago
  That is irony, your comment's contrast being the result of your complaint. :) In general, I don't think webfonts are yet appropriate for body text. headlines, maybe. But it's a crap shoot if your reader's UA will render them right or not it seems.
  - currysausage 12y ago
    > In general, I don't think webfonts are yet appropriate for body text. Personally, I would agree with that, yes. For body text, I highly prefer a screen-optimized Arial or Georgia over any oh-so-trendy webfont. But I acknowledge that this is the fault of Windows' subpar font rendering. I understand it if designers who have access to a OS with better rendering, and probably even to a hi-res display, don't want to be stuck with "font-family: Helvetica, Arial, sans-serif;" for life. But this is not the point that I was trying to make. Here, they choose a font-face/font-size combination that makes the strokes 0.5px thick. It just physically can't work. What I see on the screen is probably just the artifacts of subpixel hinting. It will look better or worse depending on the rendering engine, but it's still a terrible design to rely on the half-pixels.
    - aikah 12y ago
      I agree, the site is just not readable on my windows machine. If one is going to use thin fonts, he should use it on titles ,not in the body of the text. I'm really suprised at people telling you to "buy glasses or change computer screen",this is typography 101,and this website (HN) often boast about taking UI/UX seriously. Designers,dont do fancy stuffs unless you test them on a wide range of browser or devices.This is a design fail.
pacaro 12y ago
I'm amused that Nikita Shamgunov's bio suggests that he was a 'distinguished senior database engineer' at Microsoft. The title "Distinguished Engineer" at Microsoft refers to a pretty select group of people, not just any senior dev
- bhouston 12y ago
  The lower case "distinguished senior database engineer" seems distinctly different than the awarded title "Distinguished Engineer." I am unsure if he meets the dictionary definition of "distinguished", but if he does, it isn't wrong to use it.
- nikita 12y ago
  That's right. "Distinguished Engineer" is a title at Microsoft. At the time of my tenure at Microsoft my title was "Senior Engineer". The bio was written without referring to the title, but rather the fact that I was "distinguished" with Gold Star awards and HiPo program. I asked to remove "distinguished" from the bio to remove any confusion. It's already live.
  - pacaro 12y ago
    This is a classy response. Corporate bios, like any pr, can walk a fine line in what they communicate.
- isabella1985 12y ago
  distinguished - "successful, authoritative, and commanding great respect" Is English your second language per chance? Not meaning to be rude but you're taking things very literally and seem to think that MS has somehow trademarked the word "distinguished." If a school gives out "Academic Excellence Awards" to it's students, it doesn't mean that only those students who get the award(I don't know what Mr. Shamgunov's title at MS was) can describe themselves as excellent students. As noted below, the fact that it's in lower case and precedes the words "senior database" makes it pretty clear that it's being used as a descriptor not as an official title. Especially since unless you're from MS, no one would even know about it. I see such literal interpretation of English from immigrants that come to the US when they are adults. Unsolicited advise, try reading fictional books to get a hang of colloquial English :)
angryasian 12y ago
is memsql a free product, I can seem to find anything about pricing or otherwise on their site ?
- gcommer 12y ago
  The 60x cheaper benchmark is based on hardware costs, comparing 6 EC2 nodes (they say ~$20k worth of servers) vs a > $million Oracle server.
- deleted 12y ago
  [deleted]
- MasterScrat 12y ago
  I looked for that too and couldn't find it either. This is a very annoying trend. I had the same problem checking out VoltDB. Is this free? What license? What price? In the end looking it up on Wikipedia is the fastest way; for MemSQL it states "Proprietary License".
tkyjonathan 12y ago
There is a lot of use case optimization going on here. The sorting inside the columnar store is in and of itself a huge performance boost. And saying that it scans 138Billion rows per second is completely wrong because it uses a lot of short cuts to get the result and it doesnt even use rows to store the data. There is nothing stopping Oracle using the same optimization for their use case, although they would be better suited if someone from the crowd queried something that MemSQL didnt sort/optimize for.
- nikita 12y ago
  It's hard to define what a "fair" columnar store scan is. We use a technique that is called "segment elimination", which allows to skip over compressed segments of data as you scan. However every solid columnar database uses such techniques. It just happens that Oracle 12c doesn't have the ability to sort columnar indexes, and neither does it have the ability to store columnar indexes on flash or disk, which bloats the overall cost.
  - mbesto 12y ago
    How does your tech differ from SAP HANA's columnar DB?
  - jma24 12y ago
    The much bigger problem with Oracle In-Memory is that a lot of operations like joins and sorts often happen in the SGA, in row form. So you end up needing a truckload of SGA for temporary calculations. And when the SGA runs out, which it invariably does, you end up using TEMPDB. And everything comes to a crawl.
- zdw 12y ago
  Isn't this tanamount to saying: "You used an O(log(n)) solution, but I mandated a O(n) solution, and therefore your solution is wrong even though it's faster" At some point, how you solved the problem becomes moot if you get the same answer and your way is faster in most cases. That's the entire point of most algorithms texts.
xtacy 12y ago
I was hoping the blogpost would give at least a pointer or some insights about the actual experiment. Am I missing some context here?
capkutay 12y ago
MemSQL is doing great work...However from a strategic point of view, I wonder how they plan to compete against 2 of the world's largest DB vendors (SAP and Oracle) at their core product. Oracle's flagship DB has more R&D, sales, marketing than a startup could ever muster even with the help of big name VC's. But I'm sure that's part of startup glory. "no one thought we could do it!" is how it always starts. I suppose they could be an acquisition target for HP or EMC or a company thats lagging in the data management world
- ffk 12y ago
  They will be able to pull in customers who cannot afford to spend 3 million on hardware for a database but still need very high performance. There should be enough to build a business in that niche. Those who can afford the high price tag will likely continue to buy from Oracle and the SAP in the short term. Over time, MemSQL can probably win some of these customers as they establish more credibility and a proven track record.
jandrewrogers 12y ago
It is generally not that difficult to greatly out-perform Oracle on specific benchmarks. Postgres can do it too. Companies do not buy Oracle for the performance per se but as time has gone on Oracle has definitely been losing ground in that arena. MemSQL makes a good product but nothing about their architecture is particularly novel, and other systems have a similar design. I've designed several database engines; I would expect numbers like they demonstrated on a decent columnar implementation with the tradeoffs MemSQL has made. And Oracle has never been optimized for these types of queries because those tradeoffs adversely impact performance in other areas Oracle cares about. Features like compiling queries are mundane; many commercial databases did that a decade ago and it is a standard design element today. Same with the two-tier in-memory and columnar disk storage. I'm sure MemSQL is fast, but "world's fastest" should be assumed to be hyperbole. Most of their benchmarks are table stakes for a modern, real-time analytical database kernel. It is apparently a well-designed product but nothing many other companies are not doing. Here is what I tell people a properly designed database kernel on modern mid-range hardware should be able to do per server: continuously insert millions of records per second all the way through storage, leaving virtually all of the cores idle for some large number of concurrent queries. If you can't saturate 10 GbE on ingest concurrent with saturating 10 GbE on query output, your design is most likely obsolete or broken. Things have moved a very long way since I first started designing massive scale database systems (on Oracle, way back in the day).
- carterschonwald 12y ago
  any good reading for the engineering/algorithms one can use to hit those throughput targets?
  - nikita 12y ago
    - C-store paper - Provably correct lockfree skiplist - Mass tree - BW Tree
- jl6 12y ago
  What in your opinion counts as modern mid-range hardware?
jma24 12y ago
Doing some basic math... Wikipedia is around 80m rows a day, so 4 months of Wikipedia is around 9.5bn rows. But they show 17bn on the graph. Typical columnar compression gives about 11GB per 1bn rows, so 17bn rows should be 187GB. The AWS machines they are using should be c3.4xlarge which are 30GB, and 6 of them is 180GB. But you can't run an in-memory column store at 100% RAM, you need to run it at 50-70% so you have capacity for calculations. Is it just me or do the results not make any sense? Seems likely they actually had 9.5bn or 4 months data, which conveniently is what the graph shows?

156 points by frostmatthew 12y ago

The name of this makes me think of that MongoDB is web scale video[1]'s comment, 'Is /dev/null web scale?'. [1] https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs

threeseed 12y ago

Utterly stupid post from a company who should know better. The biggest reasons that companies buy expensive appliances is (a) data sovereignty and (b) ongoing support. The cloud e.g. AWS does have decent answers for this yet. And the companies that are buying these types of appliances like we do couldn't care less about a few hundred thousand dollars.

koenigdavidmj 12y ago
> The biggest reasons that companies buy expensive appliances is (a) data sovereignty and (b) ongoing support Don't underestimate reason (c): because some C-levels went golfing together.
- etherael 12y ago
  That's actually more likely to be reason (a).
jsolson 12y ago
In terms of (a), why not build/host the cluster on your own gear? For (b), how do you earn that trust other than continuing to try taking on customers and... well, supporting them?
tracker1 12y ago
From TFA: ... $5/hr on Amazon or less than $20k if you wanted to buy 6 servers outright. I actually read the article...
bhouston 12y ago
> The biggest reasons that companies buy expensive appliances is (a) data sovereignty and (b) ongoing support. The cloud e.g. AWS does have decent answers for this yet. But they state that clients could buy 6 machines outright for just $20K if you didn't want to trust your data to AWS cloud. So while you can be negative on the cloud, but their solution seems pretty reasonable (cost and speed wise) for the use case presented -- I don't think it is "utterly stupid."
hyperbovine 12y ago
Larry, I thought you stepped down?
t1m 12y ago
I think you are missing the point. They chose to deploy on AWS because the 'modestly priced' instances they are talking about are gigantically less powerful than Oracle's 'appliances'. The article is about software, algorithms and modern database technology. Oracle has basically been flogging the same code base for twenty odd years, and claiming they are competitive by running their ancient, single node system on more powerful nodes. It is pure BS. Using a modern, distributed, in-memory store like MemSQL on crap nodes (AWS), still beats the pants off Oracle's ancient single node POS running on a super-computer.

deleted 12y ago

[deleted]

timetraveller 12y ago

I wonder if it's a good idea to list comcast as one of you clients...

dguaraglia 12y ago
Sure, why not? It's a big company with a lot of brand recognition. The people most likely to be interested in this kind of product will first think 'oh, they have a big client!' before they think 'oh, they are used by a company I dislike because of anecdotal data or their stance on so called net neutrality.' Heck, I'd be even more interested if they told me they serve the NSA or Palantir, as much as I dislike everything they stand for.
- ericfrenkiel 12y ago
  This would be of interest to you then: http://siliconangle.com/blog/2014/09/29/heres-why-the-intelligence-community-bought-a-stake-in-memsql/ http://siliconangle.com/blog/2014/09/29/heres-why-the-intell...
  - dguaraglia 12y ago
    Ohhh... interesting. Thanks for the link!
philip1209 12y ago
"Big companies choose to give us their money instead of to other companies." What's the issue there?

currysausage 12y ago

A font with strokes 1px thick when you scale it to 200 percent? What a great design! I'm sure it looks so cool on the designer's retina display. You don't want me to read your page? Fine, then I won't. Edit: Hehe, now the contrast of my comment is about as bad as the contrast of the blog. I guess criticizing Silicon Valley's favorite design fad of the 2010s is too much for the HN crowd to stomach.

function_seven 12y ago
That is irony, your comment's contrast being the result of your complaint. :) In general, I don't think webfonts are yet appropriate for body text. headlines, maybe. But it's a crap shoot if your reader's UA will render them right or not it seems.
- currysausage 12y ago
  > In general, I don't think webfonts are yet appropriate for body text. Personally, I would agree with that, yes. For body text, I highly prefer a screen-optimized Arial or Georgia over any oh-so-trendy webfont. But I acknowledge that this is the fault of Windows' subpar font rendering. I understand it if designers who have access to a OS with better rendering, and probably even to a hi-res display, don't want to be stuck with "font-family: Helvetica, Arial, sans-serif;" for life. But this is not the point that I was trying to make. Here, they choose a font-face/font-size combination that makes the strokes 0.5px thick. It just physically can't work. What I see on the screen is probably just the artifacts of subpixel hinting. It will look better or worse depending on the rendering engine, but it's still a terrible design to rely on the half-pixels.
  - aikah 12y ago
    I agree, the site is just not readable on my windows machine. If one is going to use thin fonts, he should use it on titles ,not in the body of the text. I'm really suprised at people telling you to "buy glasses or change computer screen",this is typography 101,and this website (HN) often boast about taking UI/UX seriously. Designers,dont do fancy stuffs unless you test them on a wide range of browser or devices.This is a design fail.

pacaro 12y ago

I'm amused that Nikita Shamgunov's bio suggests that he was a 'distinguished senior database engineer' at Microsoft. The title "Distinguished Engineer" at Microsoft refers to a pretty select group of people, not just any senior dev

bhouston 12y ago
The lower case "distinguished senior database engineer" seems distinctly different than the awarded title "Distinguished Engineer." I am unsure if he meets the dictionary definition of "distinguished", but if he does, it isn't wrong to use it.
nikita 12y ago
That's right. "Distinguished Engineer" is a title at Microsoft. At the time of my tenure at Microsoft my title was "Senior Engineer". The bio was written without referring to the title, but rather the fact that I was "distinguished" with Gold Star awards and HiPo program. I asked to remove "distinguished" from the bio to remove any confusion. It's already live.
- pacaro 12y ago
  This is a classy response. Corporate bios, like any pr, can walk a fine line in what they communicate.
isabella1985 12y ago
distinguished - "successful, authoritative, and commanding great respect" Is English your second language per chance? Not meaning to be rude but you're taking things very literally and seem to think that MS has somehow trademarked the word "distinguished." If a school gives out "Academic Excellence Awards" to it's students, it doesn't mean that only those students who get the award(I don't know what Mr. Shamgunov's title at MS was) can describe themselves as excellent students. As noted below, the fact that it's in lower case and precedes the words "senior database" makes it pretty clear that it's being used as a descriptor not as an official title. Especially since unless you're from MS, no one would even know about it. I see such literal interpretation of English from immigrants that come to the US when they are adults. Unsolicited advise, try reading fictional books to get a hang of colloquial English :)

angryasian 12y ago

is memsql a free product, I can seem to find anything about pricing or otherwise on their site ?

gcommer 12y ago
The 60x cheaper benchmark is based on hardware costs, comparing 6 EC2 nodes (they say ~$20k worth of servers) vs a > $million Oracle server.
deleted 12y ago
[deleted]
MasterScrat 12y ago
I looked for that too and couldn't find it either. This is a very annoying trend. I had the same problem checking out VoltDB. Is this free? What license? What price? In the end looking it up on Wikipedia is the fastest way; for MemSQL it states "Proprietary License".

tkyjonathan 12y ago

There is a lot of use case optimization going on here. The sorting inside the columnar store is in and of itself a huge performance boost. And saying that it scans 138Billion rows per second is completely wrong because it uses a lot of short cuts to get the result and it doesnt even use rows to store the data. There is nothing stopping Oracle using the same optimization for their use case, although they would be better suited if someone from the crowd queried something that MemSQL didnt sort/optimize for.

nikita 12y ago
It's hard to define what a "fair" columnar store scan is. We use a technique that is called "segment elimination", which allows to skip over compressed segments of data as you scan. However every solid columnar database uses such techniques. It just happens that Oracle 12c doesn't have the ability to sort columnar indexes, and neither does it have the ability to store columnar indexes on flash or disk, which bloats the overall cost.
- mbesto 12y ago
  How does your tech differ from SAP HANA's columnar DB?
- jma24 12y ago
  The much bigger problem with Oracle In-Memory is that a lot of operations like joins and sorts often happen in the SGA, in row form. So you end up needing a truckload of SGA for temporary calculations. And when the SGA runs out, which it invariably does, you end up using TEMPDB. And everything comes to a crawl.
zdw 12y ago
Isn't this tanamount to saying: "You used an O(log(n)) solution, but I mandated a O(n) solution, and therefore your solution is wrong even though it's faster" At some point, how you solved the problem becomes moot if you get the same answer and your way is faster in most cases. That's the entire point of most algorithms texts.

xtacy 12y ago

I was hoping the blogpost would give at least a pointer or some insights about the actual experiment. Am I missing some context here?

capkutay 12y ago

MemSQL is doing great work...However from a strategic point of view, I wonder how they plan to compete against 2 of the world's largest DB vendors (SAP and Oracle) at their core product. Oracle's flagship DB has more R&D, sales, marketing than a startup could ever muster even with the help of big name VC's. But I'm sure that's part of startup glory. "no one thought we could do it!" is how it always starts. I suppose they could be an acquisition target for HP or EMC or a company thats lagging in the data management world

ffk 12y ago
They will be able to pull in customers who cannot afford to spend 3 million on hardware for a database but still need very high performance. There should be enough to build a business in that niche. Those who can afford the high price tag will likely continue to buy from Oracle and the SAP in the short term. Over time, MemSQL can probably win some of these customers as they establish more credibility and a proven track record.

jandrewrogers 12y ago

It is generally not that difficult to greatly out-perform Oracle on specific benchmarks. Postgres can do it too. Companies do not buy Oracle for the performance per se but as time has gone on Oracle has definitely been losing ground in that arena. MemSQL makes a good product but nothing about their architecture is particularly novel, and other systems have a similar design. I've designed several database engines; I would expect numbers like they demonstrated on a decent columnar implementation with the tradeoffs MemSQL has made. And Oracle has never been optimized for these types of queries because those tradeoffs adversely impact performance in other areas Oracle cares about. Features like compiling queries are mundane; many commercial databases did that a decade ago and it is a standard design element today. Same with the two-tier in-memory and columnar disk storage. I'm sure MemSQL is fast, but "world's fastest" should be assumed to be hyperbole. Most of their benchmarks are table stakes for a modern, real-time analytical database kernel. It is apparently a well-designed product but nothing many other companies are not doing. Here is what I tell people a properly designed database kernel on modern mid-range hardware should be able to do per server: continuously insert millions of records per second all the way through storage, leaving virtually all of the cores idle for some large number of concurrent queries. If you can't saturate 10 GbE on ingest concurrent with saturating 10 GbE on query output, your design is most likely obsolete or broken. Things have moved a very long way since I first started designing massive scale database systems (on Oracle, way back in the day).

carterschonwald 12y ago
any good reading for the engineering/algorithms one can use to hit those throughput targets?
- nikita 12y ago
  - C-store paper - Provably correct lockfree skiplist - Mass tree - BW Tree
jl6 12y ago
What in your opinion counts as modern mid-range hardware?

jma24 12y ago

Doing some basic math... Wikipedia is around 80m rows a day, so 4 months of Wikipedia is around 9.5bn rows. But they show 17bn on the graph. Typical columnar compression gives about 11GB per 1bn rows, so 17bn rows should be 187GB. The AWS machines they are using should be c3.4xlarge which are 30GB, and 6 of them is 180GB. But you can't run an in-memory column store at 100% RAM, you need to run it at 50-70% so you have capacity for calculations. Is it just me or do the results not make any sense? Seems likely they actually had 9.5bn or 4 months data, which conveniently is what the graph shows?