6 ms·

Voltdb = memory only database, thus unfair comparision...

by kwyjibo 16y ago

Retric 16y ago
RAM is cheap and 32+GB is a lot of data so for a wide range of real world workloads can fit the database in memory. (You can get a server with 96GB on a PowerEdge R710 for less than $6,066$ today if it's 20x faster than a traditional database you will save money.) The problem is MySQL etc doesn’t are not designed for this so while they become a lot faster they are still relatively slow. Also, RAM is only getting larger so once you make the transition to a RAM based DB you are unlikely to need to transition back. PS: I suspect most real world workloads revolve around small datasets that connect to big blobs of data. Think Users/Projects linking to Pictures, Documents, and Video etc. You don't edit documents in your database but you do want to track version information, users, permissions timestamps, approval etc. It's often a good idea to mix a Key Value style data store with a front end SQL database.
jasongullickson 16y ago
I think this could be a worthwhile comparison if tested on a platform where the in-memory store is as safe as traditional hard-disk storage, say a machine with some form of static (non-volatile) RAM for main memory. Thus the question would be, does the performance advantage of VoltDB's approach outweigh the penalty of static RAM performance?
- dedward 16y ago
  What would that static ram device be, and how would you scale it out?
  - jasongullickson 16y ago
    I'm not sure that such a device exists.
    - nuxi 16y ago
      HP is working on memristors - http://www.hpl.hp.com/news/2008/apr-jun/engineering_memristor.html?jumpid=reg_R1002_USEN http://www.hpl.hp.com/news/2008/apr-jun/engineering_memristo.... According to wikipedia they already created such a prototype - http://en.wikipedia.org/wiki/Memristor#Potential_applications http://en.wikipedia.org/wiki/Memristor#Potential_application....
      - jasongullickson 16y ago
        I should have been more specific. I'm familiar with all sorts of non-volatile memory (flash, etc.) but I'm not aware of any modern computer that uses it as main memory (that is, directly addressable RAM as opposed to connected via a storage bus). I think the last thing I used that would qualify as such was the Apple Newton.
  - domnit 16y ago
    Phase-change memory.
sgift 16y ago
Even if it is fair I only get two points from this article (and their product page): * A system which holds all data in memory is faster than a system which uses memory and disks. True, but not really news. * If you leave out features your implementation can be faster (e.g. they only support transactions which span a single stored procedure). Again .. no news here. Something else I've missed?
- JoachimSchipper 16y ago
  Not really, but do note that it is spectacularly faster.
- rayval 16y ago
  * A system that is designed for mostly disk will run faster when run in memory, but won't be as fast as one designed from the outset for in-memory operation. * Leaving out features is harder than adding features, because you have to judiciously decide which ones to leave out. A large bushel o' features can have many different subsets, and only some of these subsets are worthwhile.
  - jhugg 16y ago
    I like this point. At VoltDB, we like to think we've removed a different set of features than many other scalable solutions, leaving a product that is useful for a set of problems that might not have a great option right now. That said, VoltDB's architecture was built for in-memory and horizontal scaling from day 1. Based on hstore (http://hstore.cs.brown.edu/ http://hstore.cs.brown.edu/), it's not simply a removal of what's in existing systems.
barredo 16y ago
So, it would be better if compared with MySQL's "MEMORY" storage engine?
- jhugg 16y ago
  While VoltDB's persistence story is perhaps bleeding edge (active replication with rolling snapshots as a double backup), MySQL MEMORY storage has none.
- gaius 16y ago
  Hell, with SQLite's :memory: database (which can be persisted to disk using the Backup API).
drallison 16y ago
RAM is the new disk. See http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf does a nice job of looking at the practicality. See also the April 14, 2010 video at http://ee380.stanford.edu http://ee380.stanford.edu. The Stanford RAMCloud project page, http://fiz.stanford.edu:8081/display/ramcloud/Home http://fiz.stanford.edu:8081/display/ramcloud/Home, tracks recent activity in the area.

Voltdb = memory only database, thus unfair comparision...

by kwyjibo 16y ago

Voltdb = memory only database, thus unfair comparision...

RAM is cheap and 32+GB is a lot of data so for a wide range of real world workloads can fit the database in memory. (You can get a server with 96GB on a PowerEdge R710 for less than $6,066$ today if it's 20x faster than a traditional database you will save money.) The problem is MySQL etc doesn’t are not designed for this so while they become a lot faster they are still relatively slow. Also, RAM is only getting larger so once you make the transition to a RAM based DB you are unlikely to need to transition back. PS: I suspect most real world workloads revolve around small datasets that connect to big blobs of data. Think Users/Projects linking to Pictures, Documents, and Video etc. You don't edit documents in your database but you do want to track version information, users, permissions timestamps, approval etc. It's often a good idea to mix a Key Value style data store with a front end SQL database.

jasongullickson 16y ago

I think this could be a worthwhile comparison if tested on a platform where the in-memory store is as safe as traditional hard-disk storage, say a machine with some form of static (non-volatile) RAM for main memory. Thus the question would be, does the performance advantage of VoltDB's approach outweigh the penalty of static RAM performance?

dedward 16y ago
What would that static ram device be, and how would you scale it out?
- jasongullickson 16y ago
  I'm not sure that such a device exists.
  - nuxi 16y ago
    HP is working on memristors - http://www.hpl.hp.com/news/2008/apr-jun/engineering_memristor.html?jumpid=reg_R1002_USEN http://www.hpl.hp.com/news/2008/apr-jun/engineering_memristo.... According to wikipedia they already created such a prototype - http://en.wikipedia.org/wiki/Memristor#Potential_applications http://en.wikipedia.org/wiki/Memristor#Potential_application....
    - jasongullickson 16y ago
      I should have been more specific. I'm familiar with all sorts of non-volatile memory (flash, etc.) but I'm not aware of any modern computer that uses it as main memory (that is, directly addressable RAM as opposed to connected via a storage bus). I think the last thing I used that would qualify as such was the Apple Newton.
- domnit 16y ago
  Phase-change memory.

sgift 16y ago

Even if it is fair I only get two points from this article (and their product page): * A system which holds all data in memory is faster than a system which uses memory and disks. True, but not really news. * If you leave out features your implementation can be faster (e.g. they only support transactions which span a single stored procedure). Again .. no news here. Something else I've missed?

JoachimSchipper 16y ago
Not really, but do note that it is spectacularly faster.
rayval 16y ago
* A system that is designed for mostly disk will run faster when run in memory, but won't be as fast as one designed from the outset for in-memory operation. * Leaving out features is harder than adding features, because you have to judiciously decide which ones to leave out. A large bushel o' features can have many different subsets, and only some of these subsets are worthwhile.
- jhugg 16y ago
  I like this point. At VoltDB, we like to think we've removed a different set of features than many other scalable solutions, leaving a product that is useful for a set of problems that might not have a great option right now. That said, VoltDB's architecture was built for in-memory and horizontal scaling from day 1. Based on hstore (http://hstore.cs.brown.edu/ http://hstore.cs.brown.edu/), it's not simply a removal of what's in existing systems.

barredo 16y ago

So, it would be better if compared with MySQL's "MEMORY" storage engine?

jhugg 16y ago
While VoltDB's persistence story is perhaps bleeding edge (active replication with rolling snapshots as a double backup), MySQL MEMORY storage has none.
gaius 16y ago
Hell, with SQLite's :memory: database (which can be persisted to disk using the Backup API).

drallison 16y ago

RAM is the new disk. See http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf does a nice job of looking at the practicality. See also the April 14, 2010 video at http://ee380.stanford.edu http://ee380.stanford.edu. The Stanford RAMCloud project page, http://fiz.stanford.edu:8081/display/ramcloud/Home http://fiz.stanford.edu:8081/display/ramcloud/Home, tracks recent activity in the area.