10 ms·

VoltDB - in memory, ACID compliant, partitioned, SQL database

60 points by bensummers 16y ago

braindead_in 16y ago
I guess in memory means that it never writes data to the disks. So what happens when the machine reboots?
- cx01 16y ago
  From the FAQ: "Durability: VoltDB provides both replication of partitions (known as K-safety) and periodic database snapshots to ensure the availability of the data."
- mmagin 16y ago
  Their FAQ states: Durability: VoltDB provides both replication of partitions (known as K-safety) and periodic database snapshots to ensure the availability of the data. I suppose it depends on your definition of "D". Seems like this might be acceptable if you had different replicas on different power sources, etc.
- jws 16y ago
  I expect there is a persistent store of some sort, you'll have to register to find out. I'm also unable to find a license without registering. It does appear from the FAQ that you build your transactions as Java stored procedures. Constraining a generic SQL database that way will make it much easier to distribute and scale.
- deleted 16y ago
  [deleted]
- tlack 16y ago
  In-memory DBs can take a lot of excellent shortcuts but I have to wonder if their usability is limited. After all, I'd think most interesting data sets end up quickly eclipsing the available RAM in most off the shelf/affordable servers.
  - cx01 16y ago
    Keep in mind that VoltDB targets the OLTP market, where datasets tend to be not that large. If you build a small cluster of 50 nodes with 256GB RAM each, you have about 4.3TB of storage (with a replication factor of 3).
st3fan 16y ago
Have to fill in a form to see a whitepaper. Yeah whatever.
- Pahalial 16y ago
  They're just gauging interest and making sure to get emails they can push adoption to later, it's a classic sales technique. Don't hate. Regardless, the links they send are open: VoltDB product overview: http://www.voltdb.com/_pdf/VoltDBOverview.pdf http://www.voltdb.com/_pdf/VoltDBOverview.pdf VoltDB Technical Architecture white paper: http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pdf http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...
- jbellis 16y ago
  It's based on the work discussed in http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf
xtacy 16y ago
I don't understand how they could avoid locks. They say that each replica runs independently and hence that avoids locking; what about write updates to the same object from different clients? That would need locking of some form at some level. They should also explicitly discuss the failure models for which they're guaranteeing Durability. If you're scaling to "web-scale", and running this db in a data-centre, then, a single failure would wipe out a rack of machines. Coordinated failures are not uncommon in data-centres. What about those?
- itistoday 16y ago
  From their whitepaper: "Conventional databases experience disk and user stalls within transactions. Rather than let the CPU be idle during the stalls, those DBMSs interleave SQL execution from multiple transactions during the waits so the CPU is always busy. This is what requires much of the complex latching and locking overhead. VoltDB doesn’t experience user stalls (since transactions happen within stored procedures) or disk stalls (because VoltDB processes data in main memory). Therefore, it is able to eliminate the overhead associated with multi-threading (latching) and locking. Each VoltDB execution engine is single-threaded and contains a queue of transaction requests, which it executes sequentially—and exclusively—against its data. Elimination of stalls and the need for locking and latching overhead and allows typical VoltDB SQL operations to complete in microseconds. For single-partition transactions, each VoltDB engine operates autonomously. For multi-partition transactions, one engine distributes and coordinates work plans for the other engines. VoltDB assumes that an application designer can construct a partitioning/cloning scheme and a transaction design that makes a large majority of the transactions local to a single virtual node. Many common applications such as order-fulfillment, software as a service (SaaS), Web 2.0 and trading systems have this property." http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pdf http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...
  - stingraycharles 16y ago
    That still doesn't completely answer the question how it avoids race conditions and the likes without locking. Assume I'm executing multiple long-running transactions, each hitting and modifying a lot (millions) of different objects. How are race conditions avoided in this case ? One engine that distributes and coordinates work plans for the other engines sounds nice, but does this essentially mean that only one of these transactions is able to run at the same time, even when they could be running concurrently when using locking?
deleted 16y ago
[deleted]
superjared 16y ago
Early adopter signup gives access to the source, which is licensed under GPLv3. However, there is a binding Beta agreement which states that I may not redistribute the code/application. The VoltDB software is being published as open source software under the Gnu Public License V3. In other words, as Early Release program members, you now have access to the source files for VoltDB and the development tools for browsing the source code and the current issues lists. Please note, however, that the Beta agreement is still in effect until VoltDB is officially available. So we ask that you not redistribute the source or binaries beyond your own use at this time. Is this legally enforceable?
- lsb 16y ago
  Section 7: All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. I don't think it's enforceable, but IANAL.
  - Scriptor 16y ago
    What prevents them from just using the parts of the GPL they want?
    - koenigdavidmj 16y ago
      >Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
      - mbreese 16y ago
        Somewhat ironically, the reason for this is that the FSF holds the copyright to the GPL, so only they can change it. Well, you could probably change it, but you couldn't call it the GPL anymore... and it would probably be copyright infringement. I wonder what would happen if the GPL itself was licensed under the GPL... I suspect it would start raining turtles. :)
ck2 16y ago
So, how fast does it run ALTER ?
sovande 16y ago
GPLv3, wonder what their business model is.

60 points by bensummers 16y ago

braindead_in 16y ago

I guess in memory means that it never writes data to the disks. So what happens when the machine reboots?

cx01 16y ago
From the FAQ: "Durability: VoltDB provides both replication of partitions (known as K-safety) and periodic database snapshots to ensure the availability of the data."
mmagin 16y ago
Their FAQ states: Durability: VoltDB provides both replication of partitions (known as K-safety) and periodic database snapshots to ensure the availability of the data. I suppose it depends on your definition of "D". Seems like this might be acceptable if you had different replicas on different power sources, etc.
jws 16y ago
I expect there is a persistent store of some sort, you'll have to register to find out. I'm also unable to find a license without registering. It does appear from the FAQ that you build your transactions as Java stored procedures. Constraining a generic SQL database that way will make it much easier to distribute and scale.
deleted 16y ago
[deleted]
tlack 16y ago
In-memory DBs can take a lot of excellent shortcuts but I have to wonder if their usability is limited. After all, I'd think most interesting data sets end up quickly eclipsing the available RAM in most off the shelf/affordable servers.
- cx01 16y ago
  Keep in mind that VoltDB targets the OLTP market, where datasets tend to be not that large. If you build a small cluster of 50 nodes with 256GB RAM each, you have about 4.3TB of storage (with a replication factor of 3).

st3fan 16y ago

Have to fill in a form to see a whitepaper. Yeah whatever.

Pahalial 16y ago
They're just gauging interest and making sure to get emails they can push adoption to later, it's a classic sales technique. Don't hate. Regardless, the links they send are open: VoltDB product overview: http://www.voltdb.com/_pdf/VoltDBOverview.pdf http://www.voltdb.com/_pdf/VoltDBOverview.pdf VoltDB Technical Architecture white paper: http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pdf http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...
jbellis 16y ago
It's based on the work discussed in http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf

xtacy 16y ago

I don't understand how they could avoid locks. They say that each replica runs independently and hence that avoids locking; what about write updates to the same object from different clients? That would need locking of some form at some level. They should also explicitly discuss the failure models for which they're guaranteeing Durability. If you're scaling to "web-scale", and running this db in a data-centre, then, a single failure would wipe out a rack of machines. Coordinated failures are not uncommon in data-centres. What about those?

itistoday 16y ago
From their whitepaper: "Conventional databases experience disk and user stalls within transactions. Rather than let the CPU be idle during the stalls, those DBMSs interleave SQL execution from multiple transactions during the waits so the CPU is always busy. This is what requires much of the complex latching and locking overhead. VoltDB doesn’t experience user stalls (since transactions happen within stored procedures) or disk stalls (because VoltDB processes data in main memory). Therefore, it is able to eliminate the overhead associated with multi-threading (latching) and locking. Each VoltDB execution engine is single-threaded and contains a queue of transaction requests, which it executes sequentially—and exclusively—against its data. Elimination of stalls and the need for locking and latching overhead and allows typical VoltDB SQL operations to complete in microseconds. For single-partition transactions, each VoltDB engine operates autonomously. For multi-partition transactions, one engine distributes and coordinates work plans for the other engines. VoltDB assumes that an application designer can construct a partitioning/cloning scheme and a transaction design that makes a large majority of the transactions local to a single virtual node. Many common applications such as order-fulfillment, software as a service (SaaS), Web 2.0 and trading systems have this property." http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pdf http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper...
- stingraycharles 16y ago
  That still doesn't completely answer the question how it avoids race conditions and the likes without locking. Assume I'm executing multiple long-running transactions, each hitting and modifying a lot (millions) of different objects. How are race conditions avoided in this case ? One engine that distributes and coordinates work plans for the other engines sounds nice, but does this essentially mean that only one of these transactions is able to run at the same time, even when they could be running concurrently when using locking?

deleted 16y ago

[deleted]

superjared 16y ago

Early adopter signup gives access to the source, which is licensed under GPLv3. However, there is a binding Beta agreement which states that I may not redistribute the code/application. The VoltDB software is being published as open source software under the Gnu Public License V3. In other words, as Early Release program members, you now have access to the source files for VoltDB and the development tools for browsing the source code and the current issues lists. Please note, however, that the Beta agreement is still in effect until VoltDB is officially available. So we ask that you not redistribute the source or binaries beyond your own use at this time. Is this legally enforceable?

lsb 16y ago
Section 7: All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. I don't think it's enforceable, but IANAL.
- Scriptor 16y ago
  What prevents them from just using the parts of the GPL they want?
  - koenigdavidmj 16y ago
    >Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
    - mbreese 16y ago
      Somewhat ironically, the reason for this is that the FSF holds the copyright to the GPL, so only they can change it. Well, you could probably change it, but you couldn't call it the GPL anymore... and it would probably be copyright infringement. I wonder what would happen if the GPL itself was licensed under the GPL... I suspect it would start raining turtles. :)

ck2 16y ago

So, how fast does it run ALTER ?

sovande 16y ago

GPLv3, wonder what their business model is.