6 ms·

RethinkDB (YC S09) Raises $1.2 Million For Its Database For Solid-State Drives

114 points by jasonlbaptiste 17y ago

ShabbyDoo 17y ago
"There’s obviously risk involved with trying to redefine how people structure their databases" TechCrunch misses the point that Rethink is explicitly not doing this. The MySQL engine is below the SQL parsing layer, so as-is MySQL apps should be able to run against it.
- davidu 17y ago
  That's in theory. In practice, MySQL has different syntax for different engines when you get into more esoteric queries.
helwr 17y ago
congratulations, Slava & Michael
pg 17y ago
Incidentally, if any hackers are looking for jobs working on an interesting problem, I know the Rethinks are hiring. It would be the perfect job for a lot of hackers. You'd get to solve big problems starting with a blank slate, and you'd get to work with smart, totally pragmatic people (Slava and Mike). Plus they now have enough to actually pay you.
- quickpost 17y ago
  Not to mention they are very transparent with their compensation and stock options (Scroll down - http://rethinkdb.com/jobs/ http://rethinkdb.com/jobs/). People who run their hiring process and business like that seem like very good people to work for indeed.
  - wheaties 17y ago
    Not bad compensation for a start-up.
  - rms 17y ago
    That's really, really cool. I've never seen the stock options/equity listed on a hiring page before.
    - reitzensteinm 17y ago
      And I love that it's a percentage, not, eg, 10k options.
      - byrneseyeview 17y ago
        Do people actually quote the number of options without reference to a percentage or a stock price?
- cperciva 17y ago
  I would jump on this immediately if I was in the bay area and didn't already have my own startup. As it is, I'll have to satisfy my (data)base urges by writing my own high performance SSD-optimized key-value store.
  - ntoshev 17y ago
    Wow, are the metadata of Tarsnap that demanding?
    - cperciva 17y ago
      Tarsnap splits data into blocks of ~64 kB which are individually compressed and encrypted before being uploaded. I use S3 for back-end storage, but need to keep track of where I put each of the blocks. It's not that the metadata is demanding; it's just that there's a lot of it. For each $/month Tarsnap takes in, I have about 200,000 table entries.
andrewcooke 17y ago
i was looking at this the other day. how do they address (what i assume are) the larger space requirements of append-only databases with the lower capacities of ssds? am i wrong about the space requirements, or is moore's law going to fix it, or is there some kind of background compaction?
- coffeemug 17y ago
  A couple of points: - We garbage collect (see Mendel Rosenblum's Ph.D. thesis on log-structured systems) - Our customers care about cost per IOPS, not cost per GB. - The hot real-time stuff is usually handled by a different database and/or storage system than the older, less frequently accessed data anyway.
  - andrewcooke 17y ago
    thanks. if anyone else is curious, this seems to be the thesis - http://www.eecs.berkeley.edu/Pubs/TechRpts/1992/6267.html http://www.eecs.berkeley.edu/Pubs/TechRpts/1992/6267.html
  - runT1ME 17y ago
    Will you guys be able to support all isolation levels?
    - coffeemug 17y ago
      Isolation levels are really a poor design decision, because they imply the use of locks. Serializable is great, but impossible to implement efficiently. Repeatable read, read committed, and read uncommitted can be implemented efficiently, but allow for various unpleasant isolation artifacts. The one we're implementing is really the one everyone wants - snapshot isolation. It can be implemented very efficiently, and is stronger than repeatable read, read committed, and read uncommitted (so you should never want these three). It's not as strong as serializable, but nobody can give you a scalable serializable isolation level. Snapshot isolation also guarantees consistency, but requires all transactions to be idempotent (so they could be rerun in case of a conflict). It's the best of both worlds, in practice most other databases already behave this way anyway.
vladocar 17y ago
This is very interesting project, all the current DB are optimized for normal HD(and the standard HD is the slowest part of our PC). But with development of the Solid-State Drives we will have more and more fast drives. So the Database who will take advantage of the these new SSDrives will lead the way in Database design technology. It is right time to invest in this technology.
- jrockway 17y ago
  But really, rotating disks are not that bad for most database use cases. B-trees, the usual on-disk database structure, are designed to keep similar data on the same disk page, which means that if you request row 42, row 43 will be in memory by the time you need it. So the slowness of the disk is abstracted away; iterate over your data in index order, and it's always fast. Hash tables have a theoretical advantage over balanced trees, and an SSD would make a naive hash table implementation easier to implement. But if you are smart (like, say, BerkeleyDB), hash tables and balanced trees have almost the same real world performance. RethinkDB might be better for write-heavy operations, but that's because SSDs are better for random writes.
  - runT1ME 17y ago
    But the lack of locking is potentially big for multicore applications.
    - jrockway 17y ago
      So that data corruption can scale linearly? (Disks vs. SSDs and transactions vs. "eventual consistency" are orthogonal.)
      - coffeemug 17y ago
        Lock-free != data corruption. You just use atomic builtins[1] as primitives, instead of mutexes and semaphores. [1] http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins....
mlLK 17y ago
Congratulations guys, you deserve it, and thank you to anybody else out there writing drivers or optimizing software for changes happening in hardware that we all take for granted.
spicyj 17y ago
Does the iPhone really have an SSD?
- CrazedGeek 17y ago
  It uses flash memory, so yes. http://www.apple.com/iphone/specs.html http://www.apple.com/iphone/specs.html
ojbyrne 17y ago
It says "maintanence" on their technical details page. Hopefully they can fix that.
- mglukhovsky 17y ago
  Fixed, thank you for pointing it out.
ALee 17y ago
Big congrats. Smart people tackling tough problems get my vote of confidence.
known 17y ago
Can't we implement RethinkDB as features of MySQL or PostgreSQL

114 points by jasonlbaptiste 17y ago

"There’s obviously risk involved with trying to redefine how people structure their databases" TechCrunch misses the point that Rethink is explicitly not doing this. The MySQL engine is below the SQL parsing layer, so as-is MySQL apps should be able to run against it.

davidu 17y ago
That's in theory. In practice, MySQL has different syntax for different engines when you get into more esoteric queries.

helwr 17y ago

congratulations, Slava & Michael

pg 17y ago

Incidentally, if any hackers are looking for jobs working on an interesting problem, I know the Rethinks are hiring. It would be the perfect job for a lot of hackers. You'd get to solve big problems starting with a blank slate, and you'd get to work with smart, totally pragmatic people (Slava and Mike). Plus they now have enough to actually pay you.

quickpost 17y ago
Not to mention they are very transparent with their compensation and stock options (Scroll down - http://rethinkdb.com/jobs/ http://rethinkdb.com/jobs/). People who run their hiring process and business like that seem like very good people to work for indeed.
- wheaties 17y ago
  Not bad compensation for a start-up.
- rms 17y ago
  That's really, really cool. I've never seen the stock options/equity listed on a hiring page before.
  - reitzensteinm 17y ago
    And I love that it's a percentage, not, eg, 10k options.
    - byrneseyeview 17y ago
      Do people actually quote the number of options without reference to a percentage or a stock price?
cperciva 17y ago
I would jump on this immediately if I was in the bay area and didn't already have my own startup. As it is, I'll have to satisfy my (data)base urges by writing my own high performance SSD-optimized key-value store.
- ntoshev 17y ago
  Wow, are the metadata of Tarsnap that demanding?
  - cperciva 17y ago
    Tarsnap splits data into blocks of ~64 kB which are individually compressed and encrypted before being uploaded. I use S3 for back-end storage, but need to keep track of where I put each of the blocks. It's not that the metadata is demanding; it's just that there's a lot of it. For each $/month Tarsnap takes in, I have about 200,000 table entries.

andrewcooke 17y ago

i was looking at this the other day. how do they address (what i assume are) the larger space requirements of append-only databases with the lower capacities of ssds? am i wrong about the space requirements, or is moore's law going to fix it, or is there some kind of background compaction?

coffeemug 17y ago
A couple of points: - We garbage collect (see Mendel Rosenblum's Ph.D. thesis on log-structured systems) - Our customers care about cost per IOPS, not cost per GB. - The hot real-time stuff is usually handled by a different database and/or storage system than the older, less frequently accessed data anyway.
- andrewcooke 17y ago
  thanks. if anyone else is curious, this seems to be the thesis - http://www.eecs.berkeley.edu/Pubs/TechRpts/1992/6267.html http://www.eecs.berkeley.edu/Pubs/TechRpts/1992/6267.html
- runT1ME 17y ago
  Will you guys be able to support all isolation levels?
  - coffeemug 17y ago
    Isolation levels are really a poor design decision, because they imply the use of locks. Serializable is great, but impossible to implement efficiently. Repeatable read, read committed, and read uncommitted can be implemented efficiently, but allow for various unpleasant isolation artifacts. The one we're implementing is really the one everyone wants - snapshot isolation. It can be implemented very efficiently, and is stronger than repeatable read, read committed, and read uncommitted (so you should never want these three). It's not as strong as serializable, but nobody can give you a scalable serializable isolation level. Snapshot isolation also guarantees consistency, but requires all transactions to be idempotent (so they could be rerun in case of a conflict). It's the best of both worlds, in practice most other databases already behave this way anyway.

vladocar 17y ago

This is very interesting project, all the current DB are optimized for normal HD(and the standard HD is the slowest part of our PC). But with development of the Solid-State Drives we will have more and more fast drives. So the Database who will take advantage of the these new SSDrives will lead the way in Database design technology. It is right time to invest in this technology.

jrockway 17y ago
But really, rotating disks are not that bad for most database use cases. B-trees, the usual on-disk database structure, are designed to keep similar data on the same disk page, which means that if you request row 42, row 43 will be in memory by the time you need it. So the slowness of the disk is abstracted away; iterate over your data in index order, and it's always fast. Hash tables have a theoretical advantage over balanced trees, and an SSD would make a naive hash table implementation easier to implement. But if you are smart (like, say, BerkeleyDB), hash tables and balanced trees have almost the same real world performance. RethinkDB might be better for write-heavy operations, but that's because SSDs are better for random writes.
- runT1ME 17y ago
  But the lack of locking is potentially big for multicore applications.
  - jrockway 17y ago
    So that data corruption can scale linearly? (Disks vs. SSDs and transactions vs. "eventual consistency" are orthogonal.)
    - coffeemug 17y ago
      Lock-free != data corruption. You just use atomic builtins[1] as primitives, instead of mutexes and semaphores. [1] http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins....

mlLK 17y ago

Congratulations guys, you deserve it, and thank you to anybody else out there writing drivers or optimizing software for changes happening in hardware that we all take for granted.

spicyj 17y ago

Does the iPhone really have an SSD?

CrazedGeek 17y ago
It uses flash memory, so yes. http://www.apple.com/iphone/specs.html http://www.apple.com/iphone/specs.html

ojbyrne 17y ago

It says "maintanence" on their technical details page. Hopefully they can fix that.

mglukhovsky 17y ago
Fixed, thank you for pointing it out.

ALee 17y ago

Big congrats. Smart people tackling tough problems get my vote of confidence.

known 17y ago

Can't we implement RethinkDB as features of MySQL or PostgreSQL