7 ms·

RethinkDB Driver for Java now available

149 points by veritas3241 11y ago

sharms 11y ago
I recently heard about RethinkDB on a few podcasts (SE Radio and The Changelog) and it sounds very promising, and has been designed to solve real time web application use cases very well. Glad to see they now support Java so we can use it easier from JVM based languages
- hardwaresofton 11y ago
  I would also like to point out that it's a pretty good document store even without the super awesome realtime bits: This video is 2 years old (and rethinkdb has gotten way more awesome since then), but it comes with some super useful things out of the box, like a very nice Web UI, distributed system support, joins, and a great way to write complex queries (ReQL): https://www.youtube.com/watch?v=qKPKsBNw604 https://www.youtube.com/watch?v=qKPKsBNw604 If a document model fits your data, I'd suggest giving rethink a try.
jdoliner 11y ago
Awesome to see an official java driver for rethink. Any plans for a golang driver?
- krat0sprakhar 11y ago
  I've tried the (unofficial) rethinkdb client for Go and it's been working great for me. https://github.com/dancannon/gorethink https://github.com/dancannon/gorethink. Is there a particular reason why you prefer an official one over this?
- coffeemug 11y ago
  The community-supported goland driver by Dan Cannon is so good that it's not worth writing a new driver. Over time we plan to adopt it under the official RethinkDB umbrella -- I should be able to talk to Dan about a collaboration soon.
  - jdoliner 11y ago
    Yeah I don't think starting from scratch would make sense, bringing it under the rethink umbrella would be better. It has a couple of things that I think could be better, in particular it could leverage go's native json support more. I have a few other ideas on this... you know where to find me though.
    - _dancannon 11y ago
      Thanks for the feedback, I try to keep GoRethink up to date and to the same high standards as the official drivers. That being said I can understand why you would want an official driver. Regarding the json support the driver actually uses the native package however due to RethinkDBs use of psuedo-types an extra decoding step is needed and that is why GoRethink doesnt let you use json tags directly. Regarding your other ideas it would be great to discuss them.
gamesbrainiac 11y ago
This is interesting since a friend of mine was using a nodejs-based microservice layer to query Rethinkdb from his java application. I personally don't like java, but if you take a look at the API they've built, its actually pretty nice.
krat0sprakhar 11y ago
I recently used the Rethinkdb client for Clojure[0] and frankly I've found it a bit too unintuitive for use. You need to rely on a "query-builder" to write even simplistic queries. The documentation is also quite incomplete forcing you dive in the codebase to see how to use a feature. I should point out that I'm not taking digs at the library author - much props to him for writing & open-sourcing it. My question to the more experienced Clojure devs in the house - now that the official java client is out, should I use this from my Clojure app or keep using the other one? [0] - https://github.com/apa512/clj-rethinkdb https://github.com/apa512/clj-rethinkdb
- coffeemug 11y ago
  Slava @ RethinkDB here. I've never programmed in Clojure (although I spent years programming in Common Lisp), so everything below is what I overheard from the engineers that were looking into this. The official Java driver can be used from a variety of JVM languages and apparently results in a pretty good experience. Segphault is writing a blog post on this, so you'll see lots of examples soon of what the Java driver looks like in various JVM languages. Apparently the only exception to this is Clojure -- you can use the Java driver from Clojure, but idiomatically the experience isn't as good as using native Clojure libraries. We're going to play around with this and figure out what to do about it. One possibility is that someone in the community might write a Clojure wrapper on top of the official Java driver -- this would be way easier than writing a Clojure driver from scratch, and would provide idiomatically appropriate interfaces. I know Aphyr is using clj-rethinkdb for his Jepsen analysis of RethinkDB, and it's working out pretty well, and I know of a few production deployments based on clj-rethinkdb. So another possibility is to invest a little more time/effort into clj-rethinkdb to bring it up to par with official drivers. We should be able to improve the state of affairs in the next few months.
  - krat0sprakhar 11y ago
    Hi Slava! Thanks for the answer. I'm aware of Aphyr's work with Rethinkdb, that's how I found about clj-rethinkdb in the first place. Aphyr is a seasoned clojure programmer, so I'm sure using any library wouldn't be too much of work him. I'm still new to Clojure and compared with the Go/Javascript version of the client, I found it a bit hard to understand (the lack of documentation also doesn't help). There's a high possibility that the reason behind experience is due to the shortcomings in my understanding of clj and not the client itself. In any case, I look forward to seeing what rethinkdb / community does with improving the client. PS: You'd be happy to know that I jumped on the Clojure ship after reading your blogpost[0] about lisp. So thanks for that :) [0] - http://www.defmacro.org/ramblings/lisp.html http://www.defmacro.org/ramblings/lisp.html
tequila_shot 11y ago
Django driver please (to work with ORM)? :-/ Edit : I am not sure why I am being down voted. Help?
- meat_fist 11y ago
  Forgive my ignorance, but can you not just use the Python RethinkDB driver? Or does it have to be Django specific? https://rethinkdb.com/docs/guide/python/ https://rethinkdb.com/docs/guide/python/
  - SuperKlaus 11y ago
    I think tequila_shot is looking for a driver that lets him use RethinkDB with the Django ORM.
    - tequila_shot 11y ago
      You are right. I am looking for a driver to work with ORM.
  - habitue 11y ago
    Django is pretty tightly coupled with its orm. The last time I looked into it, the mongo/django solution was essentially a fork of django with the data layer completely rewritten
- vvpan 11y ago
  I wouldn't hold my breath. Django isn't popular enough to create a niche driver. Besides as others point out vanilla Django was designed with SQL in mind.
  - tequila_shot 11y ago
    > Besides as others point out vanilla Django was designed with SQL in mind. I am not trying to challenge your statement and not trying to be sarcastic. Can you quote the link where this is stated so that I can learn the background?
PedroBatista 11y ago
Finally! ( congrats :) ) When we decided to use RethinkDB I had to dig up old driver source code with 50-50 compatible protocol with newer versions of ReDB, sometimes if I wrote too much data in bulk it would simply crash the JVM lol. Now I can sleep better knowing my changes of getting fired are 2% less. Now regarding the awfulness of JSON objects to/from Java Objects, any good news? (I know it's mostly a Java the lang issue)
- coffeemug 11y ago
  The official driver has a few helper methods to create JSON objects, so this hopefully shouldn't be too painful until the language adds native syntax for JSON.
  - mring33621 11y ago
    Wouldn't Nashorn in Java 8 count as 'native syntax'? Use javascript in the JVM to munge your JSON.
- RyanZAG 11y ago
  Do you mean converting from JSON to POJO? Why do you consider that awful? There are very good libraries (jackson, gson) for handling that and they work really well - especially if you're going to be using those POJOs for HTTP REST too.
  - VeilEm 11y ago
    Are you forced to use JSON with RethinkDB? Like is the mechanism RethinkDB -> JSON String -> to POJO and POJO -> JSON string -> RethinkDB with each database interaction? That seems crazy inefficient.
    - RyanZAG 11y ago
      Isn't that kind of like arguing how serializing an HTTP reply into text for the browser crazy inefficient? At any rate, JSON is the transfer format used by RethinkDB so at some point it has to be JSON for RethinkDB to read it. You could always make your own data language and integrate it all yourself, but ultimately you can't transfer your POJO directly and will always have to transform it into something. JSON is pretty decent and very compatible. So to answer your question: (RethinkDB -> JSON String) is a completely required part of the database communication. The additional POJO step is fairly efficient when using Jackson and unlikely to be a large part of your CPU usage - the simple and required byte copy may use more CPU on large POJOs. You could use some type of no-copy directly from the network buffer but then a complex database like RethinkDB would not be the right choice - you'd probably use local memmapped files in C.. In short: try using the driver and pojo conversion in a real world project and profile it. I think you'd be surprised how little overhead it really is. Computers are very good at this stuff, and the JVM's JIT is optimized for this stuff.
RyanZAG 11y ago
Looks great, and the vert.x bit was also great to see. The blockingHandler a bit less so. Any plans for making a non blocking API using netty channels? That would really put RethinkDB as an amazing choice for pairing vert.x/JVM with a data layer.
- danielmewes 11y ago
  Yes! We wanted to get the synchronous driver out first to get feedback. But a non-blocking mode is definitely coming up. https://github.com/rethinkdb/rethinkdb/issues/4802 https://github.com/rethinkdb/rethinkdb/issues/4802
jongwook 11y ago
It seems that we can make an API call nonblocking but we'll have to block and wait for such call to complete. How can they mention reactive/event-driven in the middle of the blog and not provide an asynchronous API? Java 8 CompletableFuture would've been the perfect abstraction for it.
- danielmewes 11y ago
  Daniel @ RethinkDB here. We're going to add an asynchronous API soon. We decided to get the synchronous driver out to get some early feedback and because we know that a lot of our users have been waiting for official Java support, and not all of them need an async API. You might be interested in following this GitHub issue: https://github.com/rethinkdb/rethinkdb/issues/4802 https://github.com/rethinkdb/rethinkdb/issues/4802
kevindeasis 11y ago
You know what would make Rethink more badass. If there was a cheap RethinkDB As A Service provider.
- sureshv 11y ago
  Not affiliated but https://compose.io https://compose.io might do what you want.
  - coffeemug 11y ago
    IMO $22.50/month for an autoscaled, managed database deployment with full backup, upgrades, security, etc. is a bargain. Compose is pretty great!
    - kevindeasis 11y ago
      It would be amazing if they had a sandbox or freemium model. It's pretty expensive to spend $22.50 bucks a month for project.
foo3456 11y ago
What does Java _driver_ mean -> is this a JDBC driver or just some non standard API to access this database system?
- habitue 11y ago
  This isn't a JDBC driver since RethinkDB isn't a SQL database, but it fulfills the same role as a JDBC driver. In addition, RethinkDB doesn't have a textual query language, it's structured data, so the bare driver looks a lot like an ORM
moatra 11y ago
Just in time! This will be nice reference. I recently started working on a new Scala driver that uses the v0_4 asynchronous protocol, built on top of Akka's IO module and Play's JSON module. I think I have the performance where I want it, but now I need to flesh out the DSL for proper ReQL support.
- habitue 11y ago
  Have a look at https://github.com/rethinkdb/rethinkdb/blob/next/drivers/java/term_info.json https://github.com/rethinkdb/rethinkdb/blob/next/drivers/jav... It has a full listing of all ReQL terms and their signatures and optargs
merb 11y ago
What's missing would be a Async Interface. Especially with Java8 and CompletableFuture, which will be interoping in Scala 2.12

149 points by veritas3241 11y ago

I recently heard about RethinkDB on a few podcasts (SE Radio and The Changelog) and it sounds very promising, and has been designed to solve real time web application use cases very well. Glad to see they now support Java so we can use it easier from JVM based languages

hardwaresofton 11y ago
I would also like to point out that it's a pretty good document store even without the super awesome realtime bits: This video is 2 years old (and rethinkdb has gotten way more awesome since then), but it comes with some super useful things out of the box, like a very nice Web UI, distributed system support, joins, and a great way to write complex queries (ReQL): https://www.youtube.com/watch?v=qKPKsBNw604 https://www.youtube.com/watch?v=qKPKsBNw604 If a document model fits your data, I'd suggest giving rethink a try.

jdoliner 11y ago

Awesome to see an official java driver for rethink. Any plans for a golang driver?

krat0sprakhar 11y ago
I've tried the (unofficial) rethinkdb client for Go and it's been working great for me. https://github.com/dancannon/gorethink https://github.com/dancannon/gorethink. Is there a particular reason why you prefer an official one over this?
coffeemug 11y ago
The community-supported goland driver by Dan Cannon is so good that it's not worth writing a new driver. Over time we plan to adopt it under the official RethinkDB umbrella -- I should be able to talk to Dan about a collaboration soon.
- jdoliner 11y ago
  Yeah I don't think starting from scratch would make sense, bringing it under the rethink umbrella would be better. It has a couple of things that I think could be better, in particular it could leverage go's native json support more. I have a few other ideas on this... you know where to find me though.
  - _dancannon 11y ago
    Thanks for the feedback, I try to keep GoRethink up to date and to the same high standards as the official drivers. That being said I can understand why you would want an official driver. Regarding the json support the driver actually uses the native package however due to RethinkDBs use of psuedo-types an extra decoding step is needed and that is why GoRethink doesnt let you use json tags directly. Regarding your other ideas it would be great to discuss them.

gamesbrainiac 11y ago

This is interesting since a friend of mine was using a nodejs-based microservice layer to query Rethinkdb from his java application. I personally don't like java, but if you take a look at the API they've built, its actually pretty nice.

krat0sprakhar 11y ago

I recently used the Rethinkdb client for Clojure[0] and frankly I've found it a bit too unintuitive for use. You need to rely on a "query-builder" to write even simplistic queries. The documentation is also quite incomplete forcing you dive in the codebase to see how to use a feature. I should point out that I'm not taking digs at the library author - much props to him for writing & open-sourcing it. My question to the more experienced Clojure devs in the house - now that the official java client is out, should I use this from my Clojure app or keep using the other one? [0] - https://github.com/apa512/clj-rethinkdb https://github.com/apa512/clj-rethinkdb

coffeemug 11y ago
Slava @ RethinkDB here. I've never programmed in Clojure (although I spent years programming in Common Lisp), so everything below is what I overheard from the engineers that were looking into this. The official Java driver can be used from a variety of JVM languages and apparently results in a pretty good experience. Segphault is writing a blog post on this, so you'll see lots of examples soon of what the Java driver looks like in various JVM languages. Apparently the only exception to this is Clojure -- you can use the Java driver from Clojure, but idiomatically the experience isn't as good as using native Clojure libraries. We're going to play around with this and figure out what to do about it. One possibility is that someone in the community might write a Clojure wrapper on top of the official Java driver -- this would be way easier than writing a Clojure driver from scratch, and would provide idiomatically appropriate interfaces. I know Aphyr is using clj-rethinkdb for his Jepsen analysis of RethinkDB, and it's working out pretty well, and I know of a few production deployments based on clj-rethinkdb. So another possibility is to invest a little more time/effort into clj-rethinkdb to bring it up to par with official drivers. We should be able to improve the state of affairs in the next few months.
- krat0sprakhar 11y ago
  Hi Slava! Thanks for the answer. I'm aware of Aphyr's work with Rethinkdb, that's how I found about clj-rethinkdb in the first place. Aphyr is a seasoned clojure programmer, so I'm sure using any library wouldn't be too much of work him. I'm still new to Clojure and compared with the Go/Javascript version of the client, I found it a bit hard to understand (the lack of documentation also doesn't help). There's a high possibility that the reason behind experience is due to the shortcomings in my understanding of clj and not the client itself. In any case, I look forward to seeing what rethinkdb / community does with improving the client. PS: You'd be happy to know that I jumped on the Clojure ship after reading your blogpost[0] about lisp. So thanks for that :) [0] - http://www.defmacro.org/ramblings/lisp.html http://www.defmacro.org/ramblings/lisp.html

tequila_shot 11y ago

Django driver please (to work with ORM)? :-/ Edit : I am not sure why I am being down voted. Help?

meat_fist 11y ago
Forgive my ignorance, but can you not just use the Python RethinkDB driver? Or does it have to be Django specific? https://rethinkdb.com/docs/guide/python/ https://rethinkdb.com/docs/guide/python/
- SuperKlaus 11y ago
  I think tequila_shot is looking for a driver that lets him use RethinkDB with the Django ORM.
  - tequila_shot 11y ago
    You are right. I am looking for a driver to work with ORM.
- habitue 11y ago
  Django is pretty tightly coupled with its orm. The last time I looked into it, the mongo/django solution was essentially a fork of django with the data layer completely rewritten
vvpan 11y ago
I wouldn't hold my breath. Django isn't popular enough to create a niche driver. Besides as others point out vanilla Django was designed with SQL in mind.
- tequila_shot 11y ago
  > Besides as others point out vanilla Django was designed with SQL in mind. I am not trying to challenge your statement and not trying to be sarcastic. Can you quote the link where this is stated so that I can learn the background?

PedroBatista 11y ago

Finally! ( congrats :) ) When we decided to use RethinkDB I had to dig up old driver source code with 50-50 compatible protocol with newer versions of ReDB, sometimes if I wrote too much data in bulk it would simply crash the JVM lol. Now I can sleep better knowing my changes of getting fired are 2% less. Now regarding the awfulness of JSON objects to/from Java Objects, any good news? (I know it's mostly a Java the lang issue)

coffeemug 11y ago
The official driver has a few helper methods to create JSON objects, so this hopefully shouldn't be too painful until the language adds native syntax for JSON.
- mring33621 11y ago
  Wouldn't Nashorn in Java 8 count as 'native syntax'? Use javascript in the JVM to munge your JSON.
RyanZAG 11y ago
Do you mean converting from JSON to POJO? Why do you consider that awful? There are very good libraries (jackson, gson) for handling that and they work really well - especially if you're going to be using those POJOs for HTTP REST too.
- VeilEm 11y ago
  Are you forced to use JSON with RethinkDB? Like is the mechanism RethinkDB -> JSON String -> to POJO and POJO -> JSON string -> RethinkDB with each database interaction? That seems crazy inefficient.
  - RyanZAG 11y ago
    Isn't that kind of like arguing how serializing an HTTP reply into text for the browser crazy inefficient? At any rate, JSON is the transfer format used by RethinkDB so at some point it has to be JSON for RethinkDB to read it. You could always make your own data language and integrate it all yourself, but ultimately you can't transfer your POJO directly and will always have to transform it into something. JSON is pretty decent and very compatible. So to answer your question: (RethinkDB -> JSON String) is a completely required part of the database communication. The additional POJO step is fairly efficient when using Jackson and unlikely to be a large part of your CPU usage - the simple and required byte copy may use more CPU on large POJOs. You could use some type of no-copy directly from the network buffer but then a complex database like RethinkDB would not be the right choice - you'd probably use local memmapped files in C.. In short: try using the driver and pojo conversion in a real world project and profile it. I think you'd be surprised how little overhead it really is. Computers are very good at this stuff, and the JVM's JIT is optimized for this stuff.

RyanZAG 11y ago

Looks great, and the vert.x bit was also great to see. The blockingHandler a bit less so. Any plans for making a non blocking API using netty channels? That would really put RethinkDB as an amazing choice for pairing vert.x/JVM with a data layer.

danielmewes 11y ago
Yes! We wanted to get the synchronous driver out first to get feedback. But a non-blocking mode is definitely coming up. https://github.com/rethinkdb/rethinkdb/issues/4802 https://github.com/rethinkdb/rethinkdb/issues/4802

jongwook 11y ago

It seems that we can make an API call nonblocking but we'll have to block and wait for such call to complete. How can they mention reactive/event-driven in the middle of the blog and not provide an asynchronous API? Java 8 CompletableFuture would've been the perfect abstraction for it.

danielmewes 11y ago
Daniel @ RethinkDB here. We're going to add an asynchronous API soon. We decided to get the synchronous driver out to get some early feedback and because we know that a lot of our users have been waiting for official Java support, and not all of them need an async API. You might be interested in following this GitHub issue: https://github.com/rethinkdb/rethinkdb/issues/4802 https://github.com/rethinkdb/rethinkdb/issues/4802

kevindeasis 11y ago

You know what would make Rethink more badass. If there was a cheap RethinkDB As A Service provider.

sureshv 11y ago
Not affiliated but https://compose.io https://compose.io might do what you want.
- coffeemug 11y ago
  IMO $22.50/month for an autoscaled, managed database deployment with full backup, upgrades, security, etc. is a bargain. Compose is pretty great!
  - kevindeasis 11y ago
    It would be amazing if they had a sandbox or freemium model. It's pretty expensive to spend $22.50 bucks a month for project.

foo3456 11y ago

What does Java _driver_ mean -> is this a JDBC driver or just some non standard API to access this database system?

habitue 11y ago
This isn't a JDBC driver since RethinkDB isn't a SQL database, but it fulfills the same role as a JDBC driver. In addition, RethinkDB doesn't have a textual query language, it's structured data, so the bare driver looks a lot like an ORM

moatra 11y ago

Just in time! This will be nice reference. I recently started working on a new Scala driver that uses the v0_4 asynchronous protocol, built on top of Akka's IO module and Play's JSON module. I think I have the performance where I want it, but now I need to flesh out the DSL for proper ReQL support.

habitue 11y ago
Have a look at https://github.com/rethinkdb/rethinkdb/blob/next/drivers/java/term_info.json https://github.com/rethinkdb/rethinkdb/blob/next/drivers/jav... It has a full listing of all ReQL terms and their signatures and optargs

merb 11y ago

What's missing would be a Async Interface. Especially with Java8 and CompletableFuture, which will be interoping in Scala 2.12