6 ms·

I recently used the Rethinkdb client for Clojure[0] and frankly I've found it a bit too unintuitive for use. You need to rely on a "query-builder" to write even

I recently used the Rethinkdb client for Clojure[0] and frankly I've found it a bit too unintuitive for use. You need to rely on a "query-builder" to write even simplistic queries. The documentation is also quite incomplete forcing you dive in the codebase to see how to use a feature. I should point out that I'm not taking digs at the library author - much props to him for writing & open-sourcing it. My question to the more experienced Clojure devs in the house - now that the official java client is out, should I use this from my Clojure app or keep using the other one? [0] - https://github.com/apa512/clj-rethinkdb https://github.com/apa512/clj-rethinkdb

coffeemug 11y ago
Slava @ RethinkDB here. I've never programmed in Clojure (although I spent years programming in Common Lisp), so everything below is what I overheard from the engineers that were looking into this. The official Java driver can be used from a variety of JVM languages and apparently results in a pretty good experience. Segphault is writing a blog post on this, so you'll see lots of examples soon of what the Java driver looks like in various JVM languages. Apparently the only exception to this is Clojure -- you can use the Java driver from Clojure, but idiomatically the experience isn't as good as using native Clojure libraries. We're going to play around with this and figure out what to do about it. One possibility is that someone in the community might write a Clojure wrapper on top of the official Java driver -- this would be way easier than writing a Clojure driver from scratch, and would provide idiomatically appropriate interfaces. I know Aphyr is using clj-rethinkdb for his Jepsen analysis of RethinkDB, and it's working out pretty well, and I know of a few production deployments based on clj-rethinkdb. So another possibility is to invest a little more time/effort into clj-rethinkdb to bring it up to par with official drivers. We should be able to improve the state of affairs in the next few months.
- krat0sprakhar 11y ago
  Hi Slava! Thanks for the answer. I'm aware of Aphyr's work with Rethinkdb, that's how I found about clj-rethinkdb in the first place. Aphyr is a seasoned clojure programmer, so I'm sure using any library wouldn't be too much of work him. I'm still new to Clojure and compared with the Go/Javascript version of the client, I found it a bit hard to understand (the lack of documentation also doesn't help). There's a high possibility that the reason behind experience is due to the shortcomings in my understanding of clj and not the client itself. In any case, I look forward to seeing what rethinkdb / community does with improving the client. PS: You'd be happy to know that I jumped on the Clojure ship after reading your blogpost[0] about lisp. So thanks for that :) [0] - http://www.defmacro.org/ramblings/lisp.html http://www.defmacro.org/ramblings/lisp.html
  - coffeemug 11y ago
    Cool! I think you've identified the problem correctly (using the clj driver is harder for a beginner than some of the other drivers). The solution might involve better docs, more/better code, or some combination of the two. We'll try to get on this as soon as we can!
- dantiberian 11y ago
  I'm a maintainer of clj-rethinkdb. It makes a ton of sense for clj-rethinkdb to use as much as possible from the Java driver. I might be a bit biased, but I think the query language is really idiomatic for Clojure. The docs could definitely do with some love, I'd appreciate feedback on what else we could do to bring it up to par with the official drivers.
  - coffeemug 11y ago
    Slava @ RethinkDB here. Thanks for all your awesome work on the driver!
hardwaresofton 11y ago
I actually didn't have the same problems with the clojure rethinkdb driver. I'll address my opinions in turn: - query builder reliance I'm not sure what you mean by this? I got a lot of mileage out of the library by writing partial functions that generate queries that were most commonly used. So, I'd have functions like: (def find-user-by-id "Find a user by ID" [id] (-> (r/db "database") (r/table "users") (r/find id))) or something similar. Of course, you can get a lot more generic with this -- but I found that keeping these queries with each other created good separation of "database-stuff" from the rest of my app (and made it easy for me to do versioning. - Needing to look in the codebase for documentation on how to use the methods While this is certainly unfortunate (and I belive true), I have found that it's actually good advice to tell someone to look at tests in a repo as backup-documentation on projects that are far along enough to have tests. Also... this is kind of a problem with the clojure ecosystem in general, so I'd definitely +1. Would like to take this moment to also thank apa512 for writing the library, thanks for doing so! I think I ran into a problem once when working on changefeeds (the default implementation used a seq and I think I wanted a channel or something like that), but that boiled down to me misunderstanding how to use it, rather than it being "wrong" in a sense.
- krat0sprakhar 11y ago
  Queries like the one you wrote above are indeed quite idiomatic but something as simple as providing a `read_mode` requires a query builder- (r/get (rethinkdb.query-builder/term :TABLE [(query/db db) table] {:read_mode read_mode}) id)])) whereas in JS, you can simply do r.db('heroes').table('marvel', {readMode: 'outdated'}).run(conn, callback)
  - hardwaresofton 11y ago
    Thanks for the example -- while I have never needed to actually change the read mode, you shouldn't need to actually use that query buider, read mode is an optional argument to r/table. Apologize if I misunderstood what you were trying to do, or if this was a relatively recent addition that wasn't available last you used clj-rethinkdb. Here's an example (from tests in the repo): https://github.com/apa512/clj-rethinkdb/blob/1cd09c7c8a4ce271a6cd35fdaa023c823b3465e1/test/rethinkdb/core_test.clj#L122 https://github.com/apa512/clj-rethinkdb/blob/1cd09c7c8a4ce27...
dantiberian 11y ago
Hey, I'm one of the maintainers for clj-rethinkdb. There's a few things I can discuss here. > You need to rely on a "query-builder" One of the beautiful things about RethinkDB is it's query language. It is basically S-expressions but in vectors instead of lists. This pairs really well with Clojure as Clojure is made up of S-expressions too. All of the drivers do use some kind of a query builder internally (as far as I'm aware) to build up the query data structure, then they call run on it. This process is just a bit more explicit in clj-rethinkdb, and has all kinds of benefits, like being able to make partial functions, and compose ReQL functions really nicely. > The documentation is also quite incomplete clj-rethinkdb follows the official JavaScript API quite closely. There were two options: create minimal docstrings in Clojure and leave the main documentation up to RethinkDB, or copy all of the documentation into clj-rethinkdb. There are a few issues with the second approach, the most obvious is how to efficiently copy it, but also that different versions of the RethinkDB server support different functions and options, so we'd need to keep compatibility notes, e.t.c. A really nice approach is in https://github.com/rethinkdb/docs/issues/710 https://github.com/rethinkdb/docs/issues/710, and https://github.com/rethinkdb/docs/issues/803 https://github.com/rethinkdb/docs/issues/803 but that requires more work. All of that to say, at the moment the best option is to read the JavaScript docs first, then look at the Clojure docstrings for argument order or special notes. > now that the official java client is out, should I use this from my Clojure app or keep using the other one? It makes a lot of sense for clj-rethinkdb to take use of as much of the Java driver as possible. I haven't got any news yet, but it's likely we'll be at least poaching the connection and request/response handling. I'd be happy to jump on a Hangout or text chat in https://gitter.im/apa512/clj-rethinkdb https://gitter.im/apa512/clj-rethinkdb to help you get over any speed bumps using the clj-rethinkdb driver. Hearing a first time users experience would be really good.
segphault 11y ago
Hey, I'm Ryan from RethinkDB. I wrote the OP blog post and I'm working on a follow-up that demonstrates how to use the Java driver in various alt-JVM languages. You can take advantage of Clojure's Java interop to use the new official Java driver in Clojure, but it doesn't offer a very good user experience compared to clj-rethinkdb. It will work for a lot of simple cases, but it gets weird very quickly. When you define a function in Clojure, the underlying Java object is an IFn[0]. Unfortunately, you can't pass an IFn to native Java methods that expect to receive a Java 8 lambda (an anonymous class with a single method). You can work around this limitation by manually creating an object that conforms with the client driver's various ReqlFunction[1] interfaces, but it's not very pretty: This ReQL expression in Java: r.table("fellowship").map(x -> x.getField("name")).coerceTo("array").run(conn) Becomes this in Clojure: (-> (.table r "fellowship") (.map (reify com.rethinkdb.gen.ast.ReqlFunction1 (apply [this, x] (.getField x "name")))) (.coerceTo "array") (.run conn)) Note the use of reify and the ReqlFunction1 interface. You need to be careful to pick the interface that matches the arity of your anonymous function. For example, if you have four parameters, you use ReqlFunction4 instead of ReqlFunction1. You could probably do some voodoo to abstract away some of the ugliness, but the way that clj-rethinkdb's r/fn macro works under the hood and translates into the wire protocol is frankly a lot more elegant than anything you could do with our official client driver and Clojure's Java interop IMO. That said, I'm not particularly well-versed in either Java or Clojure, so there may be something I'm overlooking. I hope this explanation is helpful to somebody. :-) TL;DR: use clj-rethinkdb if you want to write sane-looking ReQL in Clojure. [0]: https://clojure.github.io/clojure/javadoc/clojure/lang/IFn.html https://clojure.github.io/clojure/javadoc/clojure/lang/IFn.h... [1]: https://github.com/rethinkdb/rethinkdb/blob/next/drivers/java/src/main/java/com/rethinkdb/gen/ast/ReqlFunction1.java https://github.com/rethinkdb/rethinkdb/blob/next/drivers/jav...

coffeemug 11y ago

Slava @ RethinkDB here. I've never programmed in Clojure (although I spent years programming in Common Lisp), so everything below is what I overheard from the engineers that were looking into this. The official Java driver can be used from a variety of JVM languages and apparently results in a pretty good experience. Segphault is writing a blog post on this, so you'll see lots of examples soon of what the Java driver looks like in various JVM languages. Apparently the only exception to this is Clojure -- you can use the Java driver from Clojure, but idiomatically the experience isn't as good as using native Clojure libraries. We're going to play around with this and figure out what to do about it. One possibility is that someone in the community might write a Clojure wrapper on top of the official Java driver -- this would be way easier than writing a Clojure driver from scratch, and would provide idiomatically appropriate interfaces. I know Aphyr is using clj-rethinkdb for his Jepsen analysis of RethinkDB, and it's working out pretty well, and I know of a few production deployments based on clj-rethinkdb. So another possibility is to invest a little more time/effort into clj-rethinkdb to bring it up to par with official drivers. We should be able to improve the state of affairs in the next few months.

krat0sprakhar 11y ago
Hi Slava! Thanks for the answer. I'm aware of Aphyr's work with Rethinkdb, that's how I found about clj-rethinkdb in the first place. Aphyr is a seasoned clojure programmer, so I'm sure using any library wouldn't be too much of work him. I'm still new to Clojure and compared with the Go/Javascript version of the client, I found it a bit hard to understand (the lack of documentation also doesn't help). There's a high possibility that the reason behind experience is due to the shortcomings in my understanding of clj and not the client itself. In any case, I look forward to seeing what rethinkdb / community does with improving the client. PS: You'd be happy to know that I jumped on the Clojure ship after reading your blogpost[0] about lisp. So thanks for that :) [0] - http://www.defmacro.org/ramblings/lisp.html http://www.defmacro.org/ramblings/lisp.html
- coffeemug 11y ago
  Cool! I think you've identified the problem correctly (using the clj driver is harder for a beginner than some of the other drivers). The solution might involve better docs, more/better code, or some combination of the two. We'll try to get on this as soon as we can!
dantiberian 11y ago
I'm a maintainer of clj-rethinkdb. It makes a ton of sense for clj-rethinkdb to use as much as possible from the Java driver. I might be a bit biased, but I think the query language is really idiomatic for Clojure. The docs could definitely do with some love, I'd appreciate feedback on what else we could do to bring it up to par with the official drivers.
- coffeemug 11y ago
  Slava @ RethinkDB here. Thanks for all your awesome work on the driver!

hardwaresofton 11y ago

I actually didn't have the same problems with the clojure rethinkdb driver. I'll address my opinions in turn: - query builder reliance I'm not sure what you mean by this? I got a lot of mileage out of the library by writing partial functions that generate queries that were most commonly used. So, I'd have functions like: (def find-user-by-id "Find a user by ID" [id] (-> (r/db "database") (r/table "users") (r/find id))) or something similar. Of course, you can get a lot more generic with this -- but I found that keeping these queries with each other created good separation of "database-stuff" from the rest of my app (and made it easy for me to do versioning. - Needing to look in the codebase for documentation on how to use the methods While this is certainly unfortunate (and I belive true), I have found that it's actually good advice to tell someone to look at tests in a repo as backup-documentation on projects that are far along enough to have tests. Also... this is kind of a problem with the clojure ecosystem in general, so I'd definitely +1. Would like to take this moment to also thank apa512 for writing the library, thanks for doing so! I think I ran into a problem once when working on changefeeds (the default implementation used a seq and I think I wanted a channel or something like that), but that boiled down to me misunderstanding how to use it, rather than it being "wrong" in a sense.

krat0sprakhar 11y ago
Queries like the one you wrote above are indeed quite idiomatic but something as simple as providing a `read_mode` requires a query builder- (r/get (rethinkdb.query-builder/term :TABLE [(query/db db) table] {:read_mode read_mode}) id)])) whereas in JS, you can simply do r.db('heroes').table('marvel', {readMode: 'outdated'}).run(conn, callback)
- hardwaresofton 11y ago
  Thanks for the example -- while I have never needed to actually change the read mode, you shouldn't need to actually use that query buider, read mode is an optional argument to r/table. Apologize if I misunderstood what you were trying to do, or if this was a relatively recent addition that wasn't available last you used clj-rethinkdb. Here's an example (from tests in the repo): https://github.com/apa512/clj-rethinkdb/blob/1cd09c7c8a4ce271a6cd35fdaa023c823b3465e1/test/rethinkdb/core_test.clj#L122 https://github.com/apa512/clj-rethinkdb/blob/1cd09c7c8a4ce27...

dantiberian 11y ago

Hey, I'm one of the maintainers for clj-rethinkdb. There's a few things I can discuss here. > You need to rely on a "query-builder" One of the beautiful things about RethinkDB is it's query language. It is basically S-expressions but in vectors instead of lists. This pairs really well with Clojure as Clojure is made up of S-expressions too. All of the drivers do use some kind of a query builder internally (as far as I'm aware) to build up the query data structure, then they call run on it. This process is just a bit more explicit in clj-rethinkdb, and has all kinds of benefits, like being able to make partial functions, and compose ReQL functions really nicely. > The documentation is also quite incomplete clj-rethinkdb follows the official JavaScript API quite closely. There were two options: create minimal docstrings in Clojure and leave the main documentation up to RethinkDB, or copy all of the documentation into clj-rethinkdb. There are a few issues with the second approach, the most obvious is how to efficiently copy it, but also that different versions of the RethinkDB server support different functions and options, so we'd need to keep compatibility notes, e.t.c. A really nice approach is in https://github.com/rethinkdb/docs/issues/710 https://github.com/rethinkdb/docs/issues/710, and https://github.com/rethinkdb/docs/issues/803 https://github.com/rethinkdb/docs/issues/803 but that requires more work. All of that to say, at the moment the best option is to read the JavaScript docs first, then look at the Clojure docstrings for argument order or special notes. > now that the official java client is out, should I use this from my Clojure app or keep using the other one? It makes a lot of sense for clj-rethinkdb to take use of as much of the Java driver as possible. I haven't got any news yet, but it's likely we'll be at least poaching the connection and request/response handling. I'd be happy to jump on a Hangout or text chat in https://gitter.im/apa512/clj-rethinkdb https://gitter.im/apa512/clj-rethinkdb to help you get over any speed bumps using the clj-rethinkdb driver. Hearing a first time users experience would be really good.

segphault 11y ago

Hey, I'm Ryan from RethinkDB. I wrote the OP blog post and I'm working on a follow-up that demonstrates how to use the Java driver in various alt-JVM languages. You can take advantage of Clojure's Java interop to use the new official Java driver in Clojure, but it doesn't offer a very good user experience compared to clj-rethinkdb. It will work for a lot of simple cases, but it gets weird very quickly. When you define a function in Clojure, the underlying Java object is an IFn[0]. Unfortunately, you can't pass an IFn to native Java methods that expect to receive a Java 8 lambda (an anonymous class with a single method). You can work around this limitation by manually creating an object that conforms with the client driver's various ReqlFunction[1] interfaces, but it's not very pretty: This ReQL expression in Java: r.table("fellowship").map(x -> x.getField("name")).coerceTo("array").run(conn) Becomes this in Clojure: (-> (.table r "fellowship") (.map (reify com.rethinkdb.gen.ast.ReqlFunction1 (apply [this, x] (.getField x "name")))) (.coerceTo "array") (.run conn)) Note the use of reify and the ReqlFunction1 interface. You need to be careful to pick the interface that matches the arity of your anonymous function. For example, if you have four parameters, you use ReqlFunction4 instead of ReqlFunction1. You could probably do some voodoo to abstract away some of the ugliness, but the way that clj-rethinkdb's r/fn macro works under the hood and translates into the wire protocol is frankly a lot more elegant than anything you could do with our official client driver and Clojure's Java interop IMO. That said, I'm not particularly well-versed in either Java or Clojure, so there may be something I'm overlooking. I hope this explanation is helpful to somebody. :-) TL;DR: use clj-rethinkdb if you want to write sane-looking ReQL in Clojure. [0]: https://clojure.github.io/clojure/javadoc/clojure/lang/IFn.html https://clojure.github.io/clojure/javadoc/clojure/lang/IFn.h... [1]: https://github.com/rethinkdb/rethinkdb/blob/next/drivers/java/src/main/java/com/rethinkdb/gen/ast/ReqlFunction1.java https://github.com/rethinkdb/rethinkdb/blob/next/drivers/jav...