5 ms·

DataScript – Datomic in ClojureScript

143 points by sgrove 12y ago

sdegutis 12y ago
Looks great. I could see using this for some data crunching client-side, maybe even in conjunction with something like graph.js.
drcode 12y ago
Looks like a great library... It would be intriguing to try to use something like DatomicScript for the app state of an Om application. (Edit: Read in a separate blog post that the author of DatomicScript is thinking about this already) At the end of the day though, I still think someone (smarter than me) needs to figure out the full end-to-end solution for datomic+clojurescript, with both client and server elements. The core question: Does it make sense to map datomic to a REST API on the server, then consume the REST on the client? Or, does it make more sense to build something more clojure-like for the client-server interface, and remove REST from the equation entirely?
- tony_landis 12y ago
  A seamless datomic <-> clojurescript client/service framework is needed. I am just getting started with Om and hope to experiment with some ideas in that space. I don't think REST is the right approach. I would like to abstract everything to an AVET or sequence thereof.
- vdm 12y ago
  https://twitter.com/richhickey/status/355394063165440000 https://twitter.com/richhickey/status/355394063165440000
  - drcode 12y ago
    Interesting, I get the feeling though that RH also thinks this isn't the right approach, that some more though needs to be put into the client-server interface still that goes beyond vanilla datomic.
    - tonsky 12y ago
      Thing is, server-side Datomic peers work in trusted env; every peer sees everything. As you start doing something like a peer in every browser that opens your app, you need to provide it only with the part of the database. This filtering should be done two-way (what’s allowed to read, what’s allowed to push back) and is not very straightforward thing to do. It’s not even clear how this thing should be configured. Also server-side lazy loading model won’t work in JS — queries will become async. Given that, server-client communication layer can be implemented, I guess, but it won’t look like anything like vanilla Datomic peer.
- dustingetz 12y ago
  i think you want the application-facing interface to remain REST (links between entities, HATEOAS), but the REST/CRUD client's transport layer wouldn't be traditional http/ajax, rather implement similar capabilities to a Datomic Peer. So a CRUD READ would query a cached Datomic index rather than touch network. Create, Update, Delete could be http/ajax or could be something else like a websocket; I believe transport to be orthogonal to the fundamental principles of REST application design.
- mvzink 12y ago
  Exciting prospect. How about DataScript for your app state + tx-listen/om-sync mirroring transactions? Might require wrapping the datascript/transact! call, and I haven't thought about reading from the cliente. Maybe as simple as lazily mirroring datascript/q calls and filling those back into the DataScript map.
- davidrupp 12y ago
  Datomic is bundled with a REST server: http://docs.datomic.com/rest.html http://docs.datomic.com/rest.html. And it includes an endpoint to subscribe to a stream of Server-Sent Events, which lets you drink from the transaction-report firehose. But there is a lot of data in that stream. I'm writing a production Pedestal app, and I could definitely use the capability to store my data on the client in the same shape as Datomic and query it with Datalog.
skybrian 12y ago
I had no idea what this was, but after some wandering I found some background: http://en.wikipedia.org/wiki/Datalog http://en.wikipedia.org/wiki/Datalog Datalog is apparently a subset of Prolog that's used by Datomic (a commercial database) as its query language.
- danneu 12y ago
  Here's a quick introductory resource: http://www.learndatalogtoday.org http://www.learndatalogtoday.org
  - cjo 12y ago
    The SICP lectures on logic programming will provide some conceptual background if needed. http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/8a-logic-programming-part-1/ http://ocw.mit.edu/courses/electrical-engineering-and-comput...
sethev 12y ago
This looks like a very interesting project although it's a bit confusing that it's named after a proprietary database.
- ibdknox 12y ago
  Why is it confusing? It's a basic port of that proprietary database to JavaScript. It's not like it's something completely unrelated.
  - sethev 12y ago
    In the same way that OracleScript would be an odd choice of name for a SQL database implemented in JavaScript. Now that I say that, though, I guess there isn't really a name for what Datomic is except Datomic.
- puredanger 12y ago
  It has now been renamed to DataScript. https://twitter.com/nikitonsky/status/459403600775876608 https://twitter.com/nikitonsky/status/459403600775876608
gcv 12y ago
Curious: why is this limited to ClojureScript? I can see this being useful in JVM Clojure in projects which do not need real or full Datomic.
- shaunxcode 12y ago
  he actually has a clj version in the src. https://github.com/tonsky/datomicscript/blob/master/src/datomicscript.clj https://github.com/tonsky/datomicscript/blob/master/src/dato...
  - tonsky 12y ago
    It was commited accidentally. Yes, it may be translated to clojure. However, I still don’t know if I should do it, given we have in-memory db bundled with Datomic free client jar.
    - gcv 12y ago
      Isn't your work released under a less restrictive license than main Datomic?
- coolsunglasses 12y ago
  Why even limit it to Clojure and ClojureScript? I could use this from JavaScript or PureScript as an immutable local store to replicate/merge with the backend asynchronously.
  - deleted 12y ago
    [deleted]
  - cjo 12y ago
    The query language is embedded in legal Clojure data structures, so it would be tricky to port to JS. That's a drawback of embedded DSL's I suppose.
    - coolsunglasses 12y ago
imslavko 12y ago
Not familiar with Datomic. Does it mean now you can reuse querying code on the server to real Datomic and on the client (in browser) to DatomicScript? If so, it could lead to some features like latency compensation and stubs similar to Meteor's MongoDB and MiniMongo. That would be cool.
- tonsky 12y ago
  Yes, query language is almost identical.
puredanger 12y ago
...now renamed to DataScript https://twitter.com/nikitonsky/status/459403600775876608 https://twitter.com/nikitonsky/status/459403600775876608
jchrisa 12y ago
Are there any examples of using this from a JavaScript-mostly app? I can see it being useful as a way to clean up existing "global variable" apps.
- tonsky 12y ago
  There isn’t—yet—but it’s a popular request and I’m now thinking it could be made accessible from JS pretty easily
dj-wonk 12y ago
HN moderators: can you please rename the post, since the project has been renamed to DataScript: https://github.com/tonsky/datascript/commit/5beb53d7db03bd274b109c823b182214151b5058 https://github.com/tonsky/datascript/commit/5beb53d7db03bd27...
- dang 12y ago
  Sure.
dj-wonk 12y ago
How about another rename? DatomicScript was clearly a bad idea and possibly infringing. "DataScript" is just misleading. Using "_____Script" creates the appearance that is a scripting language. Perhaps "_____DB" or "_____JS" would be better.
- dj-wonk 12y ago
  How about "NotDatJS" (short for the "Not Datomic" JS Database)?
deleted 12y ago
[deleted]

143 points by sgrove 12y ago

Looks great. I could see using this for some data crunching client-side, maybe even in conjunction with something like graph.js.

drcode 12y ago

Looks like a great library... It would be intriguing to try to use something like DatomicScript for the app state of an Om application. (Edit: Read in a separate blog post that the author of DatomicScript is thinking about this already) At the end of the day though, I still think someone (smarter than me) needs to figure out the full end-to-end solution for datomic+clojurescript, with both client and server elements. The core question: Does it make sense to map datomic to a REST API on the server, then consume the REST on the client? Or, does it make more sense to build something more clojure-like for the client-server interface, and remove REST from the equation entirely?

tony_landis 12y ago
A seamless datomic <-> clojurescript client/service framework is needed. I am just getting started with Om and hope to experiment with some ideas in that space. I don't think REST is the right approach. I would like to abstract everything to an AVET or sequence thereof.
vdm 12y ago
https://twitter.com/richhickey/status/355394063165440000 https://twitter.com/richhickey/status/355394063165440000
- drcode 12y ago
  Interesting, I get the feeling though that RH also thinks this isn't the right approach, that some more though needs to be put into the client-server interface still that goes beyond vanilla datomic.
  - tonsky 12y ago
    Thing is, server-side Datomic peers work in trusted env; every peer sees everything. As you start doing something like a peer in every browser that opens your app, you need to provide it only with the part of the database. This filtering should be done two-way (what’s allowed to read, what’s allowed to push back) and is not very straightforward thing to do. It’s not even clear how this thing should be configured. Also server-side lazy loading model won’t work in JS — queries will become async. Given that, server-client communication layer can be implemented, I guess, but it won’t look like anything like vanilla Datomic peer.
dustingetz 12y ago
i think you want the application-facing interface to remain REST (links between entities, HATEOAS), but the REST/CRUD client's transport layer wouldn't be traditional http/ajax, rather implement similar capabilities to a Datomic Peer. So a CRUD READ would query a cached Datomic index rather than touch network. Create, Update, Delete could be http/ajax or could be something else like a websocket; I believe transport to be orthogonal to the fundamental principles of REST application design.
mvzink 12y ago
Exciting prospect. How about DataScript for your app state + tx-listen/om-sync mirroring transactions? Might require wrapping the datascript/transact! call, and I haven't thought about reading from the cliente. Maybe as simple as lazily mirroring datascript/q calls and filling those back into the DataScript map.
davidrupp 12y ago
Datomic is bundled with a REST server: http://docs.datomic.com/rest.html http://docs.datomic.com/rest.html. And it includes an endpoint to subscribe to a stream of Server-Sent Events, which lets you drink from the transaction-report firehose. But there is a lot of data in that stream. I'm writing a production Pedestal app, and I could definitely use the capability to store my data on the client in the same shape as Datomic and query it with Datalog.

skybrian 12y ago

I had no idea what this was, but after some wandering I found some background: http://en.wikipedia.org/wiki/Datalog http://en.wikipedia.org/wiki/Datalog Datalog is apparently a subset of Prolog that's used by Datomic (a commercial database) as its query language.

danneu 12y ago
Here's a quick introductory resource: http://www.learndatalogtoday.org http://www.learndatalogtoday.org
- cjo 12y ago
  The SICP lectures on logic programming will provide some conceptual background if needed. http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/8a-logic-programming-part-1/ http://ocw.mit.edu/courses/electrical-engineering-and-comput...

sethev 12y ago

This looks like a very interesting project although it's a bit confusing that it's named after a proprietary database.

ibdknox 12y ago
Why is it confusing? It's a basic port of that proprietary database to JavaScript. It's not like it's something completely unrelated.
- sethev 12y ago
  In the same way that OracleScript would be an odd choice of name for a SQL database implemented in JavaScript. Now that I say that, though, I guess there isn't really a name for what Datomic is except Datomic.
puredanger 12y ago
It has now been renamed to DataScript. https://twitter.com/nikitonsky/status/459403600775876608 https://twitter.com/nikitonsky/status/459403600775876608

gcv 12y ago

Curious: why is this limited to ClojureScript? I can see this being useful in JVM Clojure in projects which do not need real or full Datomic.

shaunxcode 12y ago
he actually has a clj version in the src. https://github.com/tonsky/datomicscript/blob/master/src/datomicscript.clj https://github.com/tonsky/datomicscript/blob/master/src/dato...
- tonsky 12y ago
  It was commited accidentally. Yes, it may be translated to clojure. However, I still don’t know if I should do it, given we have in-memory db bundled with Datomic free client jar.
  - gcv 12y ago
    Isn't your work released under a less restrictive license than main Datomic?
coolsunglasses 12y ago
Why even limit it to Clojure and ClojureScript? I could use this from JavaScript or PureScript as an immutable local store to replicate/merge with the backend asynchronously.
- deleted 12y ago
  [deleted]
- cjo 12y ago
  The query language is embedded in legal Clojure data structures, so it would be tricky to port to JS. That's a drawback of embedded DSL's I suppose.
  - coolsunglasses 12y ago

imslavko 12y ago

Not familiar with Datomic. Does it mean now you can reuse querying code on the server to real Datomic and on the client (in browser) to DatomicScript? If so, it could lead to some features like latency compensation and stubs similar to Meteor's MongoDB and MiniMongo. That would be cool.

tonsky 12y ago
Yes, query language is almost identical.

puredanger 12y ago

...now renamed to DataScript https://twitter.com/nikitonsky/status/459403600775876608 https://twitter.com/nikitonsky/status/459403600775876608

jchrisa 12y ago

Are there any examples of using this from a JavaScript-mostly app? I can see it being useful as a way to clean up existing "global variable" apps.

tonsky 12y ago
There isn’t—yet—but it’s a popular request and I’m now thinking it could be made accessible from JS pretty easily

dj-wonk 12y ago

HN moderators: can you please rename the post, since the project has been renamed to DataScript: https://github.com/tonsky/datascript/commit/5beb53d7db03bd274b109c823b182214151b5058 https://github.com/tonsky/datascript/commit/5beb53d7db03bd27...

dang 12y ago
Sure.

dj-wonk 12y ago

How about another rename? DatomicScript was clearly a bad idea and possibly infringing. "DataScript" is just misleading. Using "_____Script" creates the appearance that is a scripting language. Perhaps "_____DB" or "_____JS" would be better.

dj-wonk 12y ago
How about "NotDatJS" (short for the "Not Datomic" JS Database)?

deleted 12y ago

[deleted]