4 ms·

I've deployed Datomic on AWS using cloud formation to spin up two transactors for when one goes down under pressure. Throw out everything you ever thought abou

I've deployed Datomic on AWS using cloud formation to spin up two transactors for when one goes down under pressure. Throw out everything you ever thought about databases. It is from here you will learn a completely new way of dealing with data especially in environments where you need an immutable audit trail. I almost feel like Datomic solves what blockchain does (immutability) really well without the drama. Clojure was also another culture shock. I like it personally but the syntax at first was scary (but it's really simple and straightforward). Having said that I'm not 100% sure whether I realized any productivity boost working with clojure. Finding clojure developers is tough & expensive vs Java or .NET or PHP. Datomic has a bit of a learning curve and I still haven't fully crossed over, relying on google groups discussions and it's not a user friendly experience which I feel may be their limiting factor (they've made it far too difficult for the wandering developer to approach it) and the fact that it's closed source and proprietary but this is most likely because a lone developer hobbyist has no business using Datomic because it really makes sense in industries where you need a fully auditable trail, something that was typically available in event sourcing frameworks with some sort of CQRS wizardry that I've had trouble justifying using. All in all, unlimited peer is a welcome news. However, datomic in all in it's shape or form, only applies to a small group of developers it seems, which is a shame because datalog queries blows SQL out of the water. The stuff one can do with datalog on datomic tightly integrated with clojure, it's game changing, but won't apply to all use cases for folks on HN.

garysieling 10y ago
Can you elaborate on "datalog queries blows SQL out of the water"?
- hcarvalhoalves 10y ago
  Not OP, but at my company we're heavy users of Datomic. Some examples: - Declarative and pattern matching make it elegant to read. E.g., find me the ids of all customers invited by a customer named "Joe": {:find [?customer] :where [ [?inviter :customer/name "Joe"] [?customer :customer/invited-by ?inviter]]} - Syntax more appropriate for code generation and introspection (since it's just a bunch of nested lists). A big plus when pairing w/ Clojure. - Allows recursive queries. - You can reference metadata associated to any transaction that ever touched any attribute in your database within the same query constructs. E.g, find me the time customer name was last modified: {:find [?customer ?name ?last-modified] :where [ [?customer :customer/name ?name ?tx] [?tx :db/txInstant ?last-modified]]} - You can easily query the entire history of records. E.g, find me the name every customer ever had, w/ date of modification {:find [?customer ?name ?last-modified] :where [ [?customer :customer/name ?name ?tx true] [?tx :db/txInstant ?last-modified]]} Almost the same as query above, only difference would be executed using a different Datomic API call (http://docs.datomic.com/clojure/#datomic.api/history http://docs.datomic.com/clojure/#datomic.api/history). - You can easily execute a query against how your database looked like months ago. Same thing: would look like the query above, but executed using a different API call (http://docs.datomic.com/clojure/#datomic.api/as-of http://docs.datomic.com/clojure/#datomic.api/as-of) I would say these are the main ones, but there's more.
  - garysieling 10y ago
    Awesome, thanks! The recursion looks compelling. I'd be interested to know if anyone has compared Datalog and Neo4j/Cypher.
    - samuell 10y ago
      Cypher doesn't have named queries (and thus not recursive ditto either), AFAIK. I asked some Neo4j guys on a meetup some time ago, and they didn't even seem to really understand the question at first. It's a pity. I'd be very happy to use Neo4J if it had.
      - garysieling 10y ago
        Interesting. It seems like to query a graph is implicitly recursive (especially compared to SQL), although I can see how something like pagerank might need more.
    - hcarvalhoalves 10y ago
      Neo4j focuses on modeling nodes and it's relationships, and allows you to attach metadata to the later, but doesn't really have transactions as first-class objects. Datomic focuses on modeling facts (entity-attribute-value). This is a very stable data representation that allows modelling tables, documents or graphs [1], on top of carrying transaction explicitly on an additional index for the time-travel magic. I guess Neo4j might make a good DB for ETL-ing into and running certain kinds of analysis. [1] https://hashrocket.com/blog/posts/using-datomic-as-a-graph-database https://hashrocket.com/blog/posts/using-datomic-as-a-graph-d...
      - garysieling 10y ago
        Awesome, thanks. I'm building a search engine (https://www.findlectures.com https://www.findlectures.com) and I've been looking at ETL-ing the data into/out of Neo4j to do some data augmentation, but the history feature of Datomic is really compelling if I used it as a primary data store.
velocitypsycho 10y ago
Do you know if Clojure devs are expensive compared to Python and Ruby devs of similar experience? I feel the comparison to Java, .Net, & PHP maybe just a comparison of huge mainstream language to more "niche" languages.
- erichmond 10y ago
  Here in NYC, our clojure devs are not much more expensive then ruby devs. With the added bonus that clojure tends to yield more efficiency then ruby. Disclaimer: 12 years ruby/rails experience 3-4 years clojure experience
jgalt212 10y ago
> I almost feel like Datomic solves what blockchain does (immutability) really well without the drama. Well said. Unless you need to solve the Byzantine general problem there are better ways to database audit trails than Blockchain. e.g. Datomic or time travel queries.
- hornytoad 10y ago
  Datomic has a centralized transactor serializing what transactions, whereas Blockchain is decentralized. So they are at opposite ends of the spectrum, each trying to solve a completely different problem. Of course "centralized" is much less dramatic.
  - jgalt212 10y ago
    yes, I am aware of that, but many fintech start ups are using blockchain with centralized data stores. and here's the link to byzantine generals' problem https://en.wikipedia.org/wiki/Byzantine_fault_tolerance#The_Byzantine_Generals.27_Problem https://en.wikipedia.org/wiki/Byzantine_fault_tolerance#The_...
di4na 10y ago
+1 on blockchain. Most of these "Blockchain" fintech company are just doing a distributed logs between trusted peers. nothing more.

garysieling 10y ago

Can you elaborate on "datalog queries blows SQL out of the water"?

hcarvalhoalves 10y ago
Not OP, but at my company we're heavy users of Datomic. Some examples: - Declarative and pattern matching make it elegant to read. E.g., find me the ids of all customers invited by a customer named "Joe": {:find [?customer] :where [ [?inviter :customer/name "Joe"] [?customer :customer/invited-by ?inviter]]} - Syntax more appropriate for code generation and introspection (since it's just a bunch of nested lists). A big plus when pairing w/ Clojure. - Allows recursive queries. - You can reference metadata associated to any transaction that ever touched any attribute in your database within the same query constructs. E.g, find me the time customer name was last modified: {:find [?customer ?name ?last-modified] :where [ [?customer :customer/name ?name ?tx] [?tx :db/txInstant ?last-modified]]} - You can easily query the entire history of records. E.g, find me the name every customer ever had, w/ date of modification {:find [?customer ?name ?last-modified] :where [ [?customer :customer/name ?name ?tx true] [?tx :db/txInstant ?last-modified]]} Almost the same as query above, only difference would be executed using a different Datomic API call (http://docs.datomic.com/clojure/#datomic.api/history http://docs.datomic.com/clojure/#datomic.api/history). - You can easily execute a query against how your database looked like months ago. Same thing: would look like the query above, but executed using a different API call (http://docs.datomic.com/clojure/#datomic.api/as-of http://docs.datomic.com/clojure/#datomic.api/as-of) I would say these are the main ones, but there's more.
- garysieling 10y ago
  Awesome, thanks! The recursion looks compelling. I'd be interested to know if anyone has compared Datalog and Neo4j/Cypher.
  - samuell 10y ago
    Cypher doesn't have named queries (and thus not recursive ditto either), AFAIK. I asked some Neo4j guys on a meetup some time ago, and they didn't even seem to really understand the question at first. It's a pity. I'd be very happy to use Neo4J if it had.
    - garysieling 10y ago
      Interesting. It seems like to query a graph is implicitly recursive (especially compared to SQL), although I can see how something like pagerank might need more.
  - hcarvalhoalves 10y ago
    Neo4j focuses on modeling nodes and it's relationships, and allows you to attach metadata to the later, but doesn't really have transactions as first-class objects. Datomic focuses on modeling facts (entity-attribute-value). This is a very stable data representation that allows modelling tables, documents or graphs [1], on top of carrying transaction explicitly on an additional index for the time-travel magic. I guess Neo4j might make a good DB for ETL-ing into and running certain kinds of analysis. [1] https://hashrocket.com/blog/posts/using-datomic-as-a-graph-database https://hashrocket.com/blog/posts/using-datomic-as-a-graph-d...
    - garysieling 10y ago
      Awesome, thanks. I'm building a search engine (https://www.findlectures.com https://www.findlectures.com) and I've been looking at ETL-ing the data into/out of Neo4j to do some data augmentation, but the history feature of Datomic is really compelling if I used it as a primary data store.

velocitypsycho 10y ago

Do you know if Clojure devs are expensive compared to Python and Ruby devs of similar experience? I feel the comparison to Java, .Net, & PHP maybe just a comparison of huge mainstream language to more "niche" languages.

erichmond 10y ago
Here in NYC, our clojure devs are not much more expensive then ruby devs. With the added bonus that clojure tends to yield more efficiency then ruby. Disclaimer: 12 years ruby/rails experience 3-4 years clojure experience

jgalt212 10y ago

> I almost feel like Datomic solves what blockchain does (immutability) really well without the drama. Well said. Unless you need to solve the Byzantine general problem there are better ways to database audit trails than Blockchain. e.g. Datomic or time travel queries.

hornytoad 10y ago
Datomic has a centralized transactor serializing what transactions, whereas Blockchain is decentralized. So they are at opposite ends of the spectrum, each trying to solve a completely different problem. Of course "centralized" is much less dramatic.
- jgalt212 10y ago
  yes, I am aware of that, but many fintech start ups are using blockchain with centralized data stores. and here's the link to byzantine generals' problem https://en.wikipedia.org/wiki/Byzantine_fault_tolerance#The_Byzantine_Generals.27_Problem https://en.wikipedia.org/wiki/Byzantine_fault_tolerance#The_...

di4na 10y ago

+1 on blockchain. Most of these "Blockchain" fintech company are just doing a distributed logs between trusted peers. nothing more.