5 ms·

Datomic (as presented in this video) seems like a great thing. But if it's so great, why are so few people talking about it? Is it because it only exists for

Datomic (as presented in this video) seems like a great thing. But if it's so great, why are so few people talking about it? Is it because it only exists for Clojure, and few people know or talk about Clojure? I love Clojure as a language, and I have wanted to use it in production; but there are so few opportunities other than solo building - where you really have to climb a steep wall due to the Clojurist mentality of /Frameworks Bad! Choose all your own libraries good!/. Instead I have found myself happy and productive in the Elixir/BEAM world, of course initially because of Phoenix. And like many people, I have just been accepting the destructive, non-historic approach to data management with typical RDBMSs. If there were a Datomic for Elixir, I'd probably use it. Must Datomic (or a data management approach like it) be tightly coupled with Clojure?

cutler 2y ago
There are plenty of Clojure frameworks. Electric Clojure is the latest but Kit with HugSQL serves all my needs.
- michaelteter 2y ago
  Electric looks awesome, but their stern warning of “we are building this for ourselves, and if you get value that’s great” gave me pause. If it reaches a point of being documented and intended for general public use, then I’ll definitely try a project with it.
  - dustingetz 2y ago
    That's right, I have a blog post cooking up about why Electric is for experts today. A major factor in this is because, like with Clojure, the users aren't paying us, so the documentation you want cannot yet afford to exist. Another is performance - to get Electric to purr you have to understand what you are asking the computer to do (I've seen the stuff senior engineers type with their AI codegen tools, that approach is simply not viable here, at least not yet). The net impact of these two factors is that if Electric is not obviously the exact thing you know you must have—i.e., you are already succeeding or have the possibility of succeeding with something else—there is high risk that your adoption will not succeed, leaving you frustrated and unhappy! Failed projects do neither of us any good, that is a recipe for a damaged brand. A bonus third factor is that the demos we've been cooking up internally—that we haven't revealed yet—are so f%cking incredible that everyone is going to be motivated to use and learn it anyway because Electric yields value that is previously unseen and unavailable anywhere else. So I am simply setting healthy expectations for success. For example, we just built the 80% that matters of the “sync engine” value prop in two weeks and 100 LOC. Implementing it in userland requires 1 LOC per query. With differential network traffic for over the wire O(1) remote incremental collection maintenance! for free! And the pattern works with any database!
  - diggan 2y ago
    > Electric looks awesome, but their stern warning of “we are building this for ourselves, and if you get value that’s great” gave me pause. Clojure (the core language) is developed in exactly the same way, Rich Hickey is pretty forthcoming with the approach they take. So if a framework with that approach gives you pause, probably Clojure the language should do the same. Relevant: https://gist.github.com/richhickey/1563cddea1002958f96e7ba9519972d9 https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...
    - michaelteter 2y ago
      But I know Clojure is in use in quite a few places, and it has quite a few really capable people supporting it in one way or another. Plus it has good documentation and several books teaching it. That's very different from a slick (and impressive) framework built by one company for themselves. Also, code written for Electric is very specific to Electric. But Clojure is just functional-first Lisp. It's very easy to rewrite in another language, even an imperative language. There wouldn't be a huge paradigm shift translating Clojure to any of half a dozen popular languages.
    - fulafel 2y ago
      Clojure not developed in the same way - it's really conservative about backwards compatibility and puts a lot of thought in existing users. Electric is still making backwards incompatible changes, in contrast. You're probably thinking about the open source community developed project vs "it's not a democracy" aspect where Clojure is definitely in the latter camp.
augustl 2y ago
I've used Datomic from both Kotin and Groovy (!) I presented on Datomic at KotlinConf too, with some live coding starting around the 31 minute mark https://www.youtube.com/watch?v=hicQvxdKvnc https://www.youtube.com/watch?v=hicQvxdKvnc You probably need to be on the JVM, as the peer library (i.e. the "good one", where you embed the full query engine and data fetching directly into your business logic) is so far only implemented for the JVM. I suppose 10+ years of weird license models and a hefty price tag haven't helped. Datomic turned free (but still proprietary) in 2023 though. But why Datomic isn't more widely adopted is a huge mystery to me...
- michaelteter 2y ago
  Thanks. I’ll check out the video. At this point I’m pretty addicted to BEAM and the Elixir ecosystem though. (And LiveView!).
  - macintux 2y ago
    I doubt it's still the case, but once upon a time Riak (written primarily in Erlang) was one of the preferred storage backends for Datomic.
- foobarian 2y ago
  > But why Datomic isn't more widely adopted is a huge mystery to me... I've been hearing about Datomic on and off over the years, and I never saw a clear answer to the simple question: what is it? What does it do? Instead I saw it most often mentioned as an example of a successful Clojure project, and now that's how I think of it. Seems like poor marketing/branding if there ever was intentional attempt at it.
  - diggan 2y ago
    > I've been hearing about Datomic on and off over the years, and I never saw a clear answer to the simple question: what is it? What does it do? I'm guessing most people, like me, first heard about both Clojure and Datomic from one of Hickey's famous talks about software development. Coming from one of those, I guess it's a bit easier to understand what the various concepts and words mean from Datomic's website for example. I guess the easiest way to put it: Datomic is a append-only (deletions via "retraction" records) database that lets you query through time (also called "Bitemporal modeling" with fancier words). The features/benefits page I think is pretty clear (https://www.datomic.com/benefits.html https://www.datomic.com/benefits.html) but again, might just be my familiarity speaking.
kragen 2y ago
unlike clojure, datomic is proprietary software, so you'd be a fool to choose to invest your time in learning how to use it unless you're rich hickey investing your time in learning how it works so you can clone it might be worthwhile
refset 2y ago
> If there were a Datomic for Elixir, I'd probably use it. XTDB may possibly of interest in that case - there's been some recent chatter about adding support for Ecto via Postgres compatibility: https://discuss.xtdb.com/t/elixir-ecto-thread-from-slack/490 https://discuss.xtdb.com/t/elixir-ecto-thread-from-slack/490 (I work on XTDB)
seanc 2y ago
I used Datomic for a few years at a job. In addition to the cost and proprietary nature, I'd say there are two reasons; 1) New and different things come with risk, and many folks are risk averse, especially in groups 2) More concretely, it is very easy to write slow queries in Datomic, and it can be a struggle to diagnose why the order of your datalog statements matters so much
fulafel 2y ago
I think the takeoff was hampered by being paid software at the time when it generated the most excitement and then the "free to use but proprietary" concided with XTDB. (And then for XTDB its v1 -> v2 reboot dampened that one)
wry_discontent 2y ago
It's because Datomic has a confusing new model, and suffers from the same beginner unfriendliness as the rest of Clojure. It's a language aimed at experienced old school programmers, and doesn't aim to be friendly to younger less experienced programmers. They needed a better marketing team for the product.
- jwr 2y ago
  > It's a language aimed at experienced old school programmers, and doesn't aim to be friendly to younger less experienced programmers. I think Rich Hickey had a point when he said "[musical] instruments are made for people who can play them". I don't see people complaining about the piano or the saxophone being difficult for beginners. They are.
apwell23 2y ago
Its closed source software thats VERY expensive and supported by one tiny company somewhere. Try selling that to management as the place where you want to keep their data.
- nikodotio 2y ago
  It is free now: https://blog.datomic.com/2023/04/datomic-is-free.html https://blog.datomic.com/2023/04/datomic-is-free.html
  - rjbwork 2y ago
    It's still closed source. If this company goes kaput, you're hosed. If you need to customize it to make progress years down the line you're hosed.
- armincerf 2y ago
  Ok but XTDB has a lot in common with Datomic and is open source but still hardly widely used. I think this is partly because most people don't consider 'temporality' as a feature a database should offer; rather, they believe temporal problems should be solved via proper schema design and application logic. Additionally, using datalog (or any esoteric query language that isn't SQL) locks you out of many battle-tested tools that enterprises rely on. The XTDB team has pivoted towards a SQL-first approach (though still supports datalog) and now 'only' has the 'But why not just use Postgres' problem to solve. Having personally moved from trying to use Postgres for everything (including lots of timeseries data with Timescale) to a dedicated and relatively unknown DB built purely for the purpose I want it for (QuestDB), I am all for more people trying to build databases that do specific things better than Postgres. However, it will be very difficult to create something that does literally everything Postgres can do but better, which probably makes Postgres the sensible choice for the majority of applications.
  - refset 2y ago
    > this is partly because most people don't consider 'temporality' as a feature a database should offer Not sure on your definition of 'people', but I think every business ultimately wants solid auditing and reporting capabilities across their IT systems. These concerns are only increasing in importance as new regulations demand stronger data provenance, but their implementation shouldn't be reliant on the process of "proper schema design" to get things right first time. Databases built for the modern world should be making this stuff bulletproof and easy. (I work on XTDB - and if Postgres already supported temporal tables I possibly wouldn't!)

cutler 2y ago

There are plenty of Clojure frameworks. Electric Clojure is the latest but Kit with HugSQL serves all my needs.

michaelteter 2y ago
Electric looks awesome, but their stern warning of “we are building this for ourselves, and if you get value that’s great” gave me pause. If it reaches a point of being documented and intended for general public use, then I’ll definitely try a project with it.
- dustingetz 2y ago
  That's right, I have a blog post cooking up about why Electric is for experts today. A major factor in this is because, like with Clojure, the users aren't paying us, so the documentation you want cannot yet afford to exist. Another is performance - to get Electric to purr you have to understand what you are asking the computer to do (I've seen the stuff senior engineers type with their AI codegen tools, that approach is simply not viable here, at least not yet). The net impact of these two factors is that if Electric is not obviously the exact thing you know you must have—i.e., you are already succeeding or have the possibility of succeeding with something else—there is high risk that your adoption will not succeed, leaving you frustrated and unhappy! Failed projects do neither of us any good, that is a recipe for a damaged brand. A bonus third factor is that the demos we've been cooking up internally—that we haven't revealed yet—are so f%cking incredible that everyone is going to be motivated to use and learn it anyway because Electric yields value that is previously unseen and unavailable anywhere else. So I am simply setting healthy expectations for success. For example, we just built the 80% that matters of the “sync engine” value prop in two weeks and 100 LOC. Implementing it in userland requires 1 LOC per query. With differential network traffic for over the wire O(1) remote incremental collection maintenance! for free! And the pattern works with any database!
- diggan 2y ago
  > Electric looks awesome, but their stern warning of “we are building this for ourselves, and if you get value that’s great” gave me pause. Clojure (the core language) is developed in exactly the same way, Rich Hickey is pretty forthcoming with the approach they take. So if a framework with that approach gives you pause, probably Clojure the language should do the same. Relevant: https://gist.github.com/richhickey/1563cddea1002958f96e7ba9519972d9 https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...
  - michaelteter 2y ago
    But I know Clojure is in use in quite a few places, and it has quite a few really capable people supporting it in one way or another. Plus it has good documentation and several books teaching it. That's very different from a slick (and impressive) framework built by one company for themselves. Also, code written for Electric is very specific to Electric. But Clojure is just functional-first Lisp. It's very easy to rewrite in another language, even an imperative language. There wouldn't be a huge paradigm shift translating Clojure to any of half a dozen popular languages.
  - fulafel 2y ago
    Clojure not developed in the same way - it's really conservative about backwards compatibility and puts a lot of thought in existing users. Electric is still making backwards incompatible changes, in contrast. You're probably thinking about the open source community developed project vs "it's not a democracy" aspect where Clojure is definitely in the latter camp.

augustl 2y ago

I've used Datomic from both Kotin and Groovy (!) I presented on Datomic at KotlinConf too, with some live coding starting around the 31 minute mark https://www.youtube.com/watch?v=hicQvxdKvnc https://www.youtube.com/watch?v=hicQvxdKvnc You probably need to be on the JVM, as the peer library (i.e. the "good one", where you embed the full query engine and data fetching directly into your business logic) is so far only implemented for the JVM. I suppose 10+ years of weird license models and a hefty price tag haven't helped. Datomic turned free (but still proprietary) in 2023 though. But why Datomic isn't more widely adopted is a huge mystery to me...

michaelteter 2y ago
Thanks. I’ll check out the video. At this point I’m pretty addicted to BEAM and the Elixir ecosystem though. (And LiveView!).
- macintux 2y ago
  I doubt it's still the case, but once upon a time Riak (written primarily in Erlang) was one of the preferred storage backends for Datomic.
foobarian 2y ago
> But why Datomic isn't more widely adopted is a huge mystery to me... I've been hearing about Datomic on and off over the years, and I never saw a clear answer to the simple question: what is it? What does it do? Instead I saw it most often mentioned as an example of a successful Clojure project, and now that's how I think of it. Seems like poor marketing/branding if there ever was intentional attempt at it.
- diggan 2y ago
  > I've been hearing about Datomic on and off over the years, and I never saw a clear answer to the simple question: what is it? What does it do? I'm guessing most people, like me, first heard about both Clojure and Datomic from one of Hickey's famous talks about software development. Coming from one of those, I guess it's a bit easier to understand what the various concepts and words mean from Datomic's website for example. I guess the easiest way to put it: Datomic is a append-only (deletions via "retraction" records) database that lets you query through time (also called "Bitemporal modeling" with fancier words). The features/benefits page I think is pretty clear (https://www.datomic.com/benefits.html https://www.datomic.com/benefits.html) but again, might just be my familiarity speaking.

kragen 2y ago

unlike clojure, datomic is proprietary software, so you'd be a fool to choose to invest your time in learning how to use it unless you're rich hickey investing your time in learning how it works so you can clone it might be worthwhile

refset 2y ago

> If there were a Datomic for Elixir, I'd probably use it. XTDB may possibly of interest in that case - there's been some recent chatter about adding support for Ecto via Postgres compatibility: https://discuss.xtdb.com/t/elixir-ecto-thread-from-slack/490 https://discuss.xtdb.com/t/elixir-ecto-thread-from-slack/490 (I work on XTDB)

seanc 2y ago

I used Datomic for a few years at a job. In addition to the cost and proprietary nature, I'd say there are two reasons; 1) New and different things come with risk, and many folks are risk averse, especially in groups 2) More concretely, it is very easy to write slow queries in Datomic, and it can be a struggle to diagnose why the order of your datalog statements matters so much

fulafel 2y ago

I think the takeoff was hampered by being paid software at the time when it generated the most excitement and then the "free to use but proprietary" concided with XTDB. (And then for XTDB its v1 -> v2 reboot dampened that one)

wry_discontent 2y ago

It's because Datomic has a confusing new model, and suffers from the same beginner unfriendliness as the rest of Clojure. It's a language aimed at experienced old school programmers, and doesn't aim to be friendly to younger less experienced programmers. They needed a better marketing team for the product.

jwr 2y ago
> It's a language aimed at experienced old school programmers, and doesn't aim to be friendly to younger less experienced programmers. I think Rich Hickey had a point when he said "[musical] instruments are made for people who can play them". I don't see people complaining about the piano or the saxophone being difficult for beginners. They are.

apwell23 2y ago

Its closed source software thats VERY expensive and supported by one tiny company somewhere. Try selling that to management as the place where you want to keep their data.

nikodotio 2y ago
It is free now: https://blog.datomic.com/2023/04/datomic-is-free.html https://blog.datomic.com/2023/04/datomic-is-free.html
- rjbwork 2y ago
  It's still closed source. If this company goes kaput, you're hosed. If you need to customize it to make progress years down the line you're hosed.
armincerf 2y ago
Ok but XTDB has a lot in common with Datomic and is open source but still hardly widely used. I think this is partly because most people don't consider 'temporality' as a feature a database should offer; rather, they believe temporal problems should be solved via proper schema design and application logic. Additionally, using datalog (or any esoteric query language that isn't SQL) locks you out of many battle-tested tools that enterprises rely on. The XTDB team has pivoted towards a SQL-first approach (though still supports datalog) and now 'only' has the 'But why not just use Postgres' problem to solve. Having personally moved from trying to use Postgres for everything (including lots of timeseries data with Timescale) to a dedicated and relatively unknown DB built purely for the purpose I want it for (QuestDB), I am all for more people trying to build databases that do specific things better than Postgres. However, it will be very difficult to create something that does literally everything Postgres can do but better, which probably makes Postgres the sensible choice for the majority of applications.
- refset 2y ago
  > this is partly because most people don't consider 'temporality' as a feature a database should offer Not sure on your definition of 'people', but I think every business ultimately wants solid auditing and reporting capabilities across their IT systems. These concerns are only increasing in importance as new regulations demand stronger data provenance, but their implementation shouldn't be reliant on the process of "proper schema design" to get things right first time. Databases built for the modern world should be making this stuff bulletproof and easy. (I work on XTDB - and if Postgres already supported temporal tables I possibly wouldn't!)