6 ms·

Ions: Develop cloud applications by deploying to a running Datomic cluster

169 points by simonpure 8y ago

jb1991 8y ago
Man, am I ever a dummy! I look at the diagram on that page and it might as well be Egyptian hieroglyphs to me. I'd have to study that thing one long time to even understand what problem Ions is solving. I'm sure it's a real problem with an interesting solution, but geez the modern Cloud world has really gotten quite complex. I feel particularly silly because I've been writing Clojure professionally for many years.
- baxtr 8y ago
  There are always two sides to this... I think it’s badly communicated. Someone said (I don’t find the quote), that if you can’t explain a complex topic in simple words, you haven’t understood it thoroughly.
  - cube2222 8y ago
    Einstein said this, but he also said: Everything Should Be Made as Simple as Possible, But Not Simpler.
  - VaedaStrike 8y ago
    If you know Clojure and AWS I really don’t think it is poorly communicated. There’s a limit to that idiom of simplicity and understanding being correlated. This was a walk through of the mechanics. The TL;DR is if you are comfortable enough in AWS and Clojure implementing your system in a scalable and simple and wholistic way just got a bunch more accessibile.
    - zimablue 8y ago
      I've written a clojure app, used aws and lambda functions and skimming this I really wasn't sure wtf they were talking about. Datomic... Something something lambda functions, deployment. I know that they have an existing ami that uses a cloudformation template to let you set up a datomic server on AWS, this is something similar maybe? I think I could get it if I read in detail but this is definitely poorly communicated, it reads like old Microsoft enterprise sales copy.
- nickik 8y ago
  This is about Datomic, not really Clojure. So if you don't have experience with Datomic Clojure understanding would not really help you. Seems to me this is a combination of Datomic and Amazon Lambda to deploy programs that can use Datomic without all surrounding deployment problems.
  - EpicEng 8y ago
    >Ions let you develop applications for the cloud by deploying your code to a running Datomic cluster I could be totally off base here as I have no real experience with clojure, but this intro sentence threw me off right away. >Ions let you develop applications for the cloud by deploying your code to a running Datomic cluster What is an ion? What is a "datomic cluster"? Are these terms just new jargon for what is essentially a deployment system? If they're a clojure thing then ok, but a quick search makes me think they're not. I understand software and I understand deployment. I read the rest of the page and I feel like I would have to problem understanding what this really is if they cut down on the jargon.
    - nickik 8y ago
      Ion is a name as far as I know. Datomic is a database consists of multiple services so you can deploy it on one machine or multiple. I called it a cluster because on AWS of course the idea is to deploy it on multiple. Amazon Lambda is a way to deploy code without telling Amazon about the underlying machines/VMs. Ion is a way to deploy Clojure and Datomic without knowing anything about the underlying machines or VMs.
    - paradigmshiv 8y ago
      Datomic is an immutable database, written in Clojure. It's pretty cool, you should check it out. The company behind Datomic (Cognitect, which also develops the Clojure language) recently started offering the database as a cloud service. This was something Datomic users had wanted for a long time, but one problem with the new service was that one of Datomic's primary features, i.e. having the database reside in memory with your application, was missing. Ions is a new service that addresses this problem by allowing you to deploy your app to the same VMs that are hosting Datomic in the cloud (a "datomic cluster"). It's basically PaaS for Datomic-backed applications. I'm a longtime Clojure user and I had some trouble understanding all this as well, so I agree that could do a better job with presentation.
- JBiserkov 8y ago
  In addition to the linked page there's also a tutorial https://docs.datomic.com/cloud/ions/ions-tutorial.html https://docs.datomic.com/cloud/ions/ions-tutorial.html and a reference https://docs.datomic.com/cloud/ions/ions-reference.html https://docs.datomic.com/cloud/ions/ions-reference.html
nickik 8y ago
I really love Datomic and it solves so many problems that I usually have when doing application development special in bank and financial context. We are already using Clojure and I would love to introduce Datomic as well but being a commercial database makes it pretty difficult. This new setup is sounds cool but I would love something like that on a Kubernetes stack. In the real world Amazon is a no-go in so many situation. I must say I often found the setup needlessly complicated. There is not much application for making setup and development easy when outside of AWS. No official way to set up of the whole stack with Docker or Kubernetes for example.
- alter_eco123 8y ago
  > We are already using Clojure and I would love to introduce Datomic as well but being a commercial database makes it pretty difficult. I don't see why paying for Datomic would be a problem, but if the license is as crazy as it used to be, that's a bit scary. For example, the license at least used to forbid even saying "Datomic" aloud in public. Publishing benchmarks was also forbidden, which is a big red flag. If Datomic performs well, wouldn't Cognitect want lots of benchmarks out there? If it doesn't, they wouldn't! > I must say I often found the setup needlessly complicated. [..] No official way to set up of the whole stack with Docker or Kubernetes for example. You don't find Kubernetes needlessly complicated? Do you actually need Docker or Kubernetes?
  - hellofunk 8y ago
    > For example, the license at least used to forbid even saying "Datomic" aloud in public. > Publishing benchmarks was also forbidden Seriously? I find this absurd, but I have no reason to think you aren't being accurate. That is some corporate paranoia.
    - ahjones 8y ago
      From the license https://www.datomic.com/datomic-pro-edition-eula.html https://www.datomic.com/datomic-pro-edition-eula.html > The Licensee hereby agrees ... it will not: ... (j) publicly display or communicate the results of internal performance testing or other benchmarking or performance evaluation of the Software;
VaedaStrike 8y ago
This looks like a Godsend :) The only qualm I have is it being only on AWS. Other than that this enables just what I could use right about now!
- mark_l_watson 8y ago
  DynamoDB is a preferred backend, and the us AWS only.
samuell 8y ago
We need more open source datalog implementations with disk based storage. I have my hopes now on Mozilla's Mentat [1], but it seems to still have quite some way to go, and also I wouldn't mind something similar implemented in Go too ... At least something simple, allowing to start working with the datalog paradigm for expressing logic and views. [1] https://github.com/mozilla/mentat https://github.com/mozilla/mentat
- swlkr 8y ago
  Here’s a really early one based on persisting datascript https://github.com/replikativ/datahike https://github.com/replikativ/datahike
vincentdm 8y ago
Can someone explain the benefits of Datomic Ions? What problems is it supposed to solve and how does it compare to existing solutions? Also, what is the story regarding local development? (I also have this concern about AWS Lambda; it seems you can only realistically run code in the actual cloud environment) We are investing heavily in the Clojure-ecosystem (but not yet Datomic). It seems Cognitect is strongly headed into a direction which involves some kind of holistic vision about a new stack, but they don't seem to explicitly communicate this vision anywhere. Edit: I asked the above question also on Clojure Reddit and got an interesting response: https://www.reddit.com/r/Clojure/comments/8p3d5s/datomic_ions_your_app_on_datomic_cloud/e09dz9e/ https://www.reddit.com/r/Clojure/comments/8p3d5s/datomic_ion...
- rockostrich 8y ago
  I can't speak for Datatomic Ions and Clojure specifically, but there's a bunch of tooling options around "running" AWS Lambda locally. The most popular last time I checked was Serverless (https://serverless.com/ https://serverless.com/)
- jeremiahwv 8y ago
  My take is that one of Datomic's original value props was that it ran in your Java app's process. That plus immutability plus cache-foo allowed you to write your Java app as-if your entire db was not only "in-memory", but "in-your-app's-process-memory." That is, um, "pretty dope". Then they introduced Datomic Client, which is more of the "normal" db paradigm. Your app includes the client lib, which connects to the running db process "somewhere else". I don't know why they did this (easier to include?), but it meant that the "db-in-your-app-process" value prop went away. Initially when they deployed Datamic Cloud (a sort of managed Datamic service deployed in AWS), they only supported the client model. That is context for your question. The problem Ions solves is how to get that "db-in-your-app-process" value prop while using Datamic Cloud. It seems to me that it effectively reverses the original method. Original method was add the DB as a lib into your app, Ions method is push your app into the db (which is running in a configured-for-you cluster on AWS). (It also allows you to not have to provision/configure servers to actually run your app code in addition to the db servers, but that seems secondary to me.) Note, that I'm not affiliated with Cognitect, and I've never used Datamic, or Clojure in a production system. So have a salt shaker.
bjconlan 8y ago
It seems Rich (yes and the team at cognitect) might have jumped the shark on this one. (Or this was released too early and the next conf. talk will clear up what the problem is and what it is that this is attempting to solve). Either way it's rare for Rich not to describe to problem a little more before throwing a solution to it. Hopefully more will come to light in time. The brief take away I see from this as a non AWS deployment based user is the ability to go beyond the current limitations of only having access to core library functions in the context of datomic. (I see that deployment of applications also plays a part in this but not sure what that solves exactly)

169 points by simonpure 8y ago

jb1991 8y ago

Man, am I ever a dummy! I look at the diagram on that page and it might as well be Egyptian hieroglyphs to me. I'd have to study that thing one long time to even understand what problem Ions is solving. I'm sure it's a real problem with an interesting solution, but geez the modern Cloud world has really gotten quite complex. I feel particularly silly because I've been writing Clojure professionally for many years.

baxtr 8y ago
There are always two sides to this... I think it’s badly communicated. Someone said (I don’t find the quote), that if you can’t explain a complex topic in simple words, you haven’t understood it thoroughly.
- cube2222 8y ago
  Einstein said this, but he also said: Everything Should Be Made as Simple as Possible, But Not Simpler.
- VaedaStrike 8y ago
  If you know Clojure and AWS I really don’t think it is poorly communicated. There’s a limit to that idiom of simplicity and understanding being correlated. This was a walk through of the mechanics. The TL;DR is if you are comfortable enough in AWS and Clojure implementing your system in a scalable and simple and wholistic way just got a bunch more accessibile.
  - zimablue 8y ago
    I've written a clojure app, used aws and lambda functions and skimming this I really wasn't sure wtf they were talking about. Datomic... Something something lambda functions, deployment. I know that they have an existing ami that uses a cloudformation template to let you set up a datomic server on AWS, this is something similar maybe? I think I could get it if I read in detail but this is definitely poorly communicated, it reads like old Microsoft enterprise sales copy.
nickik 8y ago
This is about Datomic, not really Clojure. So if you don't have experience with Datomic Clojure understanding would not really help you. Seems to me this is a combination of Datomic and Amazon Lambda to deploy programs that can use Datomic without all surrounding deployment problems.
- EpicEng 8y ago
  >Ions let you develop applications for the cloud by deploying your code to a running Datomic cluster I could be totally off base here as I have no real experience with clojure, but this intro sentence threw me off right away. >Ions let you develop applications for the cloud by deploying your code to a running Datomic cluster What is an ion? What is a "datomic cluster"? Are these terms just new jargon for what is essentially a deployment system? If they're a clojure thing then ok, but a quick search makes me think they're not. I understand software and I understand deployment. I read the rest of the page and I feel like I would have to problem understanding what this really is if they cut down on the jargon.
  - nickik 8y ago
    Ion is a name as far as I know. Datomic is a database consists of multiple services so you can deploy it on one machine or multiple. I called it a cluster because on AWS of course the idea is to deploy it on multiple. Amazon Lambda is a way to deploy code without telling Amazon about the underlying machines/VMs. Ion is a way to deploy Clojure and Datomic without knowing anything about the underlying machines or VMs.
  - paradigmshiv 8y ago
    Datomic is an immutable database, written in Clojure. It's pretty cool, you should check it out. The company behind Datomic (Cognitect, which also develops the Clojure language) recently started offering the database as a cloud service. This was something Datomic users had wanted for a long time, but one problem with the new service was that one of Datomic's primary features, i.e. having the database reside in memory with your application, was missing. Ions is a new service that addresses this problem by allowing you to deploy your app to the same VMs that are hosting Datomic in the cloud (a "datomic cluster"). It's basically PaaS for Datomic-backed applications. I'm a longtime Clojure user and I had some trouble understanding all this as well, so I agree that could do a better job with presentation.
JBiserkov 8y ago
In addition to the linked page there's also a tutorial https://docs.datomic.com/cloud/ions/ions-tutorial.html https://docs.datomic.com/cloud/ions/ions-tutorial.html and a reference https://docs.datomic.com/cloud/ions/ions-reference.html https://docs.datomic.com/cloud/ions/ions-reference.html

nickik 8y ago

I really love Datomic and it solves so many problems that I usually have when doing application development special in bank and financial context. We are already using Clojure and I would love to introduce Datomic as well but being a commercial database makes it pretty difficult. This new setup is sounds cool but I would love something like that on a Kubernetes stack. In the real world Amazon is a no-go in so many situation. I must say I often found the setup needlessly complicated. There is not much application for making setup and development easy when outside of AWS. No official way to set up of the whole stack with Docker or Kubernetes for example.

alter_eco123 8y ago
> We are already using Clojure and I would love to introduce Datomic as well but being a commercial database makes it pretty difficult. I don't see why paying for Datomic would be a problem, but if the license is as crazy as it used to be, that's a bit scary. For example, the license at least used to forbid even saying "Datomic" aloud in public. Publishing benchmarks was also forbidden, which is a big red flag. If Datomic performs well, wouldn't Cognitect want lots of benchmarks out there? If it doesn't, they wouldn't! > I must say I often found the setup needlessly complicated. [..] No official way to set up of the whole stack with Docker or Kubernetes for example. You don't find Kubernetes needlessly complicated? Do you actually need Docker or Kubernetes?
- hellofunk 8y ago
  > For example, the license at least used to forbid even saying "Datomic" aloud in public. > Publishing benchmarks was also forbidden Seriously? I find this absurd, but I have no reason to think you aren't being accurate. That is some corporate paranoia.
  - ahjones 8y ago
    From the license https://www.datomic.com/datomic-pro-edition-eula.html https://www.datomic.com/datomic-pro-edition-eula.html > The Licensee hereby agrees ... it will not: ... (j) publicly display or communicate the results of internal performance testing or other benchmarking or performance evaluation of the Software;

VaedaStrike 8y ago

This looks like a Godsend :) The only qualm I have is it being only on AWS. Other than that this enables just what I could use right about now!

mark_l_watson 8y ago
DynamoDB is a preferred backend, and the us AWS only.

samuell 8y ago

We need more open source datalog implementations with disk based storage. I have my hopes now on Mozilla's Mentat [1], but it seems to still have quite some way to go, and also I wouldn't mind something similar implemented in Go too ... At least something simple, allowing to start working with the datalog paradigm for expressing logic and views. [1] https://github.com/mozilla/mentat https://github.com/mozilla/mentat

swlkr 8y ago
Here’s a really early one based on persisting datascript https://github.com/replikativ/datahike https://github.com/replikativ/datahike

vincentdm 8y ago

Can someone explain the benefits of Datomic Ions? What problems is it supposed to solve and how does it compare to existing solutions? Also, what is the story regarding local development? (I also have this concern about AWS Lambda; it seems you can only realistically run code in the actual cloud environment) We are investing heavily in the Clojure-ecosystem (but not yet Datomic). It seems Cognitect is strongly headed into a direction which involves some kind of holistic vision about a new stack, but they don't seem to explicitly communicate this vision anywhere. Edit: I asked the above question also on Clojure Reddit and got an interesting response: https://www.reddit.com/r/Clojure/comments/8p3d5s/datomic_ions_your_app_on_datomic_cloud/e09dz9e/ https://www.reddit.com/r/Clojure/comments/8p3d5s/datomic_ion...

rockostrich 8y ago
I can't speak for Datatomic Ions and Clojure specifically, but there's a bunch of tooling options around "running" AWS Lambda locally. The most popular last time I checked was Serverless (https://serverless.com/ https://serverless.com/)
jeremiahwv 8y ago
My take is that one of Datomic's original value props was that it ran in your Java app's process. That plus immutability plus cache-foo allowed you to write your Java app as-if your entire db was not only "in-memory", but "in-your-app's-process-memory." That is, um, "pretty dope". Then they introduced Datomic Client, which is more of the "normal" db paradigm. Your app includes the client lib, which connects to the running db process "somewhere else". I don't know why they did this (easier to include?), but it meant that the "db-in-your-app-process" value prop went away. Initially when they deployed Datamic Cloud (a sort of managed Datamic service deployed in AWS), they only supported the client model. That is context for your question. The problem Ions solves is how to get that "db-in-your-app-process" value prop while using Datamic Cloud. It seems to me that it effectively reverses the original method. Original method was add the DB as a lib into your app, Ions method is push your app into the db (which is running in a configured-for-you cluster on AWS). (It also allows you to not have to provision/configure servers to actually run your app code in addition to the db servers, but that seems secondary to me.) Note, that I'm not affiliated with Cognitect, and I've never used Datamic, or Clojure in a production system. So have a salt shaker.

bjconlan 8y ago

It seems Rich (yes and the team at cognitect) might have jumped the shark on this one. (Or this was released too early and the next conf. talk will clear up what the problem is and what it is that this is attempting to solve). Either way it's rare for Rich not to describe to problem a little more before throwing a solution to it. Hopefully more will come to light in time. The brief take away I see from this as a non AWS deployment based user is the ability to go beyond the current limitations of only having access to core library functions in the context of datomic. (I see that deployment of applications also plays a part in this but not sure what that solves exactly)