12 ms·
Datomic Cloud
- dh-g 9y agoThis looks great, I really like datomic but its licensing is preventative for small projects. Hope this helps it get more love and community attention.
- fulafel 9y agoCan you expand on this, what do you find wanting with the various choices? "Small project" can mean very different things between 0 and 10M$ budgets, but there seem to be a lot of options reaching to the low end, even the free Starter edition sounds pretty capable.
- ledgerdev 9y agoFor me a small project would generally mean a clojure web app with a couple hundred users, a clojure admin webapp used only occasionally, and a back end cron job that runs every 5 minutes or so and does the back end work. Any rough guesses what we are looking at to run this monthly using the Solo topology on aws?
- stuarthalloway 9y agoThat should fit easily in the Solo Topology at about $1/day.
- ledgerdev 9y agoYes, but I was wondering how much memory would be needed for each of the other apps (web/admin/cron) connecting to the Solo. Would they require a full JVM peer, with a couple gig of memory each, right?
- stuarthalloway 9y agoCloud uses clients, not peers, so those apps can be much more lightweight. I have tested down to 128M per Clojure (JVM) client, which is an order of magnitude less mem than a typical peer.
- gonewest 9y agoSuppose you have an open source or hobby project. The Free Edition has limitations that impact how you would write code -- it supports only the "peer" API and not the "client" API, and is restricted in terms of backend storage choices and number of peers. So while it's very likely true there's perfect API compatibility when you want to step up to a paid edition, it makes you design to the artificial limitations they've used to set apart "free" from "paid". Indeed, it's also not the recommended way to learn. The Datomic team says: "If you are trying Datomic for the first time, we recommend that you begin with a client library." In other words, the Free Edition doesn't include the client library that they recommend you try first. If you step up to the Starter Edition you have full functionality and free updates but only for the first year. Beyond that, you start paying. Either you run in Cloud (as low as $1/day) or you purchase the on-prem Pro Edition ($5000/year). Don't get me wrong. Cognitect has the right to charge whatever they like, and bundle features however they like. It's their intellectual property and they made the investment to create it. But if you're building a hobby project then even $1/day may be more than you want to spend, and/or more than you want to impose on outside contributors to your codebase.
- fulafel 9y agoSeems it's only the free updates that are limited to 1 year with Starter - you can keep using it forever. I wonder how/if they limit you getting a new Starter the next year? Anyone know the details? Even if this drives you to the $1/day edition, that should cover a lot of small projects. Especially as you don't have to keep it running 24/7 during development.
- dh-g 9y ago@gonewest covered why the starter isn't ideal. I have played with the starter but compared to when I worked for a company that did have a license its like a different product. This makes it hard to select it for small projects because you know you'd have to pay the 5k or switch to something else if you ever wanted to scale. 1$ a day I think is still a bit high for a non-profit driven side project, but definitely an welcome improvement.
- cls59 9y agoWhat if the deployment model of my "small project" is on-prem operation by end users? How do I evaluate the usefulness of Datomic without forcing those users into a license purchase or going through the up-front cost of buying a license that allows redistribution?
- deleted 9y ago[deleted]
- puredanger 9y agoThis is the mobile link, better link is http://blog.datomic.com/2018/01/datomic-cloud.html http://blog.datomic.com/2018/01/datomic-cloud.html
- dominotw 9y agoIs there a way to play with this without using a credit card?
- bpicolo 9y agohttps://www.datomic.com/get-datomic.html https://www.datomic.com/get-datomic.html starter version https://github.com/tonsky/datascript https://github.com/tonsky/datascript Datascript is also neat, it's an in-browser DB with datalog query syntax. There's some neat stuff possible with om.next + datascript: https://github.com/omcljs/om/wiki/DataScript-Integration-Tutorial https://github.com/omcljs/om/wiki/DataScript-Integration-Tut...
- dominotw 9y agoThank you.
- auggierose 9y agoHow do things like cloud databases work with respect to the GDPR?
- grzm 9y agoLooks like that's addressed: > "The regulation applies if the data controller (an organization that collects data from EU residents) or processor (an organization that processes data on behalf of data controller e.g. cloud service providers) or the data subject (person) is based in the EU." https://en.wikipedia.org/wiki/General_Data_Protection_Regulation#Scope https://en.wikipedia.org/wiki/General_Data_Protection_Regula... (However, this is off-topic for this submission specifically.)
- j_s 9y ago> (However, this is off-topic for this submission specifically.) I think it's on-topic for a database that (in my barely informed understanding) is famous for never deleting anything!
- grzm 9y agoThis is explicitly supported in Datomic via an operation known as "excision": > "Excision is the complete removal of a set of datoms matching a predicate. Excision should be a very infrequent operation, and is designed to support the following two scenarios:" > "- removing data for privacy reasons" > "- removing data older than some domain-defined retention period" > "Excision should never be used to correct erroneous data, and is unsuitable for that task as it does not restore any previous view of the facts. Consider using ordinary retraction to correct errors without removing history." https://docs.datomic.com/on-prem/excision.html https://docs.datomic.com/on-prem/excision.html I haven't read whether or not there's a difference here with respect to the cloud offering, as I wasn't able to find a corresponding Reference section in the cloud docs. However, given that :db/excise is a transaction operation, it's likely supported. Edit to add: Looks like it currently isn't, see 'davidrupp below. That's important to keep in mind.
- deleted 9y ago
- deleted 9y ago[deleted]
- danscan 9y agoThis is awesome! Whenever I'm working with MySQL I'm usually daydreaming about using Datomic instead. I'd love to see more adoption in the community.
- kgwxd 9y agoI really wish they'd open source Datomic and just make money on hosting. I can't bring myself to spend anymore of my life learning proprietary software in such a foundational domain.
- bringtheaction 9y agoSpeaking of open source Datomic, check out mentat. It's not at all the same as an open source Datomic, because it's made for use on single devices not clusters, but it's inspired by Datomic so at the same time it is very much like Datomic. https://github.com/mozilla/mentat https://github.com/mozilla/mentat See also this blog post for some background and history: https://www.ncalexander.net/blog/2017/05/31/tofino-data-storage-and-how-we-got-to-mentat/ https://www.ncalexander.net/blog/2017/05/31/tofino-data-stor...
- samuell 9y agoThanks, added to my WIP "awesome datalog" :) https://github.com/samuell/awesome-datalog https://github.com/samuell/awesome-datalog
- jasonkostempski 9y ago"The first version of Project Mentat, named Datomish" For some reason, that almost made me spit my drink. This looks cool, I'll check it out.
- systems 9y agothey wont ... or at least not any time soon if you search online they did address this repeated request in summary they do it for money, they dont know how to do it for money if its open-source, they already contributed clojure to the open source community someone already was vocally aggressive about this in a blog: http://z.caudate.me/on-whose-authority/ http://z.caudate.me/on-whose-authority/ and rich hickey didnt like it: https://www.reddit.com/r/Clojure/comments/73yznc/on_whose_authority/do1olag/ https://www.reddit.com/r/Clojure/comments/73yznc/on_whose_au... -- My personal take on this its not that datomic is closed source or that it cost a lot of money its that for its cost, its a valid option for fewer people i use ms sql at work, and it cost a lot of money but .. you can easily make the business case for it for datomic ... it would be much harder datomic is a niche product .. which is not bad .. it is just where its at
- bontaq 9y agoI'm not sure it's still true, but awhile ago their EULA banned the publication of performance statistics. Be very careful using this and do your own performance testing.
- _sdegutis 9y agoI found it really fun to use for 3 years in production. But in retrospect I think Postgres is probably a better fit for what we needed.
- dominotw 9y ago> But in retrospect I think Postgres is probably a better fit for what we needed. Would love to hear more about this.
- lgrapenthin 9y agoIf after three years you think so only "probably", you probably still aren't confident about your requirements. You don't say to your girlfriend "We had fun the last three years, but probably I'm better off with my ex" unless you have no clue what you are looking for.
- ExactoKnight 9y agoI'm using Postgres for a financial app, and the idea of never updating a value in place sounds very appealing over PostgreSQL's construct of updating in place. I'd love to hear your thoughts about how using Datomic in this immutable sense plays out in practice, because Rich Hickey's talk in favour of it is compelling. https://www.infoq.com/presentations/Datomic-Database-Value https://www.infoq.com/presentations/Datomic-Database-Value
- augustz 9y agoStill true based on what I see: Restrictions: The Licensee hereby agrees, without the prior written consent of Cognitect, which may be withheld or conditioned at Cognitect’s sole discretion, it will not:... j) publicly display or communicate the results of internal performance testing or other benchmarking or performance evaluation of the Software; This is a similar style I believe to the Oracle licensing model which used to prevent similar stuff I think. Building your stuff on a platform (proprietary) with this attitude towards licensing... I'd say run away... fast.
- steelbird 9y agoIs $1 a day a good price?
- andrewchambers 9y agoIf you have a product which makes $2 a day using it then it is.
- fulafel 9y agoGreat milestone, congratulations to the Datomic team! Seems the developer forums (https://forum.datomic.com/ https://forum.datomic.com/) are a good place to visit for practicalities after the web site.
- ekzy 9y agoThe forum is fairly recent and not everyone is checking it... You could get help on the clojurians slack in the #datomic channel
- pmarreck 9y agoAre there any open-source immutable databases yet? Or would the hegemony of SQL semantics (which seem quite tied to mutability) somehow disincentivize such an undertaking?
- mhluongo 9y agoIIRC Datomic is open-source, just not freely licensed? Bitcoin is also definitely an open-source immutable database ;) EDIT: Maybe not, wow.
- pmarreck 9y agoHi Matt :) Yeah, doesn't Bitcoin use BerkeleyDB underneath or something? Superficially it's immutable (when just API'ing to the blockchain) but underneath it's backed by BerkeleyDB I believe, which is your everyday mutable sql store
- ExactoKnight 9y agoCitation dramatically needed. Maybe for querying read values of unspent transaction balances from blockchain ledgers, but the data integrity where it matters is on the chain where people can't screw with it...
- lvh 9y agoFSVO "database": datascript is in-memory, similar to a slimmed down version of datomic. I think it's fair to say that datascript:datomic :: sqlite:postgres or something. It's in-memory, but there are third-party extensions using hitchiker trees that make it persistent. There's also Mozilla's mentat and its predecessor atomish which talk to SQLite on the backend. So, sorta?
- pmarreck 9y agoDid not know about Datascript and Mentat. Thanks!
- noahdesu 9y ago
- gregwebs 9y agoThis is a big change in pricing approach. Previously the licensing was $5k/year and still is for on-prem. Although you could run datomic for free for 1 year and longer as long as you never upgraded it. https://www.datomic.com/get-datomic.html https://www.datomic.com/get-datomic.html Now they are offering an approach where one can at least start at $1/day (Every time you scale out this goes up? What is the cost of support?). The cheapest way to get started is probably still to run what is now called "on-prem AWS" (for up to 1 year), which there is already cloudformation for. https://docs.datomic.com/on-prem/aws.html https://docs.datomic.com/on-prem/aws.html
- stuarthalloway 9y agoCloud is definitely the preferred way to get started if you are on AWS. When running the On-Prem version you also have to manage (and pay for) storage. Cloud also puts less load on DynamoDB, which can dominate your spend.
- nickik 9y agoI love Datomic, it solves so many problems that I face daily and you end up building a sort-off datomic on top of your SQL DB anyway. I would have prefered a Datomic for Kubernetes to get this cross-platform and allow it to be hosted on different and private clouds because of my job I can not use AWS. I can't use Kubernetes either but there is at future possiblity that it would be possible. The downside would probebly be that it would offer less integration as the ecosystem is not so far along as AWS. Still an easly containerised version for devlopment would be really nice. Thanks to the Datomic team.
- danpeddle 9y agoThere's a few people asking about how to get something up quickly to play with - this was a problem for me, and it isn't really well explained, but for those interested, Alexander Kiel's datomic free docker image has worked well for me - https://hub.docker.com/r/akiel/datomic-free/ https://hub.docker.com/r/akiel/datomic-free/ - and an example of it being used with docker compose is here: https://github.com/dazld/urlomic/blob/cc3d1218218b5f08751dfad41eabe68ceabeed7d/docker-compose.yml https://github.com/dazld/urlomic/blob/cc3d1218218b5f08751dfa... (on a wip branch for some playground thing). I'm far from a pro, but hope this helps someone interested in this tech.
- stuarthalloway 9y agoDatomic Cloud makes it much easier to get started on AWS. It takes only a few minutes to sign up, see https://www.datomic.com/videos.html https://www.datomic.com/videos.html for a walkthrough.
- danpeddle 9y agoFor sure! looking forward to becoming a customer. I guess the journey I went through was missing some intermediate step between playing at the repl with an in memory db (which I believe you’ve mentioned before as being a good way to get hands dirty from zero), to having something in dev which was backed by storage, to a toy service, to production. Jumping into AWS, at least in my work environment, is a big jump requiring lots of supporting infrastructure etc. We’ll get there if it works out, but the dockerized setup is really helping in learning and evaluation. Officially supported ways of doing this is something i’m sure you’ve considered before, it would be nice if cognitect could reconsider this.
- stuarthalloway 9y agoYou don't have to be very far "in" AWS to try Cloud. The bastion server will let you connect from anywhere, so the idea is that the only infrastructure you need is a laptop and an internet connection. That said, we understand the value of a local development mode and that is possible in the future.
- reilly3000 9y agoI’ve been in love with the idea of Datomic for a few years now but haven’t had a project to really dig into it with. The AMI makes it irresistible to try... but I’m really not clear on how the client and peer libraries work outside of the JVM. Do they? It seems REST support exists with some client libs for js, Python, etc but they explicitly state in multiple places that it is defacto deprecated and won’t receive further development. I highly doubt given many of Rich’s statements that they would ever break the API, but I’m trying to push a new app towards serverless, and I’d rather not have to mess around JVM keep-warm hacks. A CLJS library that wasn’t orphaned 4 years ago and not based on the REST api would be enough to push me over the edge.
- datahead 9y agoFor anyone interested in background on Datomic, these are fun and informative resources. The talk where I realized Rich is a data head too. Introducing his design goals and life experiences with db(s). https://www.infoq.com/presentations/The-Design-of-Datomic https://www.infoq.com/presentations/The-Design-of-Datomic "Clojure for the Brave and True" author, Dan Higginbotham, on key themes and overview. http://www.flyingmachinestudios.com/programming/datomic-for-five-year-olds/ http://www.flyingmachinestudios.com/programming/datomic-for-... Congrats to the Datomic team and Cognitect. I hope this move to cloud opens Datomic's design principals and ideas to a wider user base.