6 ms·

I'm not sure it's still true, but awhile ago their EULA banned the publication of performance statistics. Be very careful using this and do your own performanc

by bontaq 9y ago

I'm not sure it's still true, but awhile ago their EULA banned the publication of performance statistics. Be very careful using this and do your own performance testing.

_sdegutis 9y ago
I found it really fun to use for 3 years in production. But in retrospect I think Postgres is probably a better fit for what we needed.
- dominotw 9y ago
  > But in retrospect I think Postgres is probably a better fit for what we needed. Would love to hear more about this.
- lgrapenthin 9y ago
  If after three years you think so only "probably", you probably still aren't confident about your requirements. You don't say to your girlfriend "We had fun the last three years, but probably I'm better off with my ex" unless you have no clue what you are looking for.
- ExactoKnight 9y ago
  I'm using Postgres for a financial app, and the idea of never updating a value in place sounds very appealing over PostgreSQL's construct of updating in place. I'd love to hear your thoughts about how using Datomic in this immutable sense plays out in practice, because Rich Hickey's talk in favour of it is compelling. https://www.infoq.com/presentations/Datomic-Database-Value https://www.infoq.com/presentations/Datomic-Database-Value
augustz 9y ago
Still true based on what I see: Restrictions: The Licensee hereby agrees, without the prior written consent of Cognitect, which may be withheld or conditioned at Cognitect’s sole discretion, it will not:... j) publicly display or communicate the results of internal performance testing or other benchmarking or performance evaluation of the Software; This is a similar style I believe to the Oracle licensing model which used to prevent similar stuff I think. Building your stuff on a platform (proprietary) with this attitude towards licensing... I'd say run away... fast.
- cle 9y ago
  Performance testing is a fickle art and publicizing misleading or non-generalizable results can be extremely damaging to a small business like this, even if the results are not representative nor applicable to most people.
  - dragandj 9y ago
    Then Datomic should at least release their own, properly done and fair, benchmarks.
    - mschaef 9y ago
      You say that like it's an easy thing to do. For a relatively novel data model with potentially unusual performance characteristics, the chance of finding a properly done, fair, and generally understandable benchmark is virtually impossible.
      - fulafel 9y ago
        I think the model from the user point of view has been seen many times over the years. See eg https://en.wikipedia.org/wiki/List_of_SPARQL_implementations https://en.wikipedia.org/wiki/List_of_SPARQL_implementations https://en.wikipedia.org/wiki/Datalog#Systems_implementing_Datalog https://en.wikipedia.org/wiki/Datalog#Systems_implementing_D... Also the various RDF things from the days of semantic web enthusiasm. For active competitors, maybe the various GraphQL things and
  - adambyrtek 9y ago
    Then they should constructively respond to these misleading results not suppress publication with legal measures.
    - cle 9y ago
      A damaged reputation is hard to repair, if even possible. As a small business, the choice seems to be pretty straightforward--pay multiple salaries to do PR, damage control, reputation management, etc. (and compete with huge corporations with deep pockets)...or pay those same salaries to engineers to build a better product for your customers, and get your customers to promise not to create PR headaches for you.
vemv 9y ago
I recently read that a lot of DBs do this, or at least have done so historically. Might be like patents - being more of a defense mechanism, rather than an actual device intended to be used against customers.
jay-anderson 9y ago
https://danluu.com/anon-benchmark/ https://danluu.com/anon-benchmark/ - "That time Larry Ellison allegedly tried to have a professor fired for benchmarking Oracle" - Looks like other database vendors do the thing. That doesn't excuse the behavior though.
- ExactoKnight 9y ago
  Then people using PostgreSQL remember how their database is free, rather than a $100,000 a year despite having more flexibility and features, and you see why Larry Ellison would do this. !@#$ Oracle...
spinlock 9y ago
My experience with Datomic was that it is not appropriate for a system that has performance requirements. We developed a system using redis while the data team was developing the data service on Datomic. When we tried to integrate, the whole system slowed to a crawl and we needed to do a rewrite. My advice is to develop with Datomic from the start and not separate it out into a service. Any other database will be so much faster that you won't know how bad your performance is until it's too late. Another piece of advice would be to seriously consider if you need immutability in your database. If it's not a hard requirement, I would not use Datomic.
- lgrapenthin 9y ago
  Having built a variety of products with Datomic and performance requirements I can only disagree. Especially queries are insanely fast compared to other databases. I don't know what your data team was doing, but they must have done it wrong. My advice is to develop without a data team from the start ;)
  - nas 9y ago
    Datomic is interesting. It is somewhat similar to object databases like ZODB. I use an object database called Durus (spiritual derivative of ZODB). Both ZODB and Datomic make use of aggressive caching in the application. That is a huge performance boost for applications that do a lot of reading and little writing. ZODB essentially puts the indexes in the application, rather than having the DB layer take care of indexes. Again, I understand this is similar to how Datomic works. Developing an application on top of ZODB vs on top of an SQL database leads to a very different data model design. You can't just swap one database layer out for the other and expect things to work well. I'm not surprised if a team who is used to developing on top of SQL-like DBs did not have good luck when moving to Datomic. It is just a very different model.
  - qaq 9y ago
    would you give an example?
    - 9y ago
sethev 9y ago
Having used products that bragged about their performance benchmarks (won't name any names), I respect the Datomic team's position on this. Benchmarks lead to gamed results and you still have to test with your workload.
- bontaq 9y ago
  Indeed, I just think the ban on it makes certain things harder to find before you're using it in a production system (with proper production loads). Pagination and sorting, for example, do not exist in the usual sense. https://groups.google.com/forum/#!topic/datomic/gr9fscuF6oE https://groups.google.com/forum/#!topic/datomic/gr9fscuF6oE https://groups.google.com/forum/#!topic/datomic/NgVviV9Sw8g https://groups.google.com/forum/#!topic/datomic/NgVviV9Sw8g It'd be swell if they put up a list of differences, because using the pull query API + clojure really is quite cool -- it's just good to know what you're getting into.
datahead 9y ago
Our team signed a contract with Cognitect for Datomic late 2017. This clause was in-place and stood out to me as well. I had a chance to ask their legal team about it. The clause is written in legal-ese, which always sounds overbearing. I asked the question in the positive sense, "what if have some really nice metrics from our use cases, and want to talk about them at a conference?" They simply asked to be consulted and request written permission to share. The intent, like others have noted, is to request (legally: insist) that Cognitect have a chance to review and point out potential implementation issues (good or bad) prior to customers making performance statements about their product. The clause can/does put a damper on 'notes from the field' reports, which often help when deciding on tech direction.I look for community based reports to reinforce perceptions of a tool (to a degree). Completely agree with OP, do your own performance testing. One thing I will say is that it would be hard for someone who hasn't invested in learning the inner-workings of Datomic's decoupled architecture to pick apart storage speed vs. transactor speed. For example, storage speed (SQL, DEV, Dynamo, etc.) is not a concern of Datomic, but a key dependency to measurable perf. This may change in the AWS service announced today, and become more uniform on dynamo and S3 "storage resources". https://docs.datomic.com/cloud/whatis/architecture.html#storage-resources https://docs.datomic.com/cloud/whatis/architecture.html#stor... Datomic is a unique product and there are many ways to make it sing (or blow up) depending on how you use it. We designed data models, streaming processes, and queries with Datomic in mind and have had success. Exactly how much success, I'm not at liberty to say just yet.

by bontaq 9y ago

I'm not sure it's still true, but awhile ago their EULA banned the publication of performance statistics. Be very careful using this and do your own performance testing.

_sdegutis 9y ago

I found it really fun to use for 3 years in production. But in retrospect I think Postgres is probably a better fit for what we needed.

dominotw 9y ago
> But in retrospect I think Postgres is probably a better fit for what we needed. Would love to hear more about this.
lgrapenthin 9y ago
If after three years you think so only "probably", you probably still aren't confident about your requirements. You don't say to your girlfriend "We had fun the last three years, but probably I'm better off with my ex" unless you have no clue what you are looking for.
ExactoKnight 9y ago
I'm using Postgres for a financial app, and the idea of never updating a value in place sounds very appealing over PostgreSQL's construct of updating in place. I'd love to hear your thoughts about how using Datomic in this immutable sense plays out in practice, because Rich Hickey's talk in favour of it is compelling. https://www.infoq.com/presentations/Datomic-Database-Value https://www.infoq.com/presentations/Datomic-Database-Value

augustz 9y ago

Still true based on what I see: Restrictions: The Licensee hereby agrees, without the prior written consent of Cognitect, which may be withheld or conditioned at Cognitect’s sole discretion, it will not:... j) publicly display or communicate the results of internal performance testing or other benchmarking or performance evaluation of the Software; This is a similar style I believe to the Oracle licensing model which used to prevent similar stuff I think. Building your stuff on a platform (proprietary) with this attitude towards licensing... I'd say run away... fast.

cle 9y ago
Performance testing is a fickle art and publicizing misleading or non-generalizable results can be extremely damaging to a small business like this, even if the results are not representative nor applicable to most people.
- dragandj 9y ago
  Then Datomic should at least release their own, properly done and fair, benchmarks.
  - mschaef 9y ago
    You say that like it's an easy thing to do. For a relatively novel data model with potentially unusual performance characteristics, the chance of finding a properly done, fair, and generally understandable benchmark is virtually impossible.
    - fulafel 9y ago
      I think the model from the user point of view has been seen many times over the years. See eg https://en.wikipedia.org/wiki/List_of_SPARQL_implementations https://en.wikipedia.org/wiki/List_of_SPARQL_implementations https://en.wikipedia.org/wiki/Datalog#Systems_implementing_Datalog https://en.wikipedia.org/wiki/Datalog#Systems_implementing_D... Also the various RDF things from the days of semantic web enthusiasm. For active competitors, maybe the various GraphQL things and
- adambyrtek 9y ago
  Then they should constructively respond to these misleading results not suppress publication with legal measures.
  - cle 9y ago
    A damaged reputation is hard to repair, if even possible. As a small business, the choice seems to be pretty straightforward--pay multiple salaries to do PR, damage control, reputation management, etc. (and compete with huge corporations with deep pockets)...or pay those same salaries to engineers to build a better product for your customers, and get your customers to promise not to create PR headaches for you.

vemv 9y ago

I recently read that a lot of DBs do this, or at least have done so historically. Might be like patents - being more of a defense mechanism, rather than an actual device intended to be used against customers.

jay-anderson 9y ago

https://danluu.com/anon-benchmark/ https://danluu.com/anon-benchmark/ - "That time Larry Ellison allegedly tried to have a professor fired for benchmarking Oracle" - Looks like other database vendors do the thing. That doesn't excuse the behavior though.

ExactoKnight 9y ago
Then people using PostgreSQL remember how their database is free, rather than a $100,000 a year despite having more flexibility and features, and you see why Larry Ellison would do this. !@#$ Oracle...

spinlock 9y ago

My experience with Datomic was that it is not appropriate for a system that has performance requirements. We developed a system using redis while the data team was developing the data service on Datomic. When we tried to integrate, the whole system slowed to a crawl and we needed to do a rewrite. My advice is to develop with Datomic from the start and not separate it out into a service. Any other database will be so much faster that you won't know how bad your performance is until it's too late. Another piece of advice would be to seriously consider if you need immutability in your database. If it's not a hard requirement, I would not use Datomic.

lgrapenthin 9y ago
Having built a variety of products with Datomic and performance requirements I can only disagree. Especially queries are insanely fast compared to other databases. I don't know what your data team was doing, but they must have done it wrong. My advice is to develop without a data team from the start ;)
- nas 9y ago
  Datomic is interesting. It is somewhat similar to object databases like ZODB. I use an object database called Durus (spiritual derivative of ZODB). Both ZODB and Datomic make use of aggressive caching in the application. That is a huge performance boost for applications that do a lot of reading and little writing. ZODB essentially puts the indexes in the application, rather than having the DB layer take care of indexes. Again, I understand this is similar to how Datomic works. Developing an application on top of ZODB vs on top of an SQL database leads to a very different data model design. You can't just swap one database layer out for the other and expect things to work well. I'm not surprised if a team who is used to developing on top of SQL-like DBs did not have good luck when moving to Datomic. It is just a very different model.
- qaq 9y ago
  would you give an example?
  - 9y ago

sethev 9y ago

Having used products that bragged about their performance benchmarks (won't name any names), I respect the Datomic team's position on this. Benchmarks lead to gamed results and you still have to test with your workload.

bontaq 9y ago
Indeed, I just think the ban on it makes certain things harder to find before you're using it in a production system (with proper production loads). Pagination and sorting, for example, do not exist in the usual sense. https://groups.google.com/forum/#!topic/datomic/gr9fscuF6oE https://groups.google.com/forum/#!topic/datomic/gr9fscuF6oE https://groups.google.com/forum/#!topic/datomic/NgVviV9Sw8g https://groups.google.com/forum/#!topic/datomic/NgVviV9Sw8g It'd be swell if they put up a list of differences, because using the pull query API + clojure really is quite cool -- it's just good to know what you're getting into.

datahead 9y ago

Our team signed a contract with Cognitect for Datomic late 2017. This clause was in-place and stood out to me as well. I had a chance to ask their legal team about it. The clause is written in legal-ese, which always sounds overbearing. I asked the question in the positive sense, "what if have some really nice metrics from our use cases, and want to talk about them at a conference?" They simply asked to be consulted and request written permission to share. The intent, like others have noted, is to request (legally: insist) that Cognitect have a chance to review and point out potential implementation issues (good or bad) prior to customers making performance statements about their product. The clause can/does put a damper on 'notes from the field' reports, which often help when deciding on tech direction.I look for community based reports to reinforce perceptions of a tool (to a degree). Completely agree with OP, do your own performance testing. One thing I will say is that it would be hard for someone who hasn't invested in learning the inner-workings of Datomic's decoupled architecture to pick apart storage speed vs. transactor speed. For example, storage speed (SQL, DEV, Dynamo, etc.) is not a concern of Datomic, but a key dependency to measurable perf. This may change in the AWS service announced today, and become more uniform on dynamo and S3 "storage resources". https://docs.datomic.com/cloud/whatis/architecture.html#storage-resources https://docs.datomic.com/cloud/whatis/architecture.html#stor... Datomic is a unique product and there are many ways to make it sing (or blow up) depending on how you use it. We designed data models, streaming processes, and queries with Datomic in mind and have had success. Exactly how much success, I'm not at liberty to say just yet.