4 ms·

This has that "too good to be true" vibe, and I can't find much information on the authors or the Syncfree Consortium organization that backs the project beside

This has that "too good to be true" vibe, and I can't find much information on the authors or the Syncfree Consortium organization that backs the project besides their own website. Is this at the cost of fast writes or flexible schema? The pitch video doesn't seem to mention any cons, yet seems to avoid mentioning the type of data or mutations supported. I guess I'll go read their publications.

gjem97 9y ago
I think the issue you're looking for is a limited number of supported datatypes. This doesn't appear to be a general-purpose database at the moment: don't expect to drop it in as a replacement for MySQL. Doesn't mean it can't be in the future, but it would have to fall back on other techniques for the datatypes that can't be built with CRDTs. http://syncfree.github.io/antidote/crdts.html http://syncfree.github.io/antidote/crdts.html
- lopatin 9y ago
  I think the data types look fine. If I'm using such a database, I don't think I want the CRDT data types to be abstracted away behind some general-purpose (SQL?) interface. Can that even be done? I'd rather just pay the small price of restructuring my data into maps and sets and have the CRDT database do the rest without any extra indirection.
  - jpdsousa 9y ago
    Well, i've been working on an SQL interface for AntidoteDB for my master thesis, and yes, it can be done. However, there are some limitations. At this point we are mostly focusing in constraints (i.e. invariant preservation), and we've already figured out stuff like numeric bounds, entity integrity and referential integrity (this last one is quite interesting IMO). Github: https://github.com/JPDSousa/AQL https://github.com/JPDSousa/AQL
dustingetz 9y ago
I don't think it encodes a graph that you query (like, say, Neo4J or Postgres with an orm); it seems to give various replicated data structure primitives, like documents, basically a kv store? So for building something like google docs.
- nine_k 9y ago
  One glaring omission is indexes. You can likely build something SQL-like to form complex queries coordinating related objects, but the efficiency of such queries is not a given. You can likely manually build something like an index to speed up lookups using e.g. the map primitive. With a plan for a complex query, you're on your own. Another likely problematic part is PK generation. You can of course use a counter, but I'm not sure how efficient will it be with burst inserts. For that, you'd have to come up with client-generated PKs that are unlikely to clash, e.g. UUIDs.
  - anne_biene 9y ago
    (one of the maintainers of Antidote) There is a lot of research still going on in the back, including indexing, access control, verification tools for apps, and some other really cool stuff. Efficient support for queries is non-trivial, indeed.
KirinDave 9y ago
It's a government sponsored project so one would expect their publicity to be limited. That's part of the reason I posted it. Stuff like this is extremely exciting. It's at the cost of playing by the rules of CRDTs. Making CRDTs consequence-free is ongoing research.
- teleclimber 9y ago
  > Making CRDTs consequence-free is ongoing research. What do you mean by that? I found possibly related language on this page the other day [0]: CRDTs are "[t]ypically not suited for editing application with consequent UI." Can you point me in the right direction? My googling got me nowhere. Thanks. [0] https://irisate.com/collaborative-editing-solutions-round-up/ https://irisate.com/collaborative-editing-solutions-round-up...
  - KirinDave 9y ago
    I dunno about that quote, Treedoc works pretty well. In general there are tradeoffs to CRDTs and not everyone loves those tradeoffs. As time goes on we find CRDTs that make better tradeoffs. For example, the earliest examples of CRDT sets grew linearly with the values added to the set, without regard for deletion. For all time. That's a pretty steep cost.
    - teleclimber 9y ago
      I get it, thank-you.
    - bpicolo 9y ago
      LSEQ appears to improve upon treedoc in ways, too: https://hal.archives-ouvertes.fr/hal-00921633/document https://hal.archives-ouvertes.fr/hal-00921633/document
  - Horusiath 9y ago
    It's not described in the docs, but elswhere [1] in their github repository you can find the note about rga, which is Replicated Growable Array CRDT, that can be used for building indexed linear sequences - shortly speaking arrays of elements with insert/remove semantics. [1](https://github.com/SyncFree/antidote/blob/842874ca5ecffe947e75fa6a67c1ad0d9c6ccb87/docs/pages/mydoc/rawapi.md https://github.com/SyncFree/antidote/blob/842874ca5ecffe947e...)
marc_shapiro 9y ago
A write is fast, because it happens directly at the closest replica, without any inter-replica synchronisation. There is no schema per se; rather the DB is object-oriented, and each application picks the object types it needs from the CRDT library. The library currently covers the basics: registers, flags, sets, maps, lists.

gjem97 9y ago

I think the issue you're looking for is a limited number of supported datatypes. This doesn't appear to be a general-purpose database at the moment: don't expect to drop it in as a replacement for MySQL. Doesn't mean it can't be in the future, but it would have to fall back on other techniques for the datatypes that can't be built with CRDTs. http://syncfree.github.io/antidote/crdts.html http://syncfree.github.io/antidote/crdts.html

lopatin 9y ago
I think the data types look fine. If I'm using such a database, I don't think I want the CRDT data types to be abstracted away behind some general-purpose (SQL?) interface. Can that even be done? I'd rather just pay the small price of restructuring my data into maps and sets and have the CRDT database do the rest without any extra indirection.
- jpdsousa 9y ago
  Well, i've been working on an SQL interface for AntidoteDB for my master thesis, and yes, it can be done. However, there are some limitations. At this point we are mostly focusing in constraints (i.e. invariant preservation), and we've already figured out stuff like numeric bounds, entity integrity and referential integrity (this last one is quite interesting IMO). Github: https://github.com/JPDSousa/AQL https://github.com/JPDSousa/AQL

dustingetz 9y ago

I don't think it encodes a graph that you query (like, say, Neo4J or Postgres with an orm); it seems to give various replicated data structure primitives, like documents, basically a kv store? So for building something like google docs.

nine_k 9y ago
One glaring omission is indexes. You can likely build something SQL-like to form complex queries coordinating related objects, but the efficiency of such queries is not a given. You can likely manually build something like an index to speed up lookups using e.g. the map primitive. With a plan for a complex query, you're on your own. Another likely problematic part is PK generation. You can of course use a counter, but I'm not sure how efficient will it be with burst inserts. For that, you'd have to come up with client-generated PKs that are unlikely to clash, e.g. UUIDs.
- anne_biene 9y ago
  (one of the maintainers of Antidote) There is a lot of research still going on in the back, including indexing, access control, verification tools for apps, and some other really cool stuff. Efficient support for queries is non-trivial, indeed.

KirinDave 9y ago

It's a government sponsored project so one would expect their publicity to be limited. That's part of the reason I posted it. Stuff like this is extremely exciting. It's at the cost of playing by the rules of CRDTs. Making CRDTs consequence-free is ongoing research.

teleclimber 9y ago
> Making CRDTs consequence-free is ongoing research. What do you mean by that? I found possibly related language on this page the other day [0]: CRDTs are "[t]ypically not suited for editing application with consequent UI." Can you point me in the right direction? My googling got me nowhere. Thanks. [0] https://irisate.com/collaborative-editing-solutions-round-up/ https://irisate.com/collaborative-editing-solutions-round-up...
- KirinDave 9y ago
  I dunno about that quote, Treedoc works pretty well. In general there are tradeoffs to CRDTs and not everyone loves those tradeoffs. As time goes on we find CRDTs that make better tradeoffs. For example, the earliest examples of CRDT sets grew linearly with the values added to the set, without regard for deletion. For all time. That's a pretty steep cost.
  - teleclimber 9y ago
    I get it, thank-you.
  - bpicolo 9y ago
    LSEQ appears to improve upon treedoc in ways, too: https://hal.archives-ouvertes.fr/hal-00921633/document https://hal.archives-ouvertes.fr/hal-00921633/document
- Horusiath 9y ago
  It's not described in the docs, but elswhere [1] in their github repository you can find the note about rga, which is Replicated Growable Array CRDT, that can be used for building indexed linear sequences - shortly speaking arrays of elements with insert/remove semantics. [1](https://github.com/SyncFree/antidote/blob/842874ca5ecffe947e75fa6a67c1ad0d9c6ccb87/docs/pages/mydoc/rawapi.md https://github.com/SyncFree/antidote/blob/842874ca5ecffe947e...)

marc_shapiro 9y ago

A write is fast, because it happens directly at the closest replica, without any inter-replica synchronisation. There is no schema per se; rather the DB is object-oriented, and each application picks the object types it needs from the CRDT library. The library currently covers the basics: registers, flags, sets, maps, lists.