5 ms·

Exactly. "...this isn't a database" :/ Distributed caches aren't new either. [e.g. https://github.com/golang/groupcache https://github.com/golang/groupcache ]

Exactly. "...this isn't a database" :/ Distributed caches aren't new either. [e.g. https://github.com/golang/groupcache https://github.com/golang/groupcache ] Can we get a title change to something that labels it a Distributed Cache? [e.g. Shown HN: gun – Massively Distributed Cache ]

marknadal 13y ago
opendais, I'm curious if these other distributed caches actually support persistence? I wasn't able to find such a system. But I'm limited in my research, I'd love to learn more about other people's work! Please share more!
- opendais 13y ago
  http://ehcache.org/ http://ehcache.org/ http://ehcache.org/documentation/configuration/fast-restart#strategy-options http://ehcache.org/documentation/configuration/fast-restart#... Based on a quick googling, this looks like it would be an example of that. Frankly, I don't normally want my cache to persist so it isn't something I know much about. [e.g. I store persistent data in Cassandra or MySQL]
  - marknadal 13y ago
    You hit on a really good point, in the several years that I've worked in the synchronization field (I do consulting) I've found that it has actually been harder to deal with /what you don't want to sync/ than with what you want to sync. It would be really bad if our temporary variables that are used on the client and server to do transformations on our actual data got somehow jumbled up into our real data. I have some clean delineations for this, but wow - I didn't know that this was going to explode so quickly. If I had known, I would have prepared some more resources and materials to explain this stuff. That said, everybody who signs up on the mailing list will get notified about the posts I will write on this! I'm looking forward to getting some serious counter-arguments/refutations to challenge gun.
    - danbruc 13y ago
      I would like to hear about the way you address the semantics of the data for merging. For example if the number of items in stock has a conflict because one node reports a 500 -> 1000 change and another node reports a 500 -> 400 change, then I probably want the result to be 900. If the same conflict occurs for the budget of something, then I probably want either 400 or 1000 but not 900.
      - marknadal 13y ago
        I'm not entirely familiar with your example cases, but the seem to be indicative of something where you would want strong consistency. By default, gun is only eventually consistent - but this does not mean it is incapable of strong consistency, it just means you have to write/use a plugin that abstracts this on top. Obviously such a merge would be slow, because you are giving up high availability while you sit locked waiting for ACKs from other nodes. Could you expound on your example problems more so I can address them more thoroughly? Point being, that is the beauty of eventually consistent systems, you can always combine the together to create stronger forms of consistency - but you can't go the other way. If you start with a strongly consistent system, you can't downgrade. With gun, strong consistency is not given to you out of the box - but it is capable of being built on top when needed.
- robterrell 13y ago
  Reminds me of running pouchdb in the browser and couchdb on the servers. Except for the automatic conflict resolution bit, which of course is where the real magic is. > It gets data synchronization and conflict resolution right > from the beginning, so it never has to rely on vulnerable > leader election or consensus locking Super curious about how this works.
  - marknadal 13y ago
    People are demanding answers to this the most, so it'll probably be my first follow-up post.
  - daleharvey 13y ago
    Just as a note, conflict resolution is fairly close to automatic in PouchDB, its a matter of listening to the changes feed and providing a function which combines conflicts into a seperate document, in future versions we will make this a lot more seamless. That aside, reading this it does sound similiar to PouchDB, we are also working on automated provisioning of servers (https://github.com/daleharvey/janus https://github.com/daleharvey/janus). Excited that more people are working on and becoming aware of similiar infrastructure.
- DEinspanjer 13y ago
  Also GridGain and my personal favorite, Hazelcast.

marknadal 13y ago

opendais, I'm curious if these other distributed caches actually support persistence? I wasn't able to find such a system. But I'm limited in my research, I'd love to learn more about other people's work! Please share more!

opendais 13y ago
http://ehcache.org/ http://ehcache.org/ http://ehcache.org/documentation/configuration/fast-restart#strategy-options http://ehcache.org/documentation/configuration/fast-restart#... Based on a quick googling, this looks like it would be an example of that. Frankly, I don't normally want my cache to persist so it isn't something I know much about. [e.g. I store persistent data in Cassandra or MySQL]
- marknadal 13y ago
  You hit on a really good point, in the several years that I've worked in the synchronization field (I do consulting) I've found that it has actually been harder to deal with /what you don't want to sync/ than with what you want to sync. It would be really bad if our temporary variables that are used on the client and server to do transformations on our actual data got somehow jumbled up into our real data. I have some clean delineations for this, but wow - I didn't know that this was going to explode so quickly. If I had known, I would have prepared some more resources and materials to explain this stuff. That said, everybody who signs up on the mailing list will get notified about the posts I will write on this! I'm looking forward to getting some serious counter-arguments/refutations to challenge gun.
  - danbruc 13y ago
    I would like to hear about the way you address the semantics of the data for merging. For example if the number of items in stock has a conflict because one node reports a 500 -> 1000 change and another node reports a 500 -> 400 change, then I probably want the result to be 900. If the same conflict occurs for the budget of something, then I probably want either 400 or 1000 but not 900.
    - marknadal 13y ago
      I'm not entirely familiar with your example cases, but the seem to be indicative of something where you would want strong consistency. By default, gun is only eventually consistent - but this does not mean it is incapable of strong consistency, it just means you have to write/use a plugin that abstracts this on top. Obviously such a merge would be slow, because you are giving up high availability while you sit locked waiting for ACKs from other nodes. Could you expound on your example problems more so I can address them more thoroughly? Point being, that is the beauty of eventually consistent systems, you can always combine the together to create stronger forms of consistency - but you can't go the other way. If you start with a strongly consistent system, you can't downgrade. With gun, strong consistency is not given to you out of the box - but it is capable of being built on top when needed.
robterrell 13y ago
Reminds me of running pouchdb in the browser and couchdb on the servers. Except for the automatic conflict resolution bit, which of course is where the real magic is. > It gets data synchronization and conflict resolution right > from the beginning, so it never has to rely on vulnerable > leader election or consensus locking Super curious about how this works.
- marknadal 13y ago
  People are demanding answers to this the most, so it'll probably be my first follow-up post.
- daleharvey 13y ago
  Just as a note, conflict resolution is fairly close to automatic in PouchDB, its a matter of listening to the changes feed and providing a function which combines conflicts into a seperate document, in future versions we will make this a lot more seamless. That aside, reading this it does sound similiar to PouchDB, we are also working on automated provisioning of servers (https://github.com/daleharvey/janus https://github.com/daleharvey/janus). Excited that more people are working on and becoming aware of similiar infrastructure.
DEinspanjer 13y ago
Also GridGain and my personal favorite, Hazelcast.