6 ms·

Riak 1.4 Released

88 points by moonboots 13y ago

http://basho.com/basho-announces-availability-of-riak-1-4/

lazyloop 13y ago
Have the weak defaults that caused 91% data loss when recovering from network partitioning been fixed yet? http://aphyr.com/posts/285-call-me-maybe-riak http://aphyr.com/posts/285-call-me-maybe-riak
- strmpnk 13y ago
  Last write wins is certainly a horrible default behavior. It seems like Basho has worked on a CRDT cookbook to make it easier to deal with conflict resolution. With that in place for critical mutable values, it's a lot easier to swallow the immutable values approach for the rest of your data (similar to Datomic), which completely sidesteps this problem.
  - macintux 13y ago
    (Disclaimer: I work for Basho.) Last write wins is indeed a bad default for data integrity; I believe we received feedback early on from customers that siblings were too confusing for developers. The good news is that Kyle was deliberately testing things that were effectively guaranteed to break, thus the high rate of data loss. Most applications won't be simultaneously writing the same keys on both sides of a network partition; there will typically be a healthier mix of new and old keys being written.
    - strmpnk 13y ago
      Definitely all true. I don't mind putting pressure on Basho for better defaults though. Maybe that will be a 2.0 milestone. Knowing how other databases compared in similar partition tests, Riak is doing just fine.
- harryh 13y ago
  I don't think you understood aphyr's post. You might want to try reading it again.
makmanalp 13y ago
As a longtime follower, let me say, nice work guys! It's amazing to see how far Riak has come. I have a question. > Secondary Indexing Improvements: Query results are now sorted and paginated, offering developers much richer semantics So the results can be sorted but are not stored in sorted order in the secondary indices? > Introducing Counters in Riak: Counters, Riak’s first distributed data type, provide automatic conflict resolution after a network partition Awesome! P.S. more info here: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md
- bonzoesc 13y ago
  Secondary Indexing Improvements: Query results are now sorted and paginated, offering developers much richer semantics So the results can be sorted but are not stored in sorted order in the secondary indices? Secondary indexes are stored sorted on disk, but segmented per vnode. Previously, they wouldn't be sorted before being returned to a client. Suppose you're querying for "Bananas" through "Bavaria"; the node that contains "Bassoon" could return its first result before the node containing "Bananas" and "Barons", which would, in old Riak versions, result in out-of-order results. Disclosure: I work at Basho, and have been working on riak-ruby-client updates to support the 2i improvements.
  - cinbun8 13y ago
    Sorting used to be done through M/R queries. Does Riak perform a M/R sort on the results from each node from this release ? Just wondering how this sort is achieved and what it means for performance, if it is requested.
    - bonzoesc 13y ago
      It does a merge sort, and I can't see that you can opt out of it: https://github.com/basho/riak_kv/blob/master/src/sms.erl https://github.com/basho/riak_kv/blob/master/src/sms.erl 2i doesn't use mapreduce in normal operation.
  - jhancock 13y ago
fosk 13y ago
I have been looking through http://docs.basho.com/riak/latest/ http://docs.basho.com/riak/latest/ but I can't find any doc about the counters.
- pharkmillups 13y ago
  We're working on expanding the official counters docs. We'll also have a blog post up about it next week. Basho Hacker Sam Elliott started a CRDT cookbook that will walk you through using the counters in 1.4 (and eventually the new data types we add). That should get you started. https://github.com/lenary/riak_crdt_cookbook/tree/counter_examples/counters https://github.com/lenary/riak_crdt_cookbook/tree/counter_ex... Enjoy. Also, come to ricon.io/west.html. We'll be talking plenty about Counters and all the huge stuff coming to Riak in the near future...

88 points by moonboots 13y ago

http://basho.com/basho-announces-availability-of-riak-1-4/

Have the weak defaults that caused 91% data loss when recovering from network partitioning been fixed yet? http://aphyr.com/posts/285-call-me-maybe-riak http://aphyr.com/posts/285-call-me-maybe-riak

strmpnk 13y ago
Last write wins is certainly a horrible default behavior. It seems like Basho has worked on a CRDT cookbook to make it easier to deal with conflict resolution. With that in place for critical mutable values, it's a lot easier to swallow the immutable values approach for the rest of your data (similar to Datomic), which completely sidesteps this problem.
- macintux 13y ago
  (Disclaimer: I work for Basho.) Last write wins is indeed a bad default for data integrity; I believe we received feedback early on from customers that siblings were too confusing for developers. The good news is that Kyle was deliberately testing things that were effectively guaranteed to break, thus the high rate of data loss. Most applications won't be simultaneously writing the same keys on both sides of a network partition; there will typically be a healthier mix of new and old keys being written.
  - strmpnk 13y ago
    Definitely all true. I don't mind putting pressure on Basho for better defaults though. Maybe that will be a 2.0 milestone. Knowing how other databases compared in similar partition tests, Riak is doing just fine.
harryh 13y ago
I don't think you understood aphyr's post. You might want to try reading it again.

makmanalp 13y ago

As a longtime follower, let me say, nice work guys! It's amazing to see how far Riak has come. I have a question. > Secondary Indexing Improvements: Query results are now sorted and paginated, offering developers much richer semantics So the results can be sorted but are not stored in sorted order in the secondary indices? > Introducing Counters in Riak: Counters, Riak’s first distributed data type, provide automatic conflict resolution after a network partition Awesome! P.S. more info here: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md

bonzoesc 13y ago
Secondary Indexing Improvements: Query results are now sorted and paginated, offering developers much richer semantics So the results can be sorted but are not stored in sorted order in the secondary indices? Secondary indexes are stored sorted on disk, but segmented per vnode. Previously, they wouldn't be sorted before being returned to a client. Suppose you're querying for "Bananas" through "Bavaria"; the node that contains "Bassoon" could return its first result before the node containing "Bananas" and "Barons", which would, in old Riak versions, result in out-of-order results. Disclosure: I work at Basho, and have been working on riak-ruby-client updates to support the 2i improvements.
- cinbun8 13y ago
  Sorting used to be done through M/R queries. Does Riak perform a M/R sort on the results from each node from this release ? Just wondering how this sort is achieved and what it means for performance, if it is requested.
  - bonzoesc 13y ago
    It does a merge sort, and I can't see that you can opt out of it: https://github.com/basho/riak_kv/blob/master/src/sms.erl https://github.com/basho/riak_kv/blob/master/src/sms.erl 2i doesn't use mapreduce in normal operation.
- jhancock 13y ago

fosk 13y ago

I have been looking through http://docs.basho.com/riak/latest/ http://docs.basho.com/riak/latest/ but I can't find any doc about the counters.

pharkmillups 13y ago
We're working on expanding the official counters docs. We'll also have a blog post up about it next week. Basho Hacker Sam Elliott started a CRDT cookbook that will walk you through using the counters in 1.4 (and eventually the new data types we add). That should get you started. https://github.com/lenary/riak_crdt_cookbook/tree/counter_examples/counters https://github.com/lenary/riak_crdt_cookbook/tree/counter_ex... Enjoy. Also, come to ricon.io/west.html. We'll be talking plenty about Counters and all the huge stuff coming to Riak in the near future...