6 ms·
I find this post interesting because working with Cassandra requires truly different patterns compared to standard RDBMSs and even other NoSql systems. I worke
by dwenzek 11y ago
I find this post interesting because working with Cassandra requires truly
different patterns compared to standard RDBMSs and even other NoSql systems.
I worked recently on a project where we used Cassandra with so many surprises,
often learned the hard way with bugs in production,
that the team felt discouraged more than once
before we succeed to setup a reliable, efficient and scalable system.
Much more than said in the post,
this is really tricky in practice to get eventual consistency and data modeling right.
Not only because the concepts are new or hard to grasp.
But rather because Cassandra fails on the principle of least surprise.
Two features, which are working fine in isolation,
may behave in unexpected way once combined.
- Getting consistency right is complex, because the actual consistency you get
is the result of consistency levels scattered along all the queries.
- Denormalization works fine at first,
but quickly reaches a point where the right combination of columns is missing.
- Having a large count of columns is fine, till you try to delete some and get tombstones.
- Tombstones are automatically removed during compactions,
unless there are too many tombstones leading to memory exhaustion.
- The joining of a node may be aborted in case of a long GC pause,
due to a greedy data model or too many tombstones.
- We even experienced a Cassandra bug - counters were corrupted by a new node joining the cluster
(this has been fixed with version 2.0)
In other words, many aspects are deeply tied together
- data modeling, query coding and system tuning.
This makes things really complex to grasp.
What worked for us, was to design in team.
This gave us maximum opportunities to share experience,
to explore the solution space,
to discuss pros and cons
and to design right.