3 ms·

Incidently, on EC2 the extra cpu in an XL or HMXL makes all the difference between "oh shit, we ran out of capacity and bootstrapping more slows things to a cra

by jbellis 16y ago

Incidently, on EC2 the extra cpu in an XL or HMXL makes all the difference between "oh shit, we ran out of capacity and bootstrapping more slows things to a crawl" and "we ran out of capacity but we can limp along during the bootstrap." YYMV. Reddit was originally on L instances. (HMXL is generally the sweet spot for Cassandra price/performance on EC2, IMO.)

puredemo 16y ago
I can't believe they were originally on L instances. I assumed they chose HMXL considering their pageviews..
- jedberg 16y ago
  That's not quite correct. Our original Cassandra nodes were on XLs. Only the new ones were on Ls, and only temporarily.
jfager 16y ago
Is it normal for people to run Cassandra on only 3 nodes in production environments? I realize that one of the selling points is starting small and then scaling out without any headaches, but 3 nodes seems extreme in that regard.
- jbellis 16y ago
  Well, roughly speaking, you can group Cassandra deployments in two categories+: new products that are hoping they need Cassandra's scaling ability someday, and existing products moving to Cassandra from something else because the pain of scaling something that wasn't designed to makes them. The first category will start small. The smallest production deployment I know is a single 256MB VM, but usually even starting small you should not have less than 2 servers (why tempt fate with a non-redundant setup when Cassandra makes it so easy to be safe?). The second group is where you see larger deployments out of the gate. 3 machines does seem small for reddit; I guess they made up the difference with memcached. Unfortunately they deployed just before the 0.6.0 final release was out, which is where we added the row cache feature that could have made memcached unnecessary. +There are people using cassandra for non-scale-related reasons, though. Most of these people are motivated by multi-datacenter replication.
jedberg 16y ago
That's not quite correct. Our original Cassandra nodes were on XLs. Only the new ones were on Ls, and that was because we couldn't get XLs during the outage. Later in the outage Amazon helped us get the XLs we needed.
- jbellis 16y ago
  Thanks for the correction.

by jbellis 16y ago

puredemo 16y ago

I can't believe they were originally on L instances. I assumed they chose HMXL considering their pageviews..

jedberg 16y ago
That's not quite correct. Our original Cassandra nodes were on XLs. Only the new ones were on Ls, and only temporarily.

jfager 16y ago

Is it normal for people to run Cassandra on only 3 nodes in production environments? I realize that one of the selling points is starting small and then scaling out without any headaches, but 3 nodes seems extreme in that regard.

jbellis 16y ago
Well, roughly speaking, you can group Cassandra deployments in two categories+: new products that are hoping they need Cassandra's scaling ability someday, and existing products moving to Cassandra from something else because the pain of scaling something that wasn't designed to makes them. The first category will start small. The smallest production deployment I know is a single 256MB VM, but usually even starting small you should not have less than 2 servers (why tempt fate with a non-redundant setup when Cassandra makes it so easy to be safe?). The second group is where you see larger deployments out of the gate. 3 machines does seem small for reddit; I guess they made up the difference with memcached. Unfortunately they deployed just before the 0.6.0 final release was out, which is where we added the row cache feature that could have made memcached unnecessary. +There are people using cassandra for non-scale-related reasons, though. Most of these people are motivated by multi-datacenter replication.

jedberg 16y ago

That's not quite correct. Our original Cassandra nodes were on XLs. Only the new ones were on Ls, and that was because we couldn't get XLs during the outage. Later in the outage Amazon helped us get the XLs we needed.

jbellis 16y ago
Thanks for the correction.