3 ms·

How would you build out at a metrics cluster that needs 1M writes/sec? 285 nodes that are easily automated to create/destroy/monitor doesn't seem like a manage

How would you build out at a metrics cluster that needs 1M writes/sec? 285 nodes that are easily automated to create/destroy/monitor doesn't seem like a management pain to me, personally. Just depends if you have the write internal tools built.

arrryarr 12y ago
Ok, so you're super-admin. And about the ROI aspect? What's the cost for achieving the 1M write/sec metric in that mammoth cluster?
- opendais 12y ago
  I asked you how you would architect something that needs 1M writes/sec if you weren't using Cassandra. Ignore the fact that netflix is making this into a single cluster. I'm not saying I'd do it by myself. I'm saying with the write tools its doable and not unreasonable.
Retric 12y ago
I have seen single servers do 250k writes/second. So it really depends on the data and access patterns more than some arbitrary one size fit's all solution.
- rsynnott 12y ago
  I suspect systems where the dataset, or at least the indexes, largely fit in memory?
  - Retric 12y ago
    Most of the active data-set fit in ram, but considering you can get 100+GB of RAM that's generally not much of an issue. Also a single mid range SSD can break 120K writes per second so 0+1 RAID arrays can get crazy fast for less money than you might think.
    - rsynnott 12y ago
      Well, I mean, you _can_ get 100+GB of RAM, but you'll certainly pay for it. The machines used in the example have 30GB of RAM and 800GB SSD. For a fairly random access pattern and most of the storage used, the RAM's not going to be helping that much. Let's say they're using 600GB of the 800GB, replication factor of three, that's 57TB over their cluster. It's pretty big.
      - Retric 12y ago
        Uhh, you can get 128GB on a basic dell server that costs less than 4k. It's 512+GB of ram that starts getting pricey now days and 1TB is actually an option. Though, I agree if you dataset is large enough and you need random access it's not going to help much.

arrryarr 12y ago

Ok, so you're super-admin. And about the ROI aspect? What's the cost for achieving the 1M write/sec metric in that mammoth cluster?

opendais 12y ago
I asked you how you would architect something that needs 1M writes/sec if you weren't using Cassandra. Ignore the fact that netflix is making this into a single cluster. I'm not saying I'd do it by myself. I'm saying with the write tools its doable and not unreasonable.

Retric 12y ago

I have seen single servers do 250k writes/second. So it really depends on the data and access patterns more than some arbitrary one size fit's all solution.

rsynnott 12y ago
I suspect systems where the dataset, or at least the indexes, largely fit in memory?
- Retric 12y ago
  Most of the active data-set fit in ram, but considering you can get 100+GB of RAM that's generally not much of an issue. Also a single mid range SSD can break 120K writes per second so 0+1 RAID arrays can get crazy fast for less money than you might think.
  - rsynnott 12y ago
    Well, I mean, you _can_ get 100+GB of RAM, but you'll certainly pay for it. The machines used in the example have 30GB of RAM and 800GB SSD. For a fairly random access pattern and most of the storage used, the RAM's not going to be helping that much. Let's say they're using 600GB of the 800GB, replication factor of three, that's 57TB over their cluster. It's pretty big.
    - Retric 12y ago
      Uhh, you can get 128GB on a basic dell server that costs less than 4k. It's 512+GB of ram that starts getting pricey now days and 1TB is actually an option. Though, I agree if you dataset is large enough and you need random access it's not going to help much.