4 ms·

Could someone tell me the disadvantages of solving the problem in the article using S3? I am actually curious.

by nitred 6y ago

redis_mlc 6y ago
They're inserting 5 billion events per second (for some reason), and AWS throttles everything (the following link mentions a limit of 3,500/second): https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance/ https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3... Also, if they're planning to do alerting, S3 would not help with that. PUTs are 1 cent per 2000: https://aws.amazon.com/s3/pricing/ https://aws.amazon.com/s3/pricing/
- buremba 6y ago
  5 billion events per second? That seems to be too much for monitoring data even at Twitter's scale. The article refers that they store the raw data and use rollup tables for aggregate queries, I wonder how much the ELT process costs in total.
  - redis_mlc 6y ago
    > 5 billion events per second? When you work for a large company, you have 2 problems: 1) Every has-been manager bikesheds over metrics retention 2) you can't say no. See #1.
  - dan-robertson 6y ago
    Indeed, the article mentions 5b metrics per minute
- deepakhj 6y ago
  You write to S3 in chunks for long term storage and keep hot data live. You don’t write every metric directly. Along with compression It shouldn’t be that 5 billion/s.
  - RhodesianHunter 6y ago
    Even with chunks it's prohibitively expensive at this scale, and you then need some intermediate system batching the inbound data either to disk (then why use S3 instead of a database) or in memory (more expensive, more risk of data loss).
RhodesianHunter 6y ago
Metrics on S3 at scale? It's about the worst tool you could use for this task. The cost of puts alone would kill you.
riyadparvez 6y ago
Off topic, doesn't Twitter use their own data centers and GCP? When did they start using AWS?

4 ms·

Could someone tell me the disadvantages of solving the problem in the article using S3? I am actually curious.

by nitred 6y ago

Could someone tell me the disadvantages of solving the problem in the article using S3? I am actually curious.

redis_mlc 6y ago
They're inserting 5 billion events per second (for some reason), and AWS throttles everything (the following link mentions a limit of 3,500/second): https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance/ https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3... Also, if they're planning to do alerting, S3 would not help with that. PUTs are 1 cent per 2000: https://aws.amazon.com/s3/pricing/ https://aws.amazon.com/s3/pricing/
- buremba 6y ago
  5 billion events per second? That seems to be too much for monitoring data even at Twitter's scale. The article refers that they store the raw data and use rollup tables for aggregate queries, I wonder how much the ELT process costs in total.
  - redis_mlc 6y ago
    > 5 billion events per second? When you work for a large company, you have 2 problems: 1) Every has-been manager bikesheds over metrics retention 2) you can't say no. See #1.
  - dan-robertson 6y ago
    Indeed, the article mentions 5b metrics per minute
- deepakhj 6y ago
  You write to S3 in chunks for long term storage and keep hot data live. You don’t write every metric directly. Along with compression It shouldn’t be that 5 billion/s.
  - RhodesianHunter 6y ago
    Even with chunks it's prohibitively expensive at this scale, and you then need some intermediate system batching the inbound data either to disk (then why use S3 instead of a database) or in memory (more expensive, more risk of data loss).
RhodesianHunter 6y ago
Metrics on S3 at scale? It's about the worst tool you could use for this task. The cost of puts alone would kill you.
riyadparvez 6y ago
Off topic, doesn't Twitter use their own data centers and GCP? When did they start using AWS?