6 ms·

Impressive, but I don't get why folks don't normalize it to events per second instead. It is 463k eps for anyone wondering.

jeffbee 5y ago
And the nameplate capacity - up to 1GB per second - is easily within the nominal abilities of one single machine.
- bob1029 5y ago
  I love reading these threads and thinking this sort of stuff to myself... How much of twitter's infrastructure is self-perpetuated or otherwise exists just to serve other infrastructure? You obviously cant fit everything their business does on a single machine, but you could probably get dangerously close with enough determination. Especially, if you just focused on the core 280 character tweet abstraction. An options order consists of more data than an average tweet, and we can certainly process them at a higher rate than twitter would need to go in practice. Many financial exchanges operate on a single thread. 1-100 million transactions per second with jitter measured in tens of microseconds. I don't see why other software products & services can't leverage similar concepts.
  - abraxas 5y ago
    Yep, probably a single good mainframe would handle all of it with a smile. And provide the uptime that most "cloud" vendors can dream of. But that wouldn't be nowhere near as cool as building "cloud infra", right?
  - saagarjha 5y ago
    There’s certainly lots of room for improvement in this kind of system, but I think it’s a bit reductionist to compare this kind of work to HFT, which hits one hot path that is ultra optimized and doesn’t have to deal with complexities like “this data needs to be replicated and kept consistent”. Your question is a bit like “planes go fast, why can’t my car use some of the same tricks to go fast as well?”: yes, your car can probably go faster, and it could probably steal some aerodynamics and material science from your plane, but reducing the problem to something simple that they should “just” do is absurd.
    - bob1029 5y ago
      > doesn’t have to deal with complexities like “this data needs to be replicated and kept consistent” I'm sure there are other excuses available, but this is not a good one. Non-repudiation is a critical system requirement of any financial exchange. The figures stated include persistence to durable media, and the whole point of running everything on a single thread is to ensure serialized processing of all activities (i.e. consistency). I would argue the need for replicating tweets is less urgent than ensuring 7-8 figure financial transactions don't go unaccounted for. We could probably make some compromises for the twitter use case to make this even faster.
- saagarjha 5y ago
  1 GB/s is on the order of how fast a computer can parse requests. Throw in processing on top of that and you’ll slow down considerably.
da39a3ee 5y ago
Isn't it 10x that? Sanity check: say 10^5 seconds/day. Then it would be 400 * 10^9 / 10^5 = 4 * 10^6 https://www.google.com/search?q=400+*+10%5E9+%2F+%2824+*+60+*+60%29 https://www.google.com/search?q=400+*+10%5E9+%2F+%2824+*+60+...
- nikhilsimha 5y ago
  You are right. 4.63 million eps. My bad.

jeffbee 5y ago

And the nameplate capacity - up to 1GB per second - is easily within the nominal abilities of one single machine.

bob1029 5y ago
I love reading these threads and thinking this sort of stuff to myself... How much of twitter's infrastructure is self-perpetuated or otherwise exists just to serve other infrastructure? You obviously cant fit everything their business does on a single machine, but you could probably get dangerously close with enough determination. Especially, if you just focused on the core 280 character tweet abstraction. An options order consists of more data than an average tweet, and we can certainly process them at a higher rate than twitter would need to go in practice. Many financial exchanges operate on a single thread. 1-100 million transactions per second with jitter measured in tens of microseconds. I don't see why other software products & services can't leverage similar concepts.
- abraxas 5y ago
  Yep, probably a single good mainframe would handle all of it with a smile. And provide the uptime that most "cloud" vendors can dream of. But that wouldn't be nowhere near as cool as building "cloud infra", right?
- saagarjha 5y ago
  There’s certainly lots of room for improvement in this kind of system, but I think it’s a bit reductionist to compare this kind of work to HFT, which hits one hot path that is ultra optimized and doesn’t have to deal with complexities like “this data needs to be replicated and kept consistent”. Your question is a bit like “planes go fast, why can’t my car use some of the same tricks to go fast as well?”: yes, your car can probably go faster, and it could probably steal some aerodynamics and material science from your plane, but reducing the problem to something simple that they should “just” do is absurd.
  - bob1029 5y ago
    > doesn’t have to deal with complexities like “this data needs to be replicated and kept consistent” I'm sure there are other excuses available, but this is not a good one. Non-repudiation is a critical system requirement of any financial exchange. The figures stated include persistence to durable media, and the whole point of running everything on a single thread is to ensure serialized processing of all activities (i.e. consistency). I would argue the need for replicating tweets is less urgent than ensuring 7-8 figure financial transactions don't go unaccounted for. We could probably make some compromises for the twitter use case to make this even faster.
saagarjha 5y ago
1 GB/s is on the order of how fast a computer can parse requests. Throw in processing on top of that and you’ll slow down considerably.

da39a3ee 5y ago

Isn't it 10x that? Sanity check: say 10^5 seconds/day. Then it would be 400 * 10^9 / 10^5 = 4 * 10^6 https://www.google.com/search?q=400+*+10%5E9+%2F+%2824+*+60+*+60%29 https://www.google.com/search?q=400+*+10%5E9+%2F+%2824+*+60+...

nikhilsimha 5y ago
You are right. 4.63 million eps. My bad.