4 ms·

6B a month =~ 2k per second. Just putting the numbers in perspective :-)

by cdman 12y ago

iampims 12y ago
That's assuming a linear distribution of clicks and we all know it's never the case. It could well be over 10K sustained for peak hours. Averages are often misleading :)
- ryanjshaw 12y ago
  Without wanting to detract from the engineering accomplishments here (which I have never come close to), it's important to note that low-latency does not appear to be a design criteria i.e. it's okay if it takes a couple minutes to process events during peak load, which means there is some leeway for smoothing the input peaks over processing time. Furthermore, these are just click events. It's okay to lose a few, so the design doesn't have to be especially good at making sure events aren't lost, as far as I can tell. A good design is as much about what is left out as what is left in, which is the lesson between the lines here in my opinion.
  - fizwhiz 12y ago
    My thoughts exactly. If you come to my house and ring the bell, but it takes me 15mins to get off the couch to reach the door, I can't really claim to be "available" with a straight face. Sure, I "technically" am available, but that level of latency is not practical.
    - i0exception 12y ago
      The system is still available if your ring gets an acknowledgement of receipt. The latency for the request to be served is a design metric and has no impact on "availability"
      - fizwhiz 12y ago
        >> The system is still available if your ring gets an acknowledgement of receipt This is the equivalent of (as per my example) yelling out "I hear ya! Coming..." and then take 15mins to reach the door. I never said that low latency implies anything about the level of availability; I merely meant that arguing about the availability of systems is incomplete without a thorough discussion of latency. In the case of Bitly, I'm just curoius about the systems that are highly available but "require" low latency vs systems that don't require it. As ryanjshaw points out, the system may have a degree of tolerance for lossy click events. If you have a heterogeneous mix of systems with different tolerance levels, that surely affects the architecture does it not?
  - josephlord 12y ago
    They say that the clicks and shortened URLs are handled synchronously. It is the recording of the clicks for analysis that is done asynchronously.

6B a month =~ 2k per second. Just putting the numbers in perspective :-)

by cdman 12y ago

6B a month =~ 2k per second. Just putting the numbers in perspective :-)

That's assuming a linear distribution of clicks and we all know it's never the case. It could well be over 10K sustained for peak hours. Averages are often misleading :)

ryanjshaw 12y ago
Without wanting to detract from the engineering accomplishments here (which I have never come close to), it's important to note that low-latency does not appear to be a design criteria i.e. it's okay if it takes a couple minutes to process events during peak load, which means there is some leeway for smoothing the input peaks over processing time. Furthermore, these are just click events. It's okay to lose a few, so the design doesn't have to be especially good at making sure events aren't lost, as far as I can tell. A good design is as much about what is left out as what is left in, which is the lesson between the lines here in my opinion.
- fizwhiz 12y ago
  My thoughts exactly. If you come to my house and ring the bell, but it takes me 15mins to get off the couch to reach the door, I can't really claim to be "available" with a straight face. Sure, I "technically" am available, but that level of latency is not practical.
  - i0exception 12y ago
    The system is still available if your ring gets an acknowledgement of receipt. The latency for the request to be served is a design metric and has no impact on "availability"
    - fizwhiz 12y ago
      >> The system is still available if your ring gets an acknowledgement of receipt This is the equivalent of (as per my example) yelling out "I hear ya! Coming..." and then take 15mins to reach the door. I never said that low latency implies anything about the level of availability; I merely meant that arguing about the availability of systems is incomplete without a thorough discussion of latency. In the case of Bitly, I'm just curoius about the systems that are highly available but "require" low latency vs systems that don't require it. As ryanjshaw points out, the system may have a degree of tolerance for lossy click events. If you have a heterogeneous mix of systems with different tolerance levels, that surely affects the architecture does it not?
- josephlord 12y ago
  They say that the clicks and shortened URLs are handled synchronously. It is the recording of the clicks for analysis that is done asynchronously.