3 ms·

-1. With hindsight it's easy to make a comment like yours, but many clients rely on the status_id to be monotonic. You would be changing behavior in a way that

by morgo 16y ago

-1. With hindsight it's easy to make a comment like yours, but many clients rely on the status_id to be monotonic. You would be changing behavior in a way that breaks apps. Twitter doesn't want to do that.

beagle3 16y ago
-1 all you like. It's engineering, not hindsight -- when building a system, I always ask myself "how does this scale" which usually translates to "on what attribute does this shard". Look at e.g. YouTube and many other sites around the same time. They knew what they were doing; Twitter didn't. I _have_ actually designed such a system in 1999, that used 48 bits, and it worked perfectly well. (Only had 28 bits for the user-id, which would have been broken at the 250M users -- alas the system never had more than than 5M; This was in the years 1999-2003). The only way you can shard absolutely monotonic is (effectively) randomly, which is an option however you assign ids; but other assignments let you build a much cheaper, much more robust system.
- nsfmc 16y ago
  I am wiser than any god or scientist, for I have squared the circle and cubed Earth's sphere, thus I have created 4 simultaneous separate 24 hour days within a 4-corner (as in a 4-corner classroom) rotation of Earth. See for yourself the absolute proof!
- morgo 16y ago
  I don't see how your comparison is remotely fair to Twitter. Youtube is just a website, if they wanted to change how they allocate video numbers, nobody except for rogue bots will notice. Twitter is an API. Even if they have the knowledge of how to fix past mistakes, they need to ask for feedback, give plenty of notice, and set a deadline of when the old version is cut off. Maybe they didn't do everything perfect day one, but I don't think there are any APIs the size of Twitter. Cut them some slack, they're not morons.
  - beagle3 16y ago
    The point was exactly that YouTube, which had a simpler problem to solve at around the same time (or earlier) did it properly. There do exist independent youtube clients (though not as many as Twitters's), but using the encoding they did, youtube has made it so that it is never going to be an issue, whereas for twitter it has already been a significant issue twice (that I'm aware of). It's very easy to dismiss sound engineering in retrospect as luck or as "how could anyone have known".

by morgo 16y ago

beagle3 16y ago

-1 all you like. It's engineering, not hindsight -- when building a system, I always ask myself "how does this scale" which usually translates to "on what attribute does this shard". Look at e.g. YouTube and many other sites around the same time. They knew what they were doing; Twitter didn't. I _have_ actually designed such a system in 1999, that used 48 bits, and it worked perfectly well. (Only had 28 bits for the user-id, which would have been broken at the 250M users -- alas the system never had more than than 5M; This was in the years 1999-2003). The only way you can shard absolutely monotonic is (effectively) randomly, which is an option however you assign ids; but other assignments let you build a much cheaper, much more robust system.

nsfmc 16y ago
I am wiser than any god or scientist, for I have squared the circle and cubed Earth's sphere, thus I have created 4 simultaneous separate 24 hour days within a 4-corner (as in a 4-corner classroom) rotation of Earth. See for yourself the absolute proof!
morgo 16y ago
I don't see how your comparison is remotely fair to Twitter. Youtube is just a website, if they wanted to change how they allocate video numbers, nobody except for rogue bots will notice. Twitter is an API. Even if they have the knowledge of how to fix past mistakes, they need to ask for feedback, give plenty of notice, and set a deadline of when the old version is cut off. Maybe they didn't do everything perfect day one, but I don't think there are any APIs the size of Twitter. Cut them some slack, they're not morons.
- beagle3 16y ago
  The point was exactly that YouTube, which had a simpler problem to solve at around the same time (or earlier) did it properly. There do exist independent youtube clients (though not as many as Twitters's), but using the encoding they did, youtube has made it so that it is never going to be an issue, whereas for twitter it has already been a significant issue twice (that I'm aware of). It's very easy to dismiss sound engineering in retrospect as luck or as "how could anyone have known".