7 ms·
All Hell May Break Loose On Twitter In 2 Hours
- icey 17y agoI think they've made a good choice by forcing the issue to happen now instead of arbitrarily occurring in the middle of the night on a weekend. The Friday afternoon part sucks though. If they were going to do this, they should have done it yesterday.
- jackowayed 17y agoYeah, they also should have told people more than 4 hours ahead of time.
- cmgarcia 17y agoAt least we received the mandatory Twitter response that they know they made a mistake in letting us know so late and will try to improve on that in the future.
- Sam_Odio 17y agoThe title's slightly sensationalist. "Hell" will not break loose, although some poorly written 3rd party apps might fail.
- jonknee 17y ago"Hopefully Twitter will be able to resolve this quickly." Gotta love tech writers and their incompetence.
- deleted 17y ago[deleted]
- dkarl 17y agoI think the writer is under the impression that Twitter is using a 32-bit signed counter and will start using negative ids when it rolls over. Evidently some people on the linked Twitter development thread had the same misconception. Actually it looks like Twitter is doing it right: they're just going to keep counting up past the maximum 32-bit signed int (after skipping a bunch of ids to move the Twitpocalypse up into business hours.)
- mattyb 17y agohttp://twitter.com/twitterapi/status/2048659057 http://twitter.com/twitterapi/status/2048659057
- nostrademons 17y agoIt sounded like the problem wasn't Twitter, but 3rd party Twitter clients. They have no way of knowing if these use a 32-bit or 64-bit ID field, so if Twitter sends back an ID that can't fit in 32 bits - what happens? Do they crash? Wrap? Take only the low-order 32 bits?
- thaumaturgy 17y agoYeah, there were a couple of real gems in that "article": > It’s possibly a coincidence, but Twitter has just welcomed two new members to its API team today. Is Twitter manning up for the battle? Right, because as a company, what you really want to do if a big hammer is about to fall on you is immediately hire a couple of new programmers who aren't familiar with your internals at all. > It’s now past 2 PM and no reports of massive failures yet. Perhaps this really is just like Y2K. Yep, because there's no such thing as a bunch of programmers who spent a lot of hours fixing code _before_ it broke. (I was one of them.)
- larrywright 17y agoIt always irritates me when people say "Y2K was such a non-event". It was a non-event because a whole lot of people fixed and tested a whole lot of code. If they hadn't, the world would have been a pretty miserable place on 1/1/2000.
- eru 17y agoPerhaps not really miserable, but a lot of things in administration (etc) would not have continued to work. (My grandfather was also one of those fixers.)
- larrywright 17y agoDepends... the code that runs an insurance company is one thing. The embedded computers would likely have been the biggest problem - things that dispense medicine, control the power grid, etc.
- eru 17y agoFew embedded systems depend on the date, do they?
- deleted 17y ago[deleted]
- Adrenalist 17y agoI'm confused: how is "ETA: 13 Jun 2009 at 11:19:38 AM GMT" == "Friday afternoon" Isn't June 13th Saturday?
- deleted 17y ago[deleted]
- TrevorJ 17y agoI'm going to go out on a limb here and say that even if Twitter vaporized in a puff of smoke in two hours and never came back, all hell would not break loose.
- kirubakaran 17y agoTo be fair, they just said that 'all hell may break loose on Twitter', not the whole world.
- jonknee 17y agoHumorously if everyone thought hell was breaking loose they would check twitter to see if it was just them.
- wglb 17y agoAgreed for most of us--if you are on the technical staff at twitter, then such a breaking-loose scale might be what happens.
- quizbiz 17y agoWhat would hell breaking loose look like? And what is it breaking loose from? :P
- GeneralMaximus 17y agoLast week I went to a "tweetup" (yes, I'm an idiot). Believe me, there are people who would probably break down completely if something happened to their beloved Twitter. EDIT: admitted the fact that I'm an idiot.
- thwarted 17y agoSince this has been known about, literally, for months, why didn't twitter setup a mock testing environment that generated this potentially problematic case, allowing twitter app developers to test in a sandbox and release a new version before the 11th hour occurs?
- jcsalterego 17y agoI'm guessing it'd be far easier to have developers just a) check their database schemas and code, and/or b) to multiply all incoming IDs by 10, than for Twitter to setup a separate mock environment.
- jonursenbach 17y agoIt'd also be much less of a potential PR nightmare for them to just have setup a sandboxing environment.
- jcsalterego 17y agoI can't imagine the logistical nightmare of having to replicate their staging environment, with the caching and message queue systems. It probably isn't `cp -r staging/ sandbox/` :-/
- jonursenbach 17y agoThey already have an internal development environment, clone that.
- thwarted 17y agoThe mock environment doesn't need to do anything other than accept all inputs and produce known outputs (presumably some with known error conditions). This can often be implemented using apache rewrite rules/aliasing and static content served with the correct MIME types. I've set up mock servers for my own testing of public APIs when I want more control over monitoring my application than using a remote service would allow. You make a request once to the real service, store the content in a file on your own web server, and change your app to hit your server rather than the public, remote API.
- jcsalterego 17y agoThe comments in here are competent, and it's semi-interesting news (especially if you're at all involved with the Twitter ecosystem) but it's just so hard to upmod yet another sensationalist/Twitter article from TechCrunch.
- jackowayed 17y ago>Update 2: It’s now past 2 PM and no reports of massive failures yet. Perhaps this really is just like Y2K. Techcrunch, do a little homework, please. If they had actually gone to the public timeline and checked, they would have seen that the status ids are still in the 213xxxxxxx's. In other words, Twitter hasn't started the twitpocalypse yet.
- deleted 17y ago[deleted]
- ErrantX 17y agoOk I am going to TechCrunch bash. But ONLY because this is a transparent example of how their writing fails recently. The story is valid and the content of the article is fairly accurate: that the status ID's are going to trip over beyond the 32-bit integer limit which presents potential problems for third parties. Twitter sensibly pushed this forward by a few hours to make sure it happened in a "were noticing shit right now" period and thus got fixed. They announce this by tweeting a tongue in cheek "Twitapocalypse" message. The crucial problem is that TC siezed on that idea and hammered at it - yes they do mention it will only affect apps BUT - They dont mantion that it wont kill Twitter, inference from the article theoretically could go either way and I imagine many people left the page worried Twitter would crash and burn (and that the Twitterati were beavering away to "fix" things). Lines like "Hopefully Twitter will be able to resolve this quickly." emphasis this idea subtely. - They heavily suggest Twitter themselves are working on this. Whiklst im sure they are trying ot make sure app developers notice it I seriouslky doubt they have all hands on deck to "fix" an issue they themselves arent facing. They were courteous enough to push it up into working hours to give devs a chance to fix errors "live" etc. But there is nothing Twitter themselves have to do, right? All in all it's a good story written very badly - as a journalist I have to take issue with the sensationalism. It hurts the vendor and wosre it hurts the readers it dupes. :) EDIT: to be explicit. It smacks less of bad writing as deliberately badly structured writing to suck in an audience. The former is frustrating but unavoidable - the second is "evil".
- sfphotoarts 17y agohow sentimental, I love the old bugs, off by one, buffer overrun, math overflow... The good old days before garbage collection and 'number' datatypes... happy says.
- jeroen 17y agoIt just happened, about 3 hours late. Lousy planning, and if I'm not mistaking it is 5 PM Pacific/8PM Eastern, which isn't exactly during working hours for most people. So either the people responsible for some of these apps are working late anyway, or any breakage will not be fixed immediately. Twitter could have easily planned this for a monday morning and communicated it a month in advance.
- brown9-2 17y agoWhy does the status of third-party client apps need to concern Twitter so much? Twitter isn't responsible for a third party's product. That's why it's called a "third party".
- smakz 17y agoAbout the actual technical root of the issue - does anyone else find it surprising that twitter is actually using integers for each status update added to it's database and incrementing as it goes? It seems to be this is particularly bad form for a number of reasons: not only does it lend itself to overflow issues such as this - it allows competitors to easily gauge twitter's usage. I would of had a generated pseudo random sequence values from the start - it's not like having a simple integer buys you anything in this case.
- grinich 17y agoI'm just glad it was after WWDC ended. Developers are by nature anti-social creatures. Without Twitter, I think shoe-gazing would have become the main event.