7 ms·
"Given the preparation work that's gone into this, we are confident that all of our services will remain available to our customers throughout the weather." - y
by RobAley 14y ago
"Given the preparation work that's gone into this, we are confident that all of our services will remain available to our customers throughout the weather." - yesterdays update.
Try not to let your fingers type cheques your datacenters can't cash...!
- jrockway 14y agoNobody was really expecting this much of Manhattan to lose power.
- smackfu 14y agoWell, it seems like their confidence was based on their single datacenter not going down. Which seems misplaced.
- jpdoctor 14y ago... especially since there is more than one historical example for all of Manhattan losing power. (One of those times involved looting and civil strife.) Combined with the verbiage about "once-in-a-generation storm", it is fortunate that any part of Manhattan had power.
- count 14y agoNobody except the whole world watching the Weather channel. I'm frankly amazed that Manhattan survived!
- jusben1369 14y agoI would tend to disagree with you jrock. Nearly all the models talked about this storm wrecking this type of havoc.
- jrockway 14y agoI meant ordinary citizens, not emergency planners. Despite being told that they could be without power, many of my friends did not believe it. "How could Manhattan lose power?"
- TillE 14y ago> "How could Manhattan lose power?" Seriously? Were they not living in New York in 2003?
- mturmon 14y agoI live in California, so I know what you mean (s/weather/earthquake/). People were forgetting about events like: http://en.wikipedia.org/wiki/Northeast_blackout_of_2003 http://en.wikipedia.org/wiki/Northeast_blackout_of_2003 http://en.wikipedia.org/wiki/New_York_City_blackout_of_1977 http://en.wikipedia.org/wiki/New_York_City_blackout_of_1977 http://en.wikipedia.org/wiki/Northeast_blackout_of_1965 http://en.wikipedia.org/wiki/Northeast_blackout_of_1965 Humans have a hard time reasoning about events on decadal time scales.
- RobAley 14y agoI'm sorry, but these chaps have been around the block a few times, they're not new start-ups. They know that there is no way on earth you can guarantee (or even reasonably be sure that) a single datacenter won't fail, even under non-emergency conditions, so their customers (I'm not one) should be calling them out on why they said that all would be A-OK. It would have been much better to say something like "We have put all reasonable preparations & precautions in place (see above), and we feel confident they will deal with most things the storm will throw at us. However please be aware that we have no fail-over datacenter available so please plan accordingly."
- darkxanthos 14y agoYou can be reasonably sure without being certain. They didn't guarantee anything. If you misread anything they've written as "guarantee" their services will never go down, you're blaming the wrong person. (Full disclosure: I am one of their customers)
- RobAley 14y ago> "we are confident that all of our services will remain available" When you get this statement from an outfit with the pedigree and experience of Fog Creek, its as close to a guarantee as you're going to find. No misreading necessary. At the end of the day, they could not be reasonably sure, there was a non-trivial risk that they should have been (and almost certainly were) aware of (this isn't the first time bad weather and datacenters have mixed), and they didn't communicate that to their customers.
- TillE 14y agoIt's pretty amusing to see you posting this when a couple days ago, you were accusing the media of "blowing it out of proportion". Oh hey, it turns out they were right. http://news.ycombinator.com/item?id=4706959 http://news.ycombinator.com/item?id=4706959
- jrockway 14y agoI never use the words that you "quoted". I said "superstorm" was a stupid term. It's a hurricane. Use a well-defined word when it's available. (The hurricane was also predicted to weaken to a tropical storm before landfall at the time I wrote that.)
- roc 14y agoEarlier yesterday it wasn't a hurricane though. And shortly after landfall it was no longer a hurricane again - even before it weakened. (cold core => post-tropical cyclone?) I think using a general catch-all that was not a narrowly-defined technical term that didn't/wouldn't universally apply was actually a prudent and defensible thing. Given their goal of collecting all the concerns of all the stages of the storm under one umbrella. Even if their motivation was just stupid news branding/sensationalism.
- antiterra 14y agoYou predicted it was just going to rain a lot; you completely ignored everything reported on storm surges. Even now, you're concerned with the hurricane classification and missed the fact that barometric pressure, tide timing and bathymetry of the New York Harbor/Long Island Sound were the currently predicted causes of flooding, not simply windspeed. As I responded to your comment, "hurricane" or "tropical storm" classifications were not appropriately descriptive, as Sandy was predicted to (and did) merge with another system to morph from a warm core tropical style system to a cold core nor'easter system. The area of the storm was particularly large, which was another reason for the "super" attribution.
- darkarmani 14y agoYou mean nobody that somehow missed the media coverage expected this. Those that watched the media coverage prepared for this level of disaster.
- smackfu 14y agoIt's not even clear what kind of prep you would do for a hurricane that ensures service will be available in a hurricane.
- RobAley 14y agoThere isn't much beyond what they did, assuming you don't have another datacenter to preemptively switch load to. But what you don't do is tell your customers it will be all good. You tell them what you've done and warn them they may be downtime so they can plan accordingly.
- drewcrawford 14y agoYou might start by moving infrastructure out of the hurricane. Off the east coast would be nice, but at the very least outside of a mandatory evacuation zone [1]. Alternatively, you could design some type of system that would allow you to fail over to a geographically redundant datacenter. Joel claimed in 2007 [2] that they had such a system, and touted it as a selling point of the reliability of the hosted service. What has happened to it is probably only something that a Fog Creek engineer can tell you. [1] http://news.ycombinator.com/item?id=4718653 http://news.ycombinator.com/item?id=4718653 [2] http://webcache.googleusercontent.com/search?q=cache:lHEK939AKiEJ:www.joelonsoftware.com/items/2007/07/09.html+&cd=5&hl=en&ct=clnk&gl=us&client=safari http://webcache.googleusercontent.com/search?q=cache:lHEK939...