3 ms·

Well aware of the pitfalls; I've worked in neteng for most of my career. But your statement of '15 seconds to a couple of minutes' - there's the rub. Do that mo

Well aware of the pitfalls; I've worked in neteng for most of my career. But your statement of '15 seconds to a couple of minutes' - there's the rub. Do that more than once and you've blown your SLA. Don't misunderstand me, I'm not defending Peakweb, simply saying that running a five nines network is hard - it typically requires deep experience in diverse problem domains.

walrus01 10y ago
If you read most any SLA - pre announced maintenance with advance notification is categorized differently than unexpected events, for contractual purposes. JunOS and IOS upgrades need to happen, crossconnects need to get moved, agg switches get replaced, etc. Customers know this. It definitely requires ccie level knowledge and at least 10-15 years experience, plus advanced Linux/BSD server admin skills to really do five nines right. It is indeed expensive and requires enthusiastic cooperation from non technical management responsible for budgets. I worked my way up from a level 1 NOC type position, so I'd like to think that after 20 years I have a good understanding of all the possible OSI layer 1 failure modes (and things you can fuck up in configuration at layers 2 and 3), yet the things some other partner and competitor regional ISPs do continue to surprise me.
- vidarh 10y ago
  It sounds like the two of you are in violent agreement. If it wasn't hard, people with your level of experience wouldn't be needed to get it right. Nobody here are saying - as far as I can tell - that they shouldn't have done better. But that'd require them to actually have people with sufficient experience and the budgets and buy-in.
  - drostie 10y ago
    There's lots of hate for Peak Web here, which I totally understand, but it sounds to me like the problem really stems from Machine Zone shopping around for the cheapest possible contract. Maybe I'm just biased because I know that the game in question was basically just trying to sell desperate men discreet shots of Kate Upton's and Mariah Carey's cleavage. I don't know the details of the countersuit, but that one exists at all is pretty telling. It suggests one of many things: it's possible that the contract did not guarantee any amount of uptime; or maybe Peak Web was not adequately informed of the load that the game was going to take on their servers and therefore they want to argue that the level of downtime was reasonable given that they weren't told to expect those kinds of loads; or maybe the contract had a termination procedure and Machine Zone decided to violate that procedure and just drop the company -- which is not something you can just do. I mean, lawyers will argue anything for cash, but it sounds like Peak Web isn't exactly rolling in the cash they'd need to do this on a whim. I don't know what the chances are that the lawyers in question are working on contingency, but it seems plausible.
    - walrus01 10y ago
      For a small to medium sized hosted to suddenly land a customer that needs $4m/mo of rack and power, or leased dedicated x86-64 servers, they almost certainly went into significant debt and/or leased a lot of the equipment themselves. I doubt peakweb has two nickels to rub together now.

walrus01 10y ago

If you read most any SLA - pre announced maintenance with advance notification is categorized differently than unexpected events, for contractual purposes. JunOS and IOS upgrades need to happen, crossconnects need to get moved, agg switches get replaced, etc. Customers know this. It definitely requires ccie level knowledge and at least 10-15 years experience, plus advanced Linux/BSD server admin skills to really do five nines right. It is indeed expensive and requires enthusiastic cooperation from non technical management responsible for budgets. I worked my way up from a level 1 NOC type position, so I'd like to think that after 20 years I have a good understanding of all the possible OSI layer 1 failure modes (and things you can fuck up in configuration at layers 2 and 3), yet the things some other partner and competitor regional ISPs do continue to surprise me.

vidarh 10y ago
It sounds like the two of you are in violent agreement. If it wasn't hard, people with your level of experience wouldn't be needed to get it right. Nobody here are saying - as far as I can tell - that they shouldn't have done better. But that'd require them to actually have people with sufficient experience and the budgets and buy-in.
- drostie 10y ago
  There's lots of hate for Peak Web here, which I totally understand, but it sounds to me like the problem really stems from Machine Zone shopping around for the cheapest possible contract. Maybe I'm just biased because I know that the game in question was basically just trying to sell desperate men discreet shots of Kate Upton's and Mariah Carey's cleavage. I don't know the details of the countersuit, but that one exists at all is pretty telling. It suggests one of many things: it's possible that the contract did not guarantee any amount of uptime; or maybe Peak Web was not adequately informed of the load that the game was going to take on their servers and therefore they want to argue that the level of downtime was reasonable given that they weren't told to expect those kinds of loads; or maybe the contract had a termination procedure and Machine Zone decided to violate that procedure and just drop the company -- which is not something you can just do. I mean, lawyers will argue anything for cash, but it sounds like Peak Web isn't exactly rolling in the cash they'd need to do this on a whim. I don't know what the chances are that the lawyers in question are working on contingency, but it seems plausible.
  - walrus01 10y ago
    For a small to medium sized hosted to suddenly land a customer that needs $4m/mo of rack and power, or leased dedicated x86-64 servers, they almost certainly went into significant debt and/or leased a lot of the equipment themselves. I doubt peakweb has two nickels to rub together now.