5 ms·

How one guy with a 20yo telephone did $30m in damages in less than one night.

67 points by joshwa 18y ago

jrockway 18y ago
I think this is a "straw that broke the camel's back" type of thing. Three guys getting some extra phone calls on one night will cost your company 80 million dollars? That means there is something seriously wrong with your business. 80 million dollars buys a lot of custom software to automate this sort of thing.
- biohacker42 18y ago
  What this guys said: http://www.reddit.com/r/programming/comments/7vvti/what_on_gods_green_earth_gets_a_fucking_pl7/c07k8f9 http://www.reddit.com/r/programming/comments/7vvti/what_on_g...
  - jrockway 18y ago
    Yeah. I refuse to comment on Reddit, though, so I rehashed it over here.
    - pavel_lishin 18y ago
      Why the strong refusal?
      - jrockway 18y ago
        I really want to have intelligent discussions, but everyone over there just posts stupid personal attacks when they disagree with you. If I'm going to spend the effort composing a post, I want people to read an think about it, not just make a snide attack. (FWIW, I am guilty of making TONS of snide personal attacks on Reddit. The community rewards it.) The thread that made me quit Reddit for good is this one: http://www.reddit.com/r/programming/comments/7uvkz/to_stop_all_the_multiple_posts_why_not_add_a/c07htm2 http://www.reddit.com/r/programming/comments/7uvkz/to_stop_a... My post is definitely on the sarcastic side, but it did take me some time to think about and compose. I just wanted to show that people aren't "idiots" for using an ORM. In return, I get a personal attack from someone who has probably never written a line of code in his life. This just doesn't happen on HN. There are people with ideas that I disagree with, but at least they have something more intelligent to say than "your dumb". So since I only have a finite amount of time, and I spend too much of it replying to social news sites, I figured I'd give up the stupid one for good. We will see how it goes. BTW, I don't think everything on HN is intelligent... but it is several orders of magnitude better than Reddit. (I wrote a bit about this last night, as well: http://news.ycombinator.com/item?id=473023 http://news.ycombinator.com/item?id=473023). Also, I'm sure I make stupid comments from time to time... but hey, nobody's perfect :)
        eli 18y ago
        I don't think everything on HN is intelligent... but it is several orders of magnitude better than Reddit. I second that. In fact, I couldn't click that link because I set my HOSTS file to block reddit.com from my work computer. I was spending way too much time on that site having stupid arguments.
        pavel_lishin 18y ago
        I think as HN gets bigger, it'll become more and more like reddit. You either have to learn to live with idiots (like me, I'll admit) or you gotta keep island-hopping.
        jrockway 18y ago
        A secondary issue was the time investment. Now that I no longer care about trying to educate Reddit, I have time to ... not do that -- there is always something better to do than to converse with people that don't want to converse with you. If HN "goes to shit", you can bet that I will get fed up and leave. It doesn't bother me much -- the people that I like on HN will probably go to the same place. (Plus, HN does good things like completely deleting comments that are just going to attract flames. Censorship sucks, but if you want to talk without being deleted, get a blog. Hopefully HN's aggressiveness here will ensure that it remains the "island" to hop to for a few more years.)
        rm999 18y ago
        That guy on reddit is a troll, and his comment was modded down. No free social site will ever be invulnerable to trolls, but they must be ignored. I post on both reddit and here, and I think your generalization that "everyone over there just posts stupid personal attacks when they disagree with you" is not constructive. Your original comment about how you refuse to comment on reddit added zero value to the conversation about a telephone company from 2000 - all it did was start a thread attacking a competing website. Doesn't this break the spirit of hackernews's rules on civility? The mentality I sometimes see on here that "we are smarter than them" doesn't make a better site. Do it, don't say it. Threads like this aren't "intelligent," they are mean-spirited and off-topic. Sorry, but the comparisons to reddit have been bugging me. I really want to be a member of this site, party because I think its foundation of intelligent and civil conversation is a great idea. But I see way too many comments that wouldn't even be kosher on reddit: http://www.reddit.com/r/reddit.com/comments/7necx/heres_a_thought_stop_talking_about_digg_we_arent/ http://www.reddit.com/r/reddit.com/comments/7necx/heres_a_th...
smanek 18y ago
There's definitely something missing from that story ... How did the 'attacker' just happen to have a wardialer and know to use it that night? Why couldn't the NOC just shut off that one guy's service? Why can't I find any reference to this in the news? (they always love a good cracker story). How was a company that big operating in such an ad-hoc manner?
- tlb 18y ago
  I think a more likely theory is that the telecom providers frequently jack their rates, to see if people notice. If someone doesn't notice, they keep jacking them and rake in profits. It's a good bet that at least one NOC is asleep at the switch on a given night.
- gurtwo 18y ago
  I find hard to believe that during such a special night all the company's fate depended on 3 guys with no backup and no direct supervision. If that's true, it seems to me very unprofessional.
  - pavel_lishin 18y ago
    After reading The Daily WTF (and assuming those stories are at least 80% truth), these sorts of tales don't really surprise me.
  - eli 18y ago
    I find it totally believable. Really, really stupid, but believable. This is some off-brand calling card company, not Verizon.
  - jobeirne 18y ago
    The wonderful thing about this story is that if it had happened a month ago, odds are that we'd be hearing about a pending bailout.
- rglullis 18y ago
  Given it is 2001 and not so much broadband adoption, I'd find it easier to believe if it was some script-kiddie who found a way to play with a modem and DTMF signal generation. Also, as already pointed out, this does not seem to be a big company at all. In a world where even a idiot can start a company, you will eventually end up with companies run by idiots.
brianobush 18y ago
possibly an inside job? or by a competitor or contract channel (quest, etc.)
ShabbyDoo 18y ago
I liken this story to the mortgage-backed securities mess/scandal/disaster. Read http://semyondukach.blogspot.com/2009/01/real-cause-of-financial-crisis.html http://semyondukach.blogspot.com/2009/01/real-cause-of-finan... -- a link previously posted here on HN. 99.99% of the nights, this company could get by without a few $M of software to automate these processes and protect it from extreme risks. However, it couldn't survive even a single night of failure. It's like a player in an online poker tournament who makes extreme and unnecessary all-in bets to steal pots: The strategy works really well until the last time you try it. I wonder if management/investors were really aware of the risk profile they had implicitly accepted.
dhimes 18y ago
I can't read any more. My bs detector is whistling way too loudly...
TimBowen 18y ago
This is a really interesting story even if it is embellished or flat out false, good read.
rmason 18y ago
I ran the story by a local telco guy tonight who said the guy has the lingo right but its bogus. He told me that two guys couldn't run a NOC with that equipment for that level of calls. He also mentioned there were some other technical errors in the story like the fact MySQL was not used at all in the industry at that time and he doubted PHP was either. He said make it $800,000 lost in one night instead of $80,000,000 and it might be a stretch but be believable.
petercooper 18y ago
The Sunday before thanksgiving is the biggest day of the year for people using call cards? Not saying it's a lie but that sounds weird to me. I'd have guessed New Years Eve/Day, Christmas Eve or something like that.. or perhaps even the first day of the work year or whatever.

67 points by joshwa 18y ago

I think this is a "straw that broke the camel's back" type of thing. Three guys getting some extra phone calls on one night will cost your company 80 million dollars? That means there is something seriously wrong with your business. 80 million dollars buys a lot of custom software to automate this sort of thing.

biohacker42 18y ago
What this guys said: http://www.reddit.com/r/programming/comments/7vvti/what_on_gods_green_earth_gets_a_fucking_pl7/c07k8f9 http://www.reddit.com/r/programming/comments/7vvti/what_on_g...
- jrockway 18y ago
  Yeah. I refuse to comment on Reddit, though, so I rehashed it over here.
  - pavel_lishin 18y ago
    Why the strong refusal?
    - jrockway 18y ago
      I really want to have intelligent discussions, but everyone over there just posts stupid personal attacks when they disagree with you. If I'm going to spend the effort composing a post, I want people to read an think about it, not just make a snide attack. (FWIW, I am guilty of making TONS of snide personal attacks on Reddit. The community rewards it.) The thread that made me quit Reddit for good is this one: http://www.reddit.com/r/programming/comments/7uvkz/to_stop_all_the_multiple_posts_why_not_add_a/c07htm2 http://www.reddit.com/r/programming/comments/7uvkz/to_stop_a... My post is definitely on the sarcastic side, but it did take me some time to think about and compose. I just wanted to show that people aren't "idiots" for using an ORM. In return, I get a personal attack from someone who has probably never written a line of code in his life. This just doesn't happen on HN. There are people with ideas that I disagree with, but at least they have something more intelligent to say than "your dumb". So since I only have a finite amount of time, and I spend too much of it replying to social news sites, I figured I'd give up the stupid one for good. We will see how it goes. BTW, I don't think everything on HN is intelligent... but it is several orders of magnitude better than Reddit. (I wrote a bit about this last night, as well: http://news.ycombinator.com/item?id=473023 http://news.ycombinator.com/item?id=473023). Also, I'm sure I make stupid comments from time to time... but hey, nobody's perfect :)
      - eli 18y ago
        I don't think everything on HN is intelligent... but it is several orders of magnitude better than Reddit. I second that. In fact, I couldn't click that link because I set my HOSTS file to block reddit.com from my work computer. I was spending way too much time on that site having stupid arguments.
      - pavel_lishin 18y ago
        I think as HN gets bigger, it'll become more and more like reddit. You either have to learn to live with idiots (like me, I'll admit) or you gotta keep island-hopping.
        jrockway 18y ago
        A secondary issue was the time investment. Now that I no longer care about trying to educate Reddit, I have time to ... not do that -- there is always something better to do than to converse with people that don't want to converse with you. If HN "goes to shit", you can bet that I will get fed up and leave. It doesn't bother me much -- the people that I like on HN will probably go to the same place. (Plus, HN does good things like completely deleting comments that are just going to attract flames. Censorship sucks, but if you want to talk without being deleted, get a blog. Hopefully HN's aggressiveness here will ensure that it remains the "island" to hop to for a few more years.)
      - rm999 18y ago
        That guy on reddit is a troll, and his comment was modded down. No free social site will ever be invulnerable to trolls, but they must be ignored. I post on both reddit and here, and I think your generalization that "everyone over there just posts stupid personal attacks when they disagree with you" is not constructive. Your original comment about how you refuse to comment on reddit added zero value to the conversation about a telephone company from 2000 - all it did was start a thread attacking a competing website. Doesn't this break the spirit of hackernews's rules on civility? The mentality I sometimes see on here that "we are smarter than them" doesn't make a better site. Do it, don't say it. Threads like this aren't "intelligent," they are mean-spirited and off-topic. Sorry, but the comparisons to reddit have been bugging me. I really want to be a member of this site, party because I think its foundation of intelligent and civil conversation is a great idea. But I see way too many comments that wouldn't even be kosher on reddit: http://www.reddit.com/r/reddit.com/comments/7necx/heres_a_thought_stop_talking_about_digg_we_arent/ http://www.reddit.com/r/reddit.com/comments/7necx/heres_a_th...

smanek 18y ago

There's definitely something missing from that story ... How did the 'attacker' just happen to have a wardialer and know to use it that night? Why couldn't the NOC just shut off that one guy's service? Why can't I find any reference to this in the news? (they always love a good cracker story). How was a company that big operating in such an ad-hoc manner?

tlb 18y ago
I think a more likely theory is that the telecom providers frequently jack their rates, to see if people notice. If someone doesn't notice, they keep jacking them and rake in profits. It's a good bet that at least one NOC is asleep at the switch on a given night.
gurtwo 18y ago
I find hard to believe that during such a special night all the company's fate depended on 3 guys with no backup and no direct supervision. If that's true, it seems to me very unprofessional.
- pavel_lishin 18y ago
  After reading The Daily WTF (and assuming those stories are at least 80% truth), these sorts of tales don't really surprise me.
- eli 18y ago
  I find it totally believable. Really, really stupid, but believable. This is some off-brand calling card company, not Verizon.
- jobeirne 18y ago
  The wonderful thing about this story is that if it had happened a month ago, odds are that we'd be hearing about a pending bailout.
rglullis 18y ago
Given it is 2001 and not so much broadband adoption, I'd find it easier to believe if it was some script-kiddie who found a way to play with a modem and DTMF signal generation. Also, as already pointed out, this does not seem to be a big company at all. In a world where even a idiot can start a company, you will eventually end up with companies run by idiots.

brianobush 18y ago

possibly an inside job? or by a competitor or contract channel (quest, etc.)

ShabbyDoo 18y ago

I liken this story to the mortgage-backed securities mess/scandal/disaster. Read http://semyondukach.blogspot.com/2009/01/real-cause-of-financial-crisis.html http://semyondukach.blogspot.com/2009/01/real-cause-of-finan... -- a link previously posted here on HN. 99.99% of the nights, this company could get by without a few $M of software to automate these processes and protect it from extreme risks. However, it couldn't survive even a single night of failure. It's like a player in an online poker tournament who makes extreme and unnecessary all-in bets to steal pots: The strategy works really well until the last time you try it. I wonder if management/investors were really aware of the risk profile they had implicitly accepted.

dhimes 18y ago

I can't read any more. My bs detector is whistling way too loudly...

TimBowen 18y ago

This is a really interesting story even if it is embellished or flat out false, good read.

rmason 18y ago

I ran the story by a local telco guy tonight who said the guy has the lingo right but its bogus. He told me that two guys couldn't run a NOC with that equipment for that level of calls. He also mentioned there were some other technical errors in the story like the fact MySQL was not used at all in the industry at that time and he doubted PHP was either. He said make it $800,000 lost in one night instead of $80,000,000 and it might be a stretch but be believable.

petercooper 18y ago

The Sunday before thanksgiving is the biggest day of the year for people using call cards? Not saying it's a lie but that sounds weird to me. I'd have guessed New Years Eve/Day, Christmas Eve or something like that.. or perhaps even the first day of the work year or whatever.