7 ms·

I've had enough years to become wiser, become a fanatic for configuration management, and get over the embarrassment: I'm the consultant that screwed things up.

I've had enough years to become wiser, become a fanatic for configuration management, and get over the embarrassment: I'm the consultant that screwed things up. Some background: the Stat department was running a variety of systems besides the Solaris workstations, and there was, within UNC-CH, a separate support organization that was cheaper and more comfortable with Microsoft products where Stat was sending their support dollars. When that organization needed Unix support, they called my employer, Network Computing Solutions, and I showed up. There was effectively no firewall at UNC-CH at the time (something something academic freedom something something), and the Stat Solaris machines were not being regularly patched. Uninvited guests had infested them, and it appeared the most likely entry point was sendmail - at the time, it was the most notorious vulnerability on the internet. Since my preference to wipe and reload was unacceptable - too much downtime and too many billable hours - the obvious thing to do was update sendmail. The rest is history.

rlonn 6y ago
I read this story years ago and thought it was hilarious. Could've happened to anyone. In my book you're a near-celebrity and it's great that you can verify the story! Thanks for making things a little more interesting and a lot more fun :)
NicoJuicy 6y ago
> and get over the embarrassment It's not that bad, these things happen. It makes an interesting story though
sillysaurusx 6y ago
Can I get your autograph? The 500 mile email story has been a story since I was a teenager. Thank you for the comment. It was delightful to hear your take on the (mildly apocryphal, but highly enjoyable) tale.
- sah2ed 6y ago
  How about a joint autograph from both the protagonist—Trey Harris, and the antagonist—the consultant to the Stats department :) ? Trey Harris also on HN [0]. [0] https://news.ycombinator.com/threads?id=TreyHarris https://news.ycombinator.com/threads?id=TreyHarris
  - techdragon 6y ago
    Yes. Seriously... I’m willing to pay for the shipping and everything. I’ve told this story countless times to people over the years. This and the “OpenOffice can’t print on Tuesday” bug are two of my favourite troubleshooting stories.
    - franky47 6y ago
      HN thread for the OpenOffice story: https://news.ycombinator.com/item?id=8171956 https://news.ycombinator.com/item?id=8171956
      - dang 6y ago
        Also https://news.ycombinator.com/item?id=11717010 https://news.ycombinator.com/item?id=11717010 and https://news.ycombinator.com/item?id=22157430 https://news.ycombinator.com/item?id=22157430
    - zemvpferreira 6y ago
      I would also be ready to pay for a signed copy of this story. I’d frame it up in the office :)
      - hardsoftnfloppy 6y ago
        Can I get a signed copy on dot matrix printout please
        markstos 6y ago
        Finding a working dot matrix printer may be the most expensive part!
czzr 6y ago
I’ve read this story many times, it’s hilarious (and could happen to anyone). Thanks for filling in that background - HN is so great for these kind of moments!
ColanR 6y ago
This should be added as an appendix to the original story. The original has been a favorite of mine for years and I love the addition.
pontifier 6y ago
This is absolutely one of the key formative stories that helped me to think about systems at light speed scales. I'm currently at the very early stages of building a science museum and will eventually try to incorporate this story into an exhibit about light speed. This along with "Nanoseconds", foot long pieces of wire like what Grace Hopper handed out, can truly help to bring this topic to life. I'm also attempting to use this as the basis for a blockchain based "proof of proximity" in which a very high number of round trip encryptions of the previous blocks hash are stored in a bitcoin block. The number of round trips would be high enough that devices even a few hundred feet apart couldn't complete the task before the next block.
- TurkishPoptart 6y ago
  I actually don't understand the last part of the OP story - can someone explain it?
  - pontifier 6y ago
    The sending mail server was configured with a zero timeout for connections. If it didn't IMMEDIATELY get a response from the destination mail server it would fail. This immediate failure took 3 milliseconds, a long enough time that some servers could actually respond back before that happened... but if the server was too far away (more than 500 miles) the connection would fail before the first packet could even get there due to the finite speed of light.
    - gurjeet 6y ago
      Thanks for writing the TL;DR version!
    - chriselles 6y ago
      As a tech/coding newb I was assuming it related to speed of signal travel to approximate 500 ish miles. Thanks so much for confirming it. I‘m going to share this one with my sons who will appreciate the humour.

rlonn 6y ago

I read this story years ago and thought it was hilarious. Could've happened to anyone. In my book you're a near-celebrity and it's great that you can verify the story! Thanks for making things a little more interesting and a lot more fun :)

NicoJuicy 6y ago

> and get over the embarrassment It's not that bad, these things happen. It makes an interesting story though

sillysaurusx 6y ago

Can I get your autograph? The 500 mile email story has been a story since I was a teenager. Thank you for the comment. It was delightful to hear your take on the (mildly apocryphal, but highly enjoyable) tale.

sah2ed 6y ago
How about a joint autograph from both the protagonist—Trey Harris, and the antagonist—the consultant to the Stats department :) ? Trey Harris also on HN [0]. [0] https://news.ycombinator.com/threads?id=TreyHarris https://news.ycombinator.com/threads?id=TreyHarris
- techdragon 6y ago
  Yes. Seriously... I’m willing to pay for the shipping and everything. I’ve told this story countless times to people over the years. This and the “OpenOffice can’t print on Tuesday” bug are two of my favourite troubleshooting stories.
  - franky47 6y ago
    HN thread for the OpenOffice story: https://news.ycombinator.com/item?id=8171956 https://news.ycombinator.com/item?id=8171956
    - dang 6y ago
      Also https://news.ycombinator.com/item?id=11717010 https://news.ycombinator.com/item?id=11717010 and https://news.ycombinator.com/item?id=22157430 https://news.ycombinator.com/item?id=22157430
  - zemvpferreira 6y ago
    I would also be ready to pay for a signed copy of this story. I’d frame it up in the office :)
    - hardsoftnfloppy 6y ago
      Can I get a signed copy on dot matrix printout please
      - markstos 6y ago
        Finding a working dot matrix printer may be the most expensive part!

czzr 6y ago

I’ve read this story many times, it’s hilarious (and could happen to anyone). Thanks for filling in that background - HN is so great for these kind of moments!

ColanR 6y ago

This should be added as an appendix to the original story. The original has been a favorite of mine for years and I love the addition.

pontifier 6y ago

This is absolutely one of the key formative stories that helped me to think about systems at light speed scales. I'm currently at the very early stages of building a science museum and will eventually try to incorporate this story into an exhibit about light speed. This along with "Nanoseconds", foot long pieces of wire like what Grace Hopper handed out, can truly help to bring this topic to life. I'm also attempting to use this as the basis for a blockchain based "proof of proximity" in which a very high number of round trip encryptions of the previous blocks hash are stored in a bitcoin block. The number of round trips would be high enough that devices even a few hundred feet apart couldn't complete the task before the next block.

TurkishPoptart 6y ago
I actually don't understand the last part of the OP story - can someone explain it?
- pontifier 6y ago
  The sending mail server was configured with a zero timeout for connections. If it didn't IMMEDIATELY get a response from the destination mail server it would fail. This immediate failure took 3 milliseconds, a long enough time that some servers could actually respond back before that happened... but if the server was too far away (more than 500 miles) the connection would fail before the first packet could even get there due to the finite speed of light.
  - gurjeet 6y ago
    Thanks for writing the TL;DR version!
  - chriselles 6y ago
    As a tech/coding newb I was assuming it related to speed of signal travel to approximate 500 ish miles. Thanks so much for confirming it. I‘m going to share this one with my sons who will appreciate the humour.