13 ms·
A colleague of mine just came bursting through my office door in a panic, thinking he brought our site down since this happened just as he made some changes to
by itzjacki 11mo ago
A colleague of mine just came bursting through my office door in a panic, thinking he brought our site down since this happened just as he made some changes to our Cloudflare config. He was pretty relieved to see this post.
- Freak_NL 11mo agoIs there a word for that feeling of relief when someone else fucked up after initially thinking it was you?
- StanAngeloff 11mo agoSchadenfreude
- simonklitj 11mo agoNot quite, that’s more like taking pleasure in the misfortune of someone else. It’s close, but the specific relief bit that it is not _your_ misfortune is not captured
- skottenborg 11mo agoInternettet er vist ikke så stort :-)
- simonklitj 11mo agoFætter! Hvor genialt at se dig her. :)
- gnfargbl 11mo agoNah, that's delight in someone else's misfortune. This is delight that the misfortune wasn't yours, which is slightly different.
- StanAngeloff 11mo ago4 years of German and I still don't quite "get" it :^) TY!
- tagyro 11mo agoWe have a saying: You know how you measure eternity? When you finish learning German.
- namblooc 11mo agoKatastrophenverursachererleichterung
- jpmonette 11mo agophewphoria
- Freak_NL 11mo agoYup, that works.
- deleted 11mo ago[deleted]
- Sholmesy 11mo agoWell, at least something good came out of this incident. Perfect.
- zzzeek 11mo agoit has to sound like a german word though
- datenhorst 11mo agopuhphorie
- puilp0502 11mo agoIs there a word for a feeling that there's gotta be a German word for this niche feeling?
- patneedham 11mo agoProbably Deutschwortsehnsucht (https://www.iamexpat.de/education/education-news/german-words-expats-should-know-sehnsucht https://www.iamexpat.de/education/education-news/german-word...)
- usrusr 11mo agoYou mean like when the Wortzusammensetzungsverdacht just hits you? (yeah, I just made that up, that's the beauty)
- chrisweekly 11mo ago
- shortrounddev2 11mo agoWhen I'm debugging something, I'm not usually looking for the solution to the problem; I'm looking for sufficient evidence that I didn't cause the problem. Once I have that, the velocity at which I work slows down
- sys_64738 11mo agoMy manager once asked if he could have a "quick word". I said "velocity".
- cromka 11mo agovindication?
- jspash 11mo agoThe problem is, I still get the wrong end of the stick when AWS or CF go down! Management doesn't care, understandably. They just want the money to keep coming in. It's hard to convince them that this is a pretty big problem. The only thing that will calm them down a bit is to tell them Twitter is also down. If that doesn't get them, I say ChatGPT is also down. Now NOBODY will get any work done! lol.
- graemep 11mo agoWho decided to go with AWS of CF? If its a management decision tell them you need the resources to have a fallback if they want their system to be more reliable than AWS or CF.
- adriand 11mo agoHaha yeah I just got off the phone and I said, look, either this gets fixed soon or there's going to be news headlines with photographs of giant queues of people milling around in airports.
- aurareturn 11mo agoEvery time a major cloud goes down, management tells us why don't we have a backup service that we can switch to. Then I tell them that a bunch of services worth a lot more than us are also down. Do you really want to spend the insane amount of resources to make sure our service stays up when the global internet is down?
- ec109685 11mo agoHaving an alt to Cloudflare isn’t preposterous.
- hylaride 11mo agoThis is why you ALWAYS have a proposal ready. I literally had my ass saved by having tickets with reliability/redundancy work clearly laid out with comments by out of touch product/people managers deprioritizing the work after attempts to pull it off the backlog (in one infamous case for a notoriously poorly conceived and expensive failure of a project that haunted us again with lost opportunity cost). The hilarious part of the whole story is that the same PMs and product managers were (and I cannot overemphasize this enough) absolutely militant orthodox agile practitioners with jira.
- mcphage 11mo agoMaybe this isn’t great, but I get a hint of that feeling when I’m on an airplane and hear a baby crying. For a number of years, if I heard a baby crying, it was probably my baby and I had to deal with it. But now my kids are past that phase, so when I hear the crying, after that initial jolt of panic I realize that it isn’t my problem, and that does give me the warm fuzzies. Even though I do feel bad for the baby and their parents.
- adriand 11mo agoRelated situation: you're at a family gathering and everyone has young kids running around. You hear a thump, and then some kid starts screaming. Conversation stops and every parent keenly listens to the screams to try and figure out whose kid just got hurt, then some other parent jumps up - it's not your kid! #phewphoria
- RhysU 11mo agoYou're not alone in this feeling. I occasionally smile when it's not my kid.
- hackeraccount 11mo agoThis is one of the secret joys of being a parent.
- spamizbad 11mo agoWhat’s funny is as I get older this feeling of relief turns more like a feeling of dread. The nice thing about problems that you cause is that you have considerable autonomy to fix them. Cloudflare goes down you’re sitting and waiting for a 3 party to fix something.
- mewpmewp2 11mo agoWhy is it dread? I always feel good when big players mess up, as it makes me feel better about my own mess ups in life previously.
- kasey_junk 11mo agoBecause my customers don’t (and shouldn’t care) it’s a third party. If I caused it there is a chance I can fix it.
- mewpmewp2 11mo agoIf customers expected third party downtime to not affect their thing then you shouldn't have picked a third party provider or spent extra resources on not having a single point of failure? If they were happy with choosing the third party with knowledge of depending on said third party provider, then it was an accepted risk.
- _factor 11mo agoSo you would rather be incompetent than powerless? Choice of third party vendor on client facing services is still on you, so maybe you prefer your incompetence be more direct and tangible? Even still, you should have policies in place to mitigate such eventualities, that way you can focus the incompetence into systematic issues instead. The larger the company, the less acceptable these failures become. Lessons learned is a better excuse for a shake and break startup than an established player that can pay to be secure. At some point, the finger has to be pointed. Personally, I don't dread it pointing elsewhere. Just means I've done my due D and C.
- 11mo ago
- stonecharioteer 11mo agoIt's close enough to Schadenfreude but not really.
- deleted 11mo ago[deleted]
- bookofjoe 11mo agoThe German word “schadenfreude” means taking pleasure in someone else’s misfortune; enjoyment rather than relief.
- bryanrasmussen 11mo agosince schaden is damage and freude is joy, not sure what it should be - maybe Schadeleichtig hmm...
- tauchunfall 11mo ago>maybe Schadeleichtig Maybe "Erleichterung" (relief)? But as a German "Schadenserleichterung" (also: notice the "s" between both compound word parts) rather sounds like a reduction of damage (since "Erleichterung" also means mitigation or alleviation).
- bryanrasmussen 11mo agoright I thought of that at first and discarded it for that reason. Which the problem really is that the normal story of how Schadenfreude works as a bit of German language how to is that the component that it is other people's damage that is sparking joy is missing from the word itself, that interpretation must be known by the word user, if you were just creating the word and nobody had heard it before in the world it would be pretty reasonable for people to think you had just created a new word for masochism.
- deleted 11mo ago[deleted]
- deleted 11mo ago[deleted]
- Rooster61 11mo agoSchadenfriend? You gain relief, but you don't exactly derive pleasure as it's someone you know that's getting the ass end of the deal
- hoistbypetard 11mo agoschadenfuckup
- deleted 11mo ago[deleted]
- NitpickLawyer 11mo agoYou missed a great opportunity to dead-pan him with something like "No, Bob, not just our site, you brought down the entire Internet, look at this post!"
- sakisv 11mo agoWell, you can never be sure that he didn't: https://www.fastly.com/blog/summary-of-june-8-outage https://www.fastly.com/blog/summary-of-june-8-outage
- itzjacki 11mo agoOh don't you worry. We are very much talking about the global outage as if he was the root cause. Like good colleagues :)
- rapnie 11mo agoHmm, wait a minute.. maybe he was the cause! (no, kidding. just upping the pressure as a good peer :)
- bryanrasmussen 11mo agoare we truly good if we don't start a class action suit against this hapless scapegoat?!
- conorcleary 11mo agoJust join the one we've started over in this cubicle!
- srmarm 11mo ago> May 12, we began a software deployment that introduced a bug that could be triggered by a specific customer configuration under specific circumstances. I'd love to know more about what those specific circumstances were!
- CableNinja 11mo agoDamn, imagine being the customer responsible for that, oof
- whstl 11mo ago
- belter 11mo agoWait for the post mortem ... It is a technical possibility, race condition propagates one customer config to all nodes... :-)
- bamboozled 11mo agoHow do we know your colleagues changes didn't take down Cloudflare though?
- disconnection 11mo agoGood that he worked it out so quick. I recently spent a day debugging email problems on Railway PaaS, because they silently closed an SMTP port without telling anyone.
- nrhrjrjrjtntbt 11mo agoThe company where this colleague works? Cloudflare.
- dcjdfvk 11mo agoEven pornhub is down becuase it uses clouflare.
- carlos_rpn 11mo agoIs Cloudflare being down the work of conservative hackers and the rest of the internet is just collateral damage?
- arbuge 11mo agoTell him it's worse than he thinks. He obviously brought the entire Cloudflare system down.
- mlrtime 11mo agoYou joke and I think its funny, but as a junior engineer I would be quite proud if some small change I made was able to take down the mighty Cloudflare.
- throwup238 11mo agoIf I were Cloudflare it would mean an immediate job offer well above market. That junior engineer is either a genius or so lucky that they must be bred by Pierson’s Puppeteers or such a perfect manifestation of a human fuzzer that their skills must be utilized.
- ethmarks 11mo agoA Ringworld reference in the wild?
- zarathustreal 11mo agoInternet points demand obscure references these days. My system prompt has its own area code
- throwup238 11mo agoI never thought I'd get the chance, but then my Claude Code on Web credits ran out and I had to find another way to entertain myself.
- UltraSane 11mo agoEven after 20 projects I have only used $60 of my $250
- 0xblinq 11mo agoDo you guys work at Cloudflare? Do you mind reverting that change just in case?
- 0xblinq 11mo agoPlot twist: They work at Cloudflare
- ants_everywhere 11mo agoChances are still good that somewhere within Cloudflare someone really did do a global configuration push that brought down the internet. When aliens study humans from this period, their book of fairy tales will include several where a terrible evil was triggered by a config push.
- sefke 11mo agoI woke up getting bombarded by multiple clients messages of sites not working, I shitted my pants because I've changed the config just yesterday. When I saw the status message "cloudflare down" I was so relieved.
- raxxorraxor 11mo agoDid your colleague perhaps change the Cloudflare config again right now? Seems to be down again.
- theoldgreybeard 11mo agoYou should tell him his config change took down half the internet.