38 ms·
Slack is down
- Snitch-Thursday 6y agoUsing our teams backup chatroom in a competing service. One of these days P2P Matrix will reach GA, then I plan to make a backup for my backups, Starfleet style.
- TeMPOraL 6y agoThat's one obscure reference, I love it. https://www.youtube.com/watch?v=UaPkSU8DNfY https://www.youtube.com/watch?v=UaPkSU8DNfY GILORA: Starfleet code requires a second backup? O'BRIEN: In case the first backup fails. GILORA: What are the chances that both a primary system and its backup would fail at the same time? O'BRIEN: It's very unlikely, but in a crunch I wouldn't like to be caught without a second backup.
- djsumdog 6y agoI forgot about that. Starfleet really was in good shape back in.
- TeMPOraL 6y agoLate 2300s were the golden years for Starfleet and the Federation. Sad to see they went downhill later on.
- iso1631 6y agoMakes perfect sense for O'Brien, DS9 had serious backup issues in the first couple of years The Forsaken (season 1 episode 17) LOJAL: I've been reading the reports of your Chief of Operations, Doctor. They gave me the impression that he was a competent engineer. BASHIR: Chief O'Brien? One of the best in Starfleet. LOJAL: Then why aren't the backup systems functioning? BASHIR: Well, you know, out here on the edge of the frontier, it's one adventure after another. Why don't I escort you back to your quarters where I'm sure we can all wait this out. Rivals (season 2 episode 11) KIRA: My terminal just self-destructed. DAX: What? KIRA: I lost an evaluation report I've been working on for weeks. DAX: Even the backups? KIRA: Even the backups. There's a reason to have a backup to the backup by Destiny (season 3 episode 15)
- geerlingguy 6y agoI got dropped into not-dark-mode with a connection issue message in each of my workspaces. I guess everyone hopping back online over the course of a few hours for the new year is too much to handle!
- gavnewalkar 6y agoMy slack (desktop + mobile) has been down for the past 30~ mins. Strangely I can still receive messages/alerts on my phone.
- deleted 6y ago[deleted]
- Topgamer7 6y agoI too am in this same boat.
- ajb 6y agoMe too. However it won't let me open DMs to certain people.
- curlypaul924 6y agoI cannot get to slack in my phone or in the browser. I wonder if this is because I haven't used the phone app in a few days, so I was already logged out, but you and others were still logged in?
- dhagz 6y agoSame, but when I go to view the message it hangs.
- traumivator 6y agoSame, I still see my colleagues typing but that's it.
- tedmiston 6y agoI've experienced this with Slack before where the push notifications come through but opening them fails to load. I imagine their infrastructure to send push notifications is decoupled from their infrastructure for chat services themselves. It'd be interesting to know if they have a master switch to disable notifications in times like this where they aren't usable anwyay.
- rocho 6y ago
- tempest_ 6y agoWho knew Salesforce could work this fast :p
- deleted 6y ago[deleted]
- chrisseaton 6y agoIt's snarky - snarky comments are against the guidelines here.
- hackerpain 6y agoIndeed.
- btbuildem 6y agoWe were just joking with the work mates -- SF bought Tableau 2 years ago and haven't ruined it yet, only because it takes them that long to do anything ;)
- ellisv 6y agoCan't ruin something that's already ruined.
- tmsh 6y agoThey had the same issue 3 months ago: https://news.ycombinator.com/item?id=24687957 https://news.ycombinator.com/item?id=24687957
- platetone 6y agooh good, it's not just me and my already bad first day of the year... is it too early to start drinking?
- _underfl0w_ 6y ago> is it too early to start drinking? Depends on the timezone you're in, though one could theoretically cite a disparity between physical and mental/emotional/temporal time zones...
- louffoster 6y agono
- iso1631 6y agoYou should probably ask Jimmy Buffett https://www.youtube.com/watch?v=BPCjC543llU https://www.youtube.com/watch?v=BPCjC543llU
- schoolornot 6y agoHow many more outages until all trust is eroded and competing services differentiate themselves on the basis of uptime?
- ceejayoz 6y agoI wouldn't be shocked if businesses saw increased productivity during these.
- falcolas 6y agoNot all - many workflows these days rely on Slack or its ilk. Benderbot, Jira/etc. connectors, calendar connectors, remote communication/standups, alerting…
- postalrat 6y agoI'm sitting here not sure if I should deploy code since most communication with the rest of my team has been cut off.
- derwiki 6y agoIf something went awry, and it caused more pain because Slack was down, how would you feel? If you’re missing comms/observability then waiting to deploy seems prudent.
- elwell 6y agoDuplicate of https://news.ycombinator.com/item?id=25632048 https://news.ycombinator.com/item?id=25632048 I think HN is hiding these posts. Maybe status threads are discouraged now? But they're much more useful than status.slack.com etc.
- geerlingguy 6y agoHN is where we all go when the Internet (or large portions of it) are down. It's more reliable than all the 'downforeveryoneorjustme' or 'downtime monitor' services.
- clappski 6y agoIt's the first page I try when I think I'm having connection issues at least, to verify it's some service that's broken rather than my local network
- spicybright 6y agoAbsolutely. I come here for comments to get an idea from other engineers of what's actually going on. Way more useful than an is it down site.
- detaro 6y ago> Maybe status threads are discouraged now They always have been, since they clearly don't fit the guidelines for what a good submission is and usually leave little for interesting discussions. (unlike postmortems of past outages, which often are good)
- yreg 6y agoYet they are usually incredibly useful for most people here. They should be allowed at least while the event is ongoing.
- noir_lord 6y agoAgreed, when a major service goes down HN is the most accurate overview, often a useful sanity check when its AWS or Slack size orgs before I open an incident with whichever party.
- tmsh 6y agoMy take last time: https://news.ycombinator.com/item?id=24690175 https://news.ycombinator.com/item?id=24690175
- deleted 6y ago[deleted]
- lol768 6y agoThese events seem to be happening almost on a monthly basis now. IRC was never this unreliable and at least with netsplits it was obvious what had happened because you'd see the clients disconnect. IME messages just fail to send with Slack, then you can retry but they're not properly idempotent and you end up sending the messages twice. It's really poor.
- karmicthreat 6y agoEFnet was always splitting every few hours. I don't really miss IRC compared to modern chat systems.
- randylahey 6y agoI'm the opposite. Back when in my early teens, friends and I would attempt to hijack opposing groups' channels via takeovers during net-splits (and ofcourse having the same done to us). What a time to be alive.
- libraryatnight 6y agoIn the early battle.net days competing clans would split and steal channels. It was tons of fun. Taught me lots about bots, proxies, simple scripting, in the process too.
- stormcode 6y agoOh yeah, those were the days. Causing server splits to get your nick back that was stolen in a previous server split...
- pmlnr 6y agoI do miss them, terribly. Lightweight, fast, brutally simple. Even with splits, it was better, and ever since IRC bouncers exist, like ZNC, they are rock solid.
- dkdk8283 6y agoI spent a few hours setting up a chat client for a reason. Slack takes all this away from me.
- seibelj 6y agoDown Detector showed a lot of different services suffering downtime at the same time https://downdetector.com/ https://downdetector.com/ I wonder if it's an AWS region issue
- partiallypro 6y agoI have stopped using Down Detector as an accurate measure because a lot of "outages" are just people having issues with a service unrelated to the service they are reporting as down. Ex: AT&T outage in Nashville caused people to report Xbox Live as down, when it wasn't actually down, etc.
- staticelf 6y agoI have issues reaching a lot of sites, especially american. Both downdetector, hacker news and others loads extremely slow or not at all. Downdetector had a bunch of failed resources for me..
- ibraheemdev 6y ago> Customers may have trouble loading channels or connecting to Slack at this time. Our team is investigating and we will follow up with more information as soon as we have it. We apologize for any disruption caused. - Jan 4, 10:14 AM EST The status for messaging and connection services has been marked as [incident] https://status.slack.com/ https://status.slack.com/
- ibraheemdev 6y agoAll services have now been marked as [outages]: > We're continuing to investigate connection issues for customers, and have upgraded the incident on our side to reflect an outage in service. All hands are on deck on our end to further investigate. We'll be back in a half hour to keep you posted. - Jan 4, 11:20 AM EST > There are no changes to report as of yet. We're still all hands on deck and continuing to dig in on our side. We'll continue to share updates every 30 minutes until the incident has been downgraded - Jan 4, 11:52 AM EST
- vehemenz 6y agoI was seeing issues about 10 minutes before their system status page was updated. I'm surprised they don't have automatic monitoring of some kind.
- loginatnine 6y agoStatus pages are probably manually updated. You don't want a false positive/bug in your monitoring to affect your public metrics.
- vehemenz 6y agoFair enough. Though I'm not sure how I'd feel about the whole world knowing about my service's outage before I do.
- histriosum 6y agoI'm positive that they have internal monitoring, and probably knew about the issues well before they decided to manually update their status page to reflect the issue. Manually updating the status page does not equal no monitoring, after all.
- deleted 6y ago[deleted]
- racl101 6y agoWelcome to the party, pal!
- djtriptych 6y agoCould it be the obvious? Everyone signing on / loading slack clients at the same time?
- jordache 6y agothen wouldn't this happen every monday morning?
- unethical_ban 6y agoPerhaps it is a large number of people checking into channels that are backlogged with lots of bot message notifications.
- floatingatoll 6y agoA lot of organizations essentially took the last two weeks off from work, which is long enough for a 10-day autoscale window to spin down servers, and then get confronted by a load spike that wasn't pre-spun for.
- derwiki 6y agoI would be shocked if Slack operations wasn’t aware of this return to work spike and didn’t pre-scale in anticipation.
- floatingatoll 6y agoI wouldn't, since my personal theory is that the outage is due to AWS and GCP autoscale capacity exhaustion. We'll find out soon enough! EDIT: And down goes Notion, too: https://news.ycombinator.com/item?id=25634159 https://news.ycombinator.com/item?id=25634159
- yreg 6y ago>AWS and GCP autoscale capacity What does this mean? What do cloud providers do when customers scale down their services? Do the providers literally power down servers? Do they sell the capacity to new customers?
- ketamine__ 6y agoMaybe we can chat with coworkers here. Is there a Carl around?
- capableweb 6y agoI'm a Carl. I'm also looking for a coworker who was trying to contact me. If it's about last saturday, I promise nothing really happened between me and her, but I'm sure she already told you.
- djsumdog 6y agoDo you guys not have e-mail? looks through Inbox of 850 new aws, batch job and logging messages oh yea, that's right..
- tinco 6y agoDon't you guys have e-mail filters? "Hey, our site has been down for 2 hours, why aren't you doing anything" Looks at 850 unread messages in ops-notifications folder ooh yeah, that's right..
- iso1631 6y ago"Looks at 850 unread messages in ops-notifications folder" In my organization it's spelt "deleted items"
- thom 6y agoI have tried to sell my organisation on a shared Google Chat doc for 90s style realtime ICQ chat in times like these, but there has been little uptake.
- J5892 6y agoWell now's the time for a big push! Oh wait, how would you share the link...
- 6y ago
- obventio56 6y agoI’m just enjoying it while it lasts
- clashmeifyoucan 6y ago> We’re still investigating the ongoing connectivity issues with Slack. There's no additional information to share just yet, but we’ll follow up in 30 minutes. Thanks for bearing with us. Seems to be working intermittently, however.
- clashmeifyoucan 6y agoSeems like Notion has a service interruption going on as well: https://status.notion.so/ https://status.notion.so/ Potentially related?
- deleted 6y ago[deleted]
- itsdrewmiller 6y agoIt’s pretty embarrassing for their 45 minute update to be “not sure what’s wrong!”
- yreg 6y agoThe status still says "We're continuing to investigate", but they tweeted[0] that they have found the issue. [0] - https://twitter.com/SlackHQ/status/1346132040249470979 https://twitter.com/SlackHQ/status/1346132040249470979
- obiefernandez 6y agoThis is definitely going to catalyze a nascent move over to Discord for my team. (~80 person consulting agency, distributed)
- floatingatoll 6y agoWhich would be unfortunate if based only on evidence of Slack being down today, given how many other sites are down as well. (Discord is up, though!)
- obiefernandez 6y agoit's not based on only that. Slack costs a lot of money and moving off of it is something that has continually come up over the last year or two. We even had a Rocketchat server up and running for awhile.
- pmlnr 6y agoThat's it, I'll move from [closed source, centralized, paid service] to [closed source, centralized, paid service]!
- peeters 6y agoI'll concede that it's possible to not know what the problem is by now, but I won't concede that this should not be called an "outage" at this point.
- richardwhiuk 6y agoHuh? It's definitely an outage.
- ghshephard 6y agoAmidst all the double negatives, I think that's what the parent poster was saying.
- sweezyjeezy 6y agoThat's what they mean - double negative. It has also been upgraded on their side to 'outage'.
- deathanatos 6y agoIt's a red "do not enter" esque thing now, but when the parent posted, I think it was still a yellow triangle. But also, the status page still proudly proclaims that the "Uptime for the current quarter: 100%" — which is clearly false at this point.
- aerovistae 6y agoI initially misread this as saying that you won't concede it shouldn't be called an outrage.
- exhaze 6y agoWhen I was at Uber, we noticed that most incidents are directly caused by human actions that modify the state of the system. Therefore, a large "backlog" of human actions that modify the system state have a much higher chance of causing an incident. My bet is that this incident is caused by a big release after a post-holiday "code freeze".
- nkassis 6y agoI would bet it's just the influx of traffic post holiday with systems that haven't been updated in so long maybe some annoying memory leaks have crept up and gone unnoticed or some other bad state that was exacerbated by return to work day for most NA folks. Code freezes were good at identifying bugs that only show up after long periods. Doubt anyone releasing big changes Monday morning.
- hnlmorg 6y agoThat might be true but when you take the global usage of Slack and their respective time zones, more than half the world would have signed into Slack this morning before SV had and I certainly didn't notice any downtime this morning in my time zone.
- radicalbyte 6y agoIt was ropey before SV woke up, I thought it was just my (normally rock solid thanks to using Ubiquity) network having issues. Guess it was Slack being Slack.
- exhaze 6y agoI haven't worked at Slack, so I can't speak with high confidence. A traffic spike is a possible reason, but I'm willing to bet that it's not the reason: > Doubt anyone releasing big changes Monday morning. This is definitely an engineering best practice, and by best practice, I mean something that Uber's, I mean Slack's SRE team strongly pushed for, and got politely overruled on. After a code freeze is lifted, it's quite common for lots of promotion-eager engineers to release big changes.
- joeblau 6y agoI have friends at many multi-billion dollar companies who are all just twiddling their thumbs right now.
- amir734jj 6y agoThis is my SignalR alternative with end-to-end encryption. Choose a password and the file and message will be encrypted in client side using that password. URL: https://symmetric-crypto-chat-room.herokuapp.com/ https://symmetric-crypto-chat-room.herokuapp.com/ Repo: https://github.com/amir734jj/SymmetricCryptoChatRoom https://github.com/amir734jj/SymmetricCryptoChatRoom
- rexreed 6y agoI'm migrating to Rocket.chat on Digital Ocean as we speak. Has anyone else made the move or tested Rocket.chat?
- why5s 6y agoMy old company used a mix of Slack and RocketChat. Functionally, it's fine but I was never a big fan of the UI and how attachments were handled. Also, cross-channel search was kinda bad. Mind you, this was well over a year ago so I'm sure things have improved.
- deleted 6y ago[deleted]
- 2mol 6y agoZoom as well, no? Beautiful to see the whole team grinding to a halt. Or maybe everybody finally is getting some time for deep focused work.
- steveharman 6y ago"It was working fine when we sold it to you"
- ShaneMcGowan 6y agostatus page says apps / API are fine yet am trying to work on a slack app but can't cause of these error, a bit annoying edit: it is now showing as a total outage on the status page
- bmhin 6y agoIt seems weird to say there are issues with connections but everything else is working fine. Like is the API technically fine on their system metrics but no one can connect to use it so it stays as green? Doesn't really help much in practice it seems if connections are having issues and everything is unusable in practice to keep them as green. Would be similar if auth was down. You can connect to us, you just can't authenticate so can't actually do anything. Edit: Looks like they updated the status to properly show an across the board outage
- barathvutukuri 6y agoShould I contact Salesforce?
- technick 6y agoYay Salesforce!
- siruncledrew 6y ago$28B well spent!
- iamben 6y agoNotion is down for me (and others) as well. Is there a cloud outage somewhere?
- koenigdavidmj 6y agoOther data points: * iMessage was taking its sweet time sending a few texts this morning. * I had momentary trouble trying to call a business from my Verizon phone, and someone I know had trouble calling from AT&T. Could just be a coincidence, but I wonder if something larger-scale is happening.
- rntrg 6y agoI came here wondering the same thing as I've encountered a handful of availability issues this morning. * Todoist MacOS app is having trouble talking to its API
- lm2s 6y agoYes, I'm wondering that also. Todoist was having issues and iOS app launching from Xcode started taking a lot of time in the middle of the day (which reminds me of the app online check fiasco not so long ago). Even HN seems to be a bit slower.
- davesque 6y agoIt's funny that this isn't considered an "Outage" by their status page's standards.
- HowardStark 6y agoSeems that it is now. It was originally just Messaging and Connections that had an "incident", so I wonder if something else happened or they manually changed the status to at least own that all their services went FUBAR.
- deleted 6y ago[deleted]
- someonehere 6y agoWell, there goes the credibility one team has in arguing that Slack makes a great knowledge repository.
- jdc0589 6y agopeople actually argue this? slack is a great coms tool, and great BACKUP if you can't find something in a real documentation/knowledge/etc.. repository.
- deleted 6y ago[deleted]
- someonehere 6y agoWhere I work yes. Small and scrappy devs who when asked to move knowledge into a Confluence or wiki page, argue that’s it’s too much work to find everything they need. They can just type in the channel they want to search in with a term and get the conversation they need. My response is if it’s important long term, it needs to be somewhere visible and exportable should the platform change. As it is now, Slack exports are horrible and large.
- scrose 6y agoI find it amazing that we can be about an hour and a half into a service being completely unusable(ie. Slack telling me it 'cannot connect'), yet it's still marked as an 'incident' instead of an 'outage' in their own status page
- DarkContinent 6y agoIt's marked as an outage now.
- ghostpepper 6y agoand yet they're still proudly proclaiming: "Uptime for the current quarter: 100%"
- ajkjk 6y agoEvery time this kind of thing happens HNers love to grip about how the status pages aren't correct yet. It's so weird -- like the people freaking out about the outage are going to be updating their uptime trackers right now or something. Who cares? It'll be fixed later.
- Merman_Mike 6y agoI think the point is that a "Status Page" should show the accurate, current status of the system. Not a place holder for "we'll fix it later". People look at a status page to know what's happening now.
- jsteemann 6y agoFrom the status page (https://status.slack.com/2021-01/9ecc1bc75347b6d1 https://status.slack.com/2021-01/9ecc1bc75347b6d1), updated just now: > We're continuing to investigate connection issues for customers, and have upgraded the incident on our side to reflect an outage in service. All hands are on deck on our end to further investigate. We'll be back in a half hour to keep you posted. > Jan 4, 5:20 PM GMT+1
- SkyPuncher 6y agoMy biggest frustration with these outages is they're hard outages across all of Slack. There's no reasonable work arounds or fallback features. A plaintext web interface would keep my team moving along while they resolve their issues.
- zucked 6y agoNothing like a reminder of how dependent you've become on Slack for communication (and archival of conversations) like an outage on the Monday after the holidays when you're not on your A-Game yourself. "Let's see, I'll look up so and so's name with Sla.... shoot" "Okay, I'll just find that thing I .... nevermind"
- itisit 6y agoThe joys of multi-tenancy.
- newman8r 6y agoA good time to host your own slack-like chat with mattermost instead
- keehun 6y agoLooks like Slack just updated their status page to show a complete outage, not just an incident for "Messaging" and "Connections."
- musing-penguin 6y agoIMO it's really poor Slack took 1 hour to update this to an outage, given the impact this seemingly had right from the off. It's also extremely bad that we're 1 hour in, and they are still "investigating", with no more details than that.
- zander312 6y agotime for nap
- tschellenbach 6y agoChat infrastructure at this level of scale is not easy to build and maintain, I appreciate all the hard work that the engineers at Slack are putting in to resolve this.
- tumidpandora 6y agoNow an outage ------------- We're continuing to investigate connection issues for customers, and have upgraded the incident on our side to reflect an outage in service. All hands are on deck on our end to further investigate. We'll be back in a half hour to keep you posted. Jan 4, 8:20 AM PST
- Tistel 6y agoI just got an invite to join my company’s Google chat.
- kamyarg 6y agoAtleast their status page works ¯\_(ツ)_/¯ (Looking at you AWS). I am really looking forward to a better competitor taking over their market share, I presume things will only get worse after Salesforce acquisition.
- nicioan 6y agoFunny how status.slack.com has reported Incidents and Outages for a while now, but still the "Uptime for the current quarter" is reported at 100% on the bottom right of the status table.
- ProAm 6y agoOne of those affects money via SLA's. Slack is still up, just not usable.
- deathanatos 6y ago> Slack is still up, just not usable. I.e., it's down. (And if you're saying that according to the legal blah blah blah of the SLA that this isn't technically "down", then there might as well not be an SLA.)
- ProAm 6y ago> And if you're saying that according to the legal blah blah blah of the SLA that this isn't technically "down", then there might as well not be an SLA. I am because Ive had these exact conversations with cloud hosted providers/products. Never once have we been refunded according to the SLA in our contracts. Never really down (according to legal).
- rocho 6y agoUp means working. It does not mean that something is displayed on the screen.
- MattGaiser 6y agoOther than the status page, I can't get anything displayed on the screen.
- spelunker 6y agoI would think that number will be updated once the fire is put out.
- rexreed 6y agoAny opinions on using Discord vs. Slack?
- FerdSlav 6y agoI personally have found that one of Discords major shortcomings is the lack of support for threaded message chains. For those times when you may have 2+ parallel conversations in a channel you end up dramatically reducing the ability to effectively communicate.
- smurda 6y ago#hugops
- lylo 6y agoThis is an excellent reminder of the danger of being locked into closed systems. I wonder how many companies (like mine) have literally ground to a halt because of this? Do other companies have a risk-documented backup plan B for times like this? Presumably the default is for everyone to resort to email? More worryingly is the number of ChatOps processes and alerting/observability systems that are in place around Slack. Not being able to chat with co-workers for an hour or two is fine, but not being able to safely manage CI/CD/deployments is a big risk.
- deleted 6y ago[deleted]
- Scandiravian 6y agoI just finished an 5 hour debugging session on what turned out to be several cascading bugs in one of the older systems at my company Can't deploy the fix, because - developers trigger deployments through slack and I don't have access to the underlying deployment system - infrastructure guys who have access aren't responding to my emails
- vlod 6y agoSounds like a great opportunity to work on not having Slack as the only trigger mechanism. :) Or a least document how to call the slack bot manually. (assuming it's just a http endpoint)
- Scandiravian 6y agoI agree 100%! Though I think it might be dangerous to prepare for the "last disaster". It'll be some other system breaking next time, so I think we instead should identify what systems that do not have some kind of redundancy and determine blast radius of those crashing I'm good at not panicking about things I can't change, but I worry about some of my colleagues who find it difficult to not have control in these situations I can't do anything to help them at the moment, so for now I'm heading to my couch with my analogue book :)
- 6y ago
- heroHACK17 6y agoJust wanted to come here and say, hey! How is everyone doing? How was your Holiday break?
- thismodernlife 6y agoI wondered about getting credits for the outage but you can't view the SLA page because the app is down. https://slack.com/intl/en-gb/terms/service-level-agreement https://slack.com/intl/en-gb/terms/service-level-agreement
- nikolay 6y agoCan't handle the post-holiday surge, or people wanted to justify their long holiday and pushed something only to witness their holiday optimism head-crashing on the surface of the reality?
- keehun 6y agoAssuming this is a bad deployment--not hardware/network issues: It will be interesting to read their post-mortem, on why rollback still has not happened yet after 2 hours of outage. You would hope that a service the level/popularity of Slack would plan for deployment-related outages and be able to roll back a deployment.
- davesque 6y agoConsidering how ubiquitous slack use seems to be in a lot of major tech companies, I wonder if it's reasonable to ask whether or not the stock market's performance this morning is somehow correlated?
- bengale 6y agoNot a great first day back for their ops team.
- pwned1 6y agoThis site seems to be lagging as well. Or is it just me?
- MattGaiser 6y agoNot just you. It is slow for me as well (Calgary, Canada).
- ChrisMarshallNY 6y agoMe too (New York).
- fleaaa 6y agoSame here (Berlin).
- stefan_ 6y agoIt's struggling. All the people freed up that think chatting in Slack is productivity?
- alexiacob 6y agoSame here (London)
- rootusrootus 6y agoIt's quite fast for me. Edit: Not consistently, I guess. 9 out of 10 times it responds instantly, then it lags once in a while.
- MattGaiser 6y agoIt is the replying that is particularly slow for me.
- deleted 6y ago[deleted]
- Scandiravian 6y agoSeveral sites and services are lagging or slower than usual for me (facebook messenger, news sites, google)
- ChrisMarshallNY 6y agoI don't use Slack that much, but I know plenty of people that work on teams that are probably at a standstill, right now. HN is also pretty slow...
- dkarl 6y agoI think there are a lot of sites that would benefit from automatically scaling up whenever Slack goes down.
- CobsterLock 6y agoi noticed the HN slow down too. maybe it isnt only a slack issue
- MattGaiser 6y agoMy company runs heavily on Slack. Part of my team got together in a video chat, but I have no idea what happened to everyone else in the company.
- Deukhoofd 6y agoHN becomes slow because people notice a service is down, and go to HN to check for more info. When Google was down for an hour a couple weeks ago, HN became almost unusable.
- xkeysc0re 6y agoLooks like my vacation continues!
- paulintrognon 6y agoIt's already being discussed here: https://news.ycombinator.com/item?id=25632346 https://news.ycombinator.com/item?id=25632346
- sytse 6y agoAt GitLab our fallback from Slack is Zoom https://about.gitlab.com/handbook/communication/#emergency-chat https://about.gitlab.com/handbook/communication/#emergency-c... I'm posting this because I found a lot of people don't know that Zoom includes a complete chat client that includes channels. And #HugOps to the engineers at Slack working on this. I appreciate that they posted a periodic update even when there was no news to report: "There are no changes to report as of yet. We're still all hands on deck and continuing to dig in on our side. We'll continue to share updates every 30 minutes until the incident has been downgraded."
- bww 6y agoJust a reminder that it's probably not a wise idea for anyone to get further in bed with Zoom than they already are. https://www.washingtonpost.com/technology/2020/12/18/zoom-helped-china-surveillance/ https://www.washingtonpost.com/technology/2020/12/18/zoom-he...
- _6fmb 6y agoWhat an overwrought headline, the employee in question has already been fired.
- fermienrico 6y agoThe optics are still very, very bad for Zoom. I have zero trust in them.
- noir_lord 6y agoThere are remarkably few organisations I somewhat trust (even then on a sliding scale) but on that spectrum Zoom sits at the "wouldn't touch them with someone elses bargepole" end.
- criley2 6y agoIt's weird that you describe the headline as "overwrought" and call the person an "employee" when the headline is more accurate than you. This was an executive, not just an employee. That's a huge distinction and I can't help but think you intentionally downgraded his position to cover-up his behavior. "Just an employee" "Not a big deal" But when you read the allegations, they seem like a very big deal that an executive was spying on users, giving their information to the Chinese government explicitly for oppressive purposes, including folks who are not in China, and went out of his way to personally censor non-Chinese groups meeting to discuss the Massacre-Which-Cannot-Be-Mentioned. I would say the headline understates the gravity (it's very much a 'by-the-books' headline that you KNOW went through ten levels of Legal), and that your hand waving here feels much more dishonest than the headline.
- mariusseufzer 6y agoAnd there we have it: Relying on big companies sucks. It's great as long as it works. Once a system breaks thousands, or even millions, of businesses suffer. (Of course they are also beneficial and a private server can also crash at any time + I don't wanna blame Slack, but we always have to keep this in mind).
- deleted 6y ago[deleted]
- yreg 6y agoIf a big company has million customers and the big company experiences an outage per quater, then a million businesses suffer every quater. If a thousand small companies have thousand customers each. And these small companies experience an outage per quater, then a million businesses suffer every quater. As the end-user-business, is it better to suffer the outage at the same time as other businesses? Is it worse? Surely there are valid arguments against relying on big companies, but I don't think this is one of them.
- beagle3 6y ago> If a thousand small companies have thousand customers each. And these small companies experience an outage per quater, then a million businesses suffer every quater. Not all companies are created the same. Microsoft, Google and Facebook have had their outages, but IME much fewer than Slack. If there are a thousand small companies, none of them have a network effect, and those that experience more outages per quarter will lose customers to those that have less outages per quarter. So they have much more incentive to improve. Whereas network-effect beneficiaries like Facebook (and to a lesser extent, Google, Microsoft and Slack) have much less of an incentive to improve. Who else would the customers go to?
- 00adefff574 6y agoHopefully it stays down forever
- mey 6y agoWhy does the slack client not show connection issues instead of just hard locking up?
- bengale 6y agoThe client on my Mac showed a page that said it was having issues connecting with a link to their status page.
- mey 6y agoThe Windows desktop app is less fortunate.
- deleted 6y ago[deleted]
- jaywalk 6y agoWhen it went down fully and I had the Windows client open, it went to a page that basically said "Slack is down, we don't know why, try restarting and see if that fixes it. Here's the status page." It would be nice if they could fix it so that a fresh start also goes to that page, at the very least.
- djmetzle 6y agoShould be an interesting post-mortem...
- ryanSrich 6y agoThis is actually impressive, in a bad way. I just have become so used to being able to run highly resilient cross region infrastructure for millions of users with just a handful of people that I forget what real downtime looks like. For their app to just go completely offline is unacceptable. Bugs and degraded services I get. But this is catastrophic.
- MuffinFlavored 6y agoI can't even begin to guess what went wrong. What are your guesses? How many screaming executives are there at Slack saying "just roll it back"?
- glouwbug 6y agoWell, they did hand off slack to Salesforce
- aphextron 6y ago>I can't even begin to guess what went wrong. What are your guesses? How many screaming executives are there at Slack saying "just roll it back"? Doubtful it's a code issue causing a total system outage. I'm assuming they have a bunch of auto scaling infrastructure that wound down over the holidays and couldn't take the spike this morning.
- eecks 6y agoMass server migration?
- unethical_ban 6y agoI am genuinely surprised that Slack wasn't ready for people to come back from holiday, to view increased queues of unread messages, to have to manually login vs. having auth tokens or cookies, etc. Either that, or they had a cosmically coincidental outage on a really bad Monday to have it. It's bad enough team comms go over Slack so much now, at least we have email fallback. What scares me is for the teams that use Slack for system alerting.
- djxfade 6y agoWe get our primary alerts through Slack. However we also have SMS and phone call backups through PagerDuty
- djsumdog 6y agoWe have an alert channel in Slack, but it's mostly ignored. Our primary alerts come via SMS/VictorOps. At one of my old jobs, we had SMS via two physical/hardware devices in our data center. One had a Telstra SIM card and the other had an Optus SIM card. (They were plugged into the same machine, but we had plans to put a second one in another data center before I left). If you really care about alerts, you should have physical hardware doing your SMS messages via two different point-of-presences.
- CSDude 6y ago(from Opsgenie) I would imagine it would be the other way around for most people.
- dysfunction 6y agoMy coworker's theory was someone was waiting for the holiday's end to deploy something risky. And I'm in that boat of depending on Slack for alerting... in fact my team was also waiting over the holidays to deploy more robust non-Slack-based alerting (in our defense the product is only a few months old and only now starting to scale to any real volume).
- hinkley 6y ago
- brayhite 6y agoNotion is sluggish as well. That combined with the reports of HN potentially being slow, is there some larger network issue at play affecting a region of servers, potentially?
- realrocker 6y agoSame for me. Data is missing in tables too.
- coldcode 6y agoMy feeling is some common infrastructure is failing or flailing, like some part of AWS, or some backbone provider. Too many flaky things going on at the same time to be independent failures.
- coldcode 6y agoMaybe another internet routing issue, where a bunch of traffic is going through some guys router in Albania. Or even someone is actively interfering with a root server.
- joana035 6y agoThere are many reports of issues with ec2 and console on down detector, doesn't surprise me that aws status page is still green.
- jmartens 6y agoMy company monitors EC2 performance and availability across North America, and EC2 has been fine this morning, according to our data (that said, they had some intermittent issues the last 3 days).
- krisdol 6y agoLever has been down for about the same amount of time as well (job recruiting platform).
- glouwbug 6y agoTime to slack
- peter_d_sherman 6y agoFor some reason, today, HN seems exceedingly exceedingly slow (to me) after logging in... Without being logged in, things are as fast as they usually are -- but post log-in, SLOWWWER THAN MOLASSESS... I tried this several times; why this is, I can only wonder... To quote Bill and Ted... "Strange things are afoot at the Circle-K..."
- hartator 6y agoEveryone switched from Slack to HN. :)
- erk__ 6y agoIt is easier to cache stuff for users who are not logged in as it is the same for everyone. and everyone is looking up on Hackernews at the moment to see what is wrong with slack, which is probably the cause of the slowness.
- manquer 6y agoWhile it is true for most applications. HN does not do any customisation of the content. I don't notice I am not logged in until commenting
- tayo42 6y agoYou would cache the rendered html, the front page has your username and points and stuff. The whole page will be unique to you because of that
- tyingq 6y agoThe point count for most articles is consistently lower on a view of the non-logged-in homepage. I assume that means they are cached more aggressively for non-logged-in. There's also the username and karma count in the top-right.
- reaperducer 6y agoWhile it is true for most applications. HN does not do any customisation of the content I don't think that's true. For example, if you hide a thread while logged in, it remains hidden when you return.
- aliljet 6y agoSlack has been a uniquely iffy service. I wonder if there's a solid decentralized alternative.
- fwip 6y agohttps://cabal.chat/ https://cabal.chat/ is a good program. It does not support all of slack's features, but is truly peer-to-peer so there's no central points of failure or servers that can go down. (Well, I suppose if they released a buggy version of the software and you updated, that's a central source, but that's true of most software.)
- cuspycode 6y agoI've had good experiences with ngircd. It's an IRC server that is very easy to self-host, and it can be installed via APT on any debian/ubuntu/raspbian etc system, and I'm sure on many others.
- dhbradshaw 6y agoWe just had to route around this so we're trying out chat.google.com for the first time. Seems ok.
- king_magic 6y agoSlack has been failing - hard - the past few months. Yeah, I get it, lots of remote workers - but Slack has had months now to prepare for an onslaught given the trends with COVID. Simply not acceptable.
- ryanisnan 6y agoWhile Slack is down, let's remind ourselves that it is not the end of the world. To their ops team, good luck in sorting out the root cause(s), to mitigating their re-occurrence, and to emerging the other side a stronger team. You've got this.
- ghostpepper 6y agoHopefully they have a backup system for internal comms
- manquer 6y agoIt is not end of the world if you are just using slack for intra team communications. However lot of the monitoring which alerts on slack and other automatic notifications are critical for many teams.
- bombcar 6y agoCritical systems dependent on another system are just as reliable as the third-party system; so this may be a good wake-up call for many.
- hn_throwaway_99 6y agoJust a note, if your company uses G Suite, chat.google.com exists and is basically an entire Slack clone. We use it as a backup when Slack goes down (obviously doesn't help for bots and ChatOps we've set up, but works well for realtime work chat).
- hacker_newz 6y agoCalling it a clone is a stretch. There are a ton of features missing.
- twistedpair 6y agoOur org just failed over to GChat as well. Piece of cake. Quite glad we never moved any critical ops work into Slack bots, since we don't control Slack.
- ARandomerDude 6y ago> Customers may have trouble connecting or using Slack I can't stand how marketing speak pervades every sphere of the world. Their entire system is offline (inconvenient certainly, but it happens) and they can't bring themselves to say "Slack is down. We're working on it and will be back ASAP." or something similar. Instead we may have trouble.
- jeffbee 6y agoHow do you know it's down completely? Maybe it's down for you and maybe even down for a majority but still up for some subset. Happens with many products.
- ARandomerDude 6y agohttps://status.slack.com/ https://status.slack.com/ Every service is marked as "Outage" as of now (also when I wrote the comment).
- gog 6y agoThe outage is not for everybody, I can connect.
- politician 6y agoThundering herd. If you can avoid it, don't connect.
- jonwachob91 6y agoIt's been a rollercoaster for me the last few hours, sometimes servers are up sometimes they are down. Point being, they are intermittently up :/
- ellisv 6y agoyet also "Uptime for the current quarter: 100%"
- lordnacho 6y agoI'd be a little nervous if I'd recently bought Slack for $20B. It's not like there aren't alternatives. You could even imagine someone has a live bridge between Mattermost and their Slack team, making the switchover seamless.
- joana035 6y agoLots of services quite red on https://downdetector.com https://downdetector.com
- ceejayoz 6y agoDown Detector largely seems to track daily workplace usage patterns more than meaningful outages.
- joana035 6y agoWhich is great to detect common issues across many companies. For example, clicking on the cards shows that many of them are related to "network connection".
- ceejayoz 6y agoIt's not, though. They're all spiking because everyone got back from the weekend. Just look at the comments. H&R Block's page there has this as the most recent comment, from an hour ago: > My sister was able to have a bank pull money off her card yes her old card dunno which bank ill find out in bit Two hours ago: > I went to atm and thought I was crazy my pin wasnt working. These reports are entirely useless.
- jmartens 6y agoUnfortunately, Down Detector doesn't actually monitor these services, so we don't know if they are truly down. Down detector relies on human behavior, and we all know humans don't act rationally.
- MR4D 6y agoInteresting that PG&E is on the list for power outage in SF. Wonder if that's related? https://downdetector.com/status/pge/ https://downdetector.com/status/pge/
- joshxyz 6y agoEarly 2021 downtime jeez good luck ops team i believe in you
- kmichler 6y agoWe're using Google Chat again. Feels ancient. https://chat.google.com/ https://chat.google.com/
- adrianpike 6y agoSame - and it's just... weird. The "everything must be in a thread" model feels really clunky.
- kmichler 6y agoThat's a good point, however threads do tend to help with keeping things organized when in a channel. It actually might be a good thing that everyone doesn't feel the need to look at slack every X minutes.
- politelemon 6y agoSubjective... I've found Slack and co's interspersed conversations far too chaotic, and temporal; threading is a great way of organising many different concurrent topics. And to be clear I don't mean Slack's implementation of threads which is hiding it away in a separate panel and which doesn't get used by everyone either.
- johnc1231 6y agoI find Zulip to be a nice balance. Threads are much more prominent than they are in Slack, but aren't clunky.
- igetspam 6y agoIf you're using gsuite already, it's a usable failover. I send all my alert notifications there, as a fallback already. Dragging people in was trivial. It's better than the group SMS that one person tried to use.
- deleted 6y ago[deleted]
- tuckerpo 6y agoTime to talk to my coworkers in person... dry heaves /s
- forgetfulness 6y agoI'm fairly certain that dry heaves are not a COVID-19 symptom, at least.
- cadence- 6y agoSlack is down. This is shaping up to be my most productive day in a long while.
- anonymou2 6y agoEveryone back to IRC!
- betaby 6y agoRelevant https://blog.apnic.net/2020/09/21/why-irc-is-still-good-in-current_year/ https://blog.apnic.net/2020/09/21/why-irc-is-still-good-in-c...
- beamatronic 6y agoI’m going to get so much work done today
- blntechie 6y agoI have always had this fantasy thinking of what happens when outages of one of these major service never come back online i.e. in this outage Slack loses info of all the accounts, users, messages etc. How would people react? What would engineers do to recover? I always found that idea fascinating. Imagine Google saying tomorrow that they lost all the accounts and emails. What kind of impact the world will have?
- MattGaiser 6y agoGoogle would be catastrophic because so much is stored there. Slack is mostly real time communication, at least for me. There are a few bits and bobs that really should be documented that are in the messages though.
- blntechie 6y agoYeah, Google would easily top the list of companies which can have catastrophic impact. Microsoft, Apple, Salesforce, Dropbox would be the next in the list I guess if we leave out the utility companies and internet providers etc.
- jon-wood 6y agoJust look at the impact a 40 minute outage of Google Auth had last month, I wouldn't be surprised if the global productivity hit during that outage was in the billions of dollars, and that was for a relatively short outage without any data loss.
- sjg007 6y agoAWS outages have basically crippled a few businesses. The longest I know of was 8-10 hours the day before Thanksgiving. Some Bay Area food company got hit by it and couldn’t deliver thanksgiving dinners.
- thesuitonym 6y agoIf this thread is to be believed, apparently a lot of engineers use slack for alerting and don't know how to check their monitoring software manually.
- lawrjone 6y agoHaving been in this situation before, with a totally-down-and-not-coming-back-up outage of a payments system, I really feel for their incident response team. I'll take this moment to remind everyone of their human tendency to read meaning into random events. There's no evidence to suggest New Year traffic has caused this, and outages like this can happen in spite of professional and competent preparation. Hugops for their team, I hope they get it back soon.
- yjftsjthsd-h 6y ago> I'll take this moment to remind everyone of their human tendency to read meaning into random events. There's no evidence to suggest New Year traffic has caused this, and outages like this can happen in spite of professional and competent preparation. On the one hand, sure we don't specifically know what's going on. On the other hand, it's the first Monday in the new year and they went down shortly after the start of the business day Eastern time; it could be coincidence, but it would be a remarkable coincidence.
- lawrjone 6y agoThere are a load of ways NY might have contributed to this, but it may not be a direct cause. What's more likely, Slack forgetting to scale their deployment back up after too much mulled wine, or a number of people on holiday meaning a simple failure has developed into something more serious? It could be anything really- my post was more about how situations like this can happen to even the most prepared. The assumption it has something to do with NY tends to assume very trivial, silly mistakes. Especially with no information, that seems a bit uncharitable.
- taneliv 6y agoIt seemingly worked ok in UTC-2 in the morning and early afternoon, then started having issues and is now a bit intermittent (or fixed, there's not much traffic on my channels, as it's evening already). Do they have that much more traffic on US east coast than in Europe?
- 6y ago
- deleted 6y ago[deleted]
- ilkkao 6y agoI just hope slack itself has a backup chat tool for incidents like this.
- Xmax 6y agoSurprisingly mobile client is working for me..!!
- djsumdog 6y agoIt's off an on. The connectivity is spotty.
- bricss 6y agohttps://status.webex.com/service/status?lang=en_US https://status.webex.com/service/status?lang=en_US
- Xmax 6y agoSurprisingly mobile client is working for me
- Justsignedup 6y agoI'm setting up backups for our company on discord. That way maybe some webhooks won't be working, but communication resumes.
- jastingo 6y agoAnyone else having a fantastic morning/afternoon without the constant pinging? Just saying - there's always a silver lining.
- jaywalk 6y agoIt has made coming back from a long Christmas vacation a lot easier. Once I got my emails taken care of, I was able to get to work without distractions. It's been nice.
- eastbayjake 6y agoIt's a nice opportunity to go for a quick walk!
- mam2 6y agoWhy would you use cloud for that ? We have mattermost at my startup
- exabrial 6y agowe run a copy of this as a super duper backup: https://github.com/shazow/ssh-chat https://github.com/shazow/ssh-chat
- lwedel 6y agoas an alternative - Cisco has - webex, less known, but does the work: https://status.webex.com/service/status?lang=en_US https://status.webex.com/service/status?lang=en_US
- exabrial 6y agoFor a product that is so simple, there are no good self-hosted alternatives. Mattermost and RocketChat are written very poorly, reliability and getting your data out is impossible. Slack goes down so often we're thinking of writing a very boring clone that uses ActiveMQ and MySQL, just because chat should be boring and needs to "just work".
- ginja 6y agoI was just considering setting up a Mattermost instance for our company since I used it for a year at a previous job without any issues (I was just a user though, I didn't deploy or maintain it). Just curious, why do you think it's poorly written or unreliable?
- exabrial 6y agoWe tried running it, so a lot of experience with it, and it wasn't great. It barely stayed online. For something so simple, you have to run a massive server, like gigs of ram and multiple core, even with a very modest user load. Take a look at the codebase, it's also a mess and impossible to fix any bugs. Finally, if you want to get your data out or report on the message activity, good luck, you'd be better off passing paper notes around. The open source version is nerfed a bit too, no LDAP authentication for instance, so it creates a lot of problems there too.
- wrycoder 6y agoZulip can be self-hosted - have you looked at that? I like the threads implementation.
- eecks 6y agoMaybe your assumption about it being so simple is incorrect
- waihtis 6y agoObligatory BGP hijack prediction, since there seems to be a bunch of other sites down too.
- ririyad 6y agoNotion is also down right now.
- marricks 6y agoDoes Google use slack? Wanted to start my year with some extra strength tinfoil and it'd just be great if the day a unionizing initiative started the major way workers could talk about said unionizing initiative went down. EDIT: according to a random quora post they do, so keep the tinfoil out!
- orthecreedence 6y ago"It will replace email."
- minitoar 6y agoI mean...wasn't gmail (which effectively IS email for many, many people) down recently?
- jolmg 6y ago> which effectively IS email for many, many people Doesn't have to be, though. One person doesn't even have to tie their address to a single provider, and seeing past received messages doesn't even need internet connectivity.
- harikb 6y agoCan we please trade in centralized Slack and single-sign-on and get back the netsplit of IRC :) ? At least I can chat with half of my colleagues :)
- kuon 6y agoThis is like the first time in a year I need slack, call that bad luck. I hope matrix/element will rise more.
- jennyyang 6y agoCould this be some sort of data corruption? I find it hard to believe that Slack could be down for this long without something that is exceedingly hard to rollback. Even if some services are completely overwhelmed with traffic, they could block a certain percentage of traffic to decrease load, and then force servers up across their datacenters and then unblock traffic. It has the hallmarks to me of some sort of datastore is down, but obviously just a random guess.
- zaptheimpaler 6y agoIt hasn't been that long and lots other web services are behaving a bit strangely or are down as well - https://downdetector.com/ https://downdetector.com/. So its probably a wider issue affecting everyone - network level is my guess.
- tsar_nikolai 6y agoTodoist reports that is down as well [0] I wonder if it would be connected in any way shape or form https://status.todoist.net https://status.todoist.net
- deeblering4 6y agoI wonder if this is one of the larger natural drops/spikes of legit users that their infrastructure have seen? * lots of users are coming back to work after the holidays today * lots of users take the holidays off and fully disconnect * significant new users added in 2020, with so many teams going remote Sounds like a possible recipe for infra scaling issues and/or cascading failures to me
- hivacruz 6y agoNever deploy a new release on Friday and Monday!
- api 6y agoSalesforce bought Slack, so maybe it's enterprise now.
- valevk 6y ago> While the issue is largely still ongoing, we believe some customers may see improvement in connecting to Slack after a refresh (CTRL/CMD + R). Nice.
- fermienrico 6y agoSlack/Notion. Stop with the features already. You're killing yourself in slow motion. Focus on performance, robustness of your infrastructure.
- decisionsmatter 6y agoJust seems slightly disingenuous to me to have "100% uptime" on the same page that says there is a current major outage.
- ajakate 6y agoEspecially when it's "uptime for the current quarter." A three hour outage since Jan 1 is already a 3% downtime automatically I would think...
- papito 6y agoHere is to hoping it's not SolarWinds related eh!
- Trex_Egg 6y agome too
- the_duke 6y agoA great opportunity to try out Element, an open source client for the federated and open Matrix network [1]: https://element.io/ https://element.io/ (edit: to clarify: not affiliated in any way, just a fan) [1] https://matrix.org/ https://matrix.org/
- julianlam 6y agoI've found that element.io itself is really slow to load. That said, we're pushing up against the limits of our free plan with Slack and will likely deploy a matrix server in due course.
- xfalcox 6y agoThe good thing about that is that if you want a fast client there are quite a few native clients to pick. https://matrix.org/clients/ https://matrix.org/clients/ For example, Mirage is Python/QT and quite fast in my experience. There are Rust clients, C++ clients, terminal based ones, etc.
- Shared404 6y agoDo you happen to know of any desktop clients that support encryption/cross-signing? I'd like to get off of Element desktop/web for a couple of reasons, but I need those features. I'd help implement them myself, but that's beyond my skill level. Edit: For anyone else wondering, matrix-commander [0] looks like it may be workable if a cli tool is acceptable for your usecase. [0] https://matrix.org/docs/projects/client/matrix-commander/ https://matrix.org/docs/projects/client/matrix-commander/ I'm planning on looking through the GUI ones at some point, but don't have time now.
- the_duke 6y agoFluffychat [1] is built with Flutter and apparently supports e2e encryption. Note: I wanted to try it out for a while, but haven't yet. [1] https://gitlab.com/famedly/fluffychat https://gitlab.com/famedly/fluffychat
- elyseum 6y agoSlack down, productivity up!
- antisthenes 6y agoService Interruption as a Service.
- _nickwhite 6y agoI read somewhere that Slack's yearly uptime SLA is 99.99%, which has already been exceeded on January 4th. Sending big hugs to their ops team.
- mahdyarhp 6y agoGive Telegram a chance. It's worth it! telegram.org
- jamespwilliams 6y agoDo you recommend it for use by teams?
- cheschire 6y agoPACE = Primary, Alternate, Contingency, Emergency If you haven't been able to justify testing your PACE plan with your bosses lately, now's a great time to go ask again.
- clubdorothe 6y agowhich tool did they use to communicate during the outage?
- mugivarra69 6y agooof.
- offtop5 6y agoLike human beings, can we all imagine that every single service can have a couple of off days a year.
- irrational 6y agoMy business coworkers are freaking out over Slack being down. But all my technical coworkers are nonplussed. It's interesting how those of us with a technical background are not too disturbed by things breaking.
- mepiethree 6y agoI'm "plussed," because an app that I manage uses slackclient, and some people depend on it to get paid. Obviously it's my fault for not handling the error, and I hotfixed it, but still, wah.
- tmoertel 6y agoInterestingly, nonplussed is one of those words having two meanings that are at odds with each other. According to Google, those two meanings are: 1. (of a person) surprised and confused so much that they are unsure how to react. "he would be completely nonplussed and embarrassed at the idea" 2. INFORMAL•NORTH AMERICAN (of a person) not disconcerted; unperturbed.
- irrational 6y agoI wasn’t aware of definition 1. I did mean the informal North American definition.
- wrycoder 6y agoI'm 79, born in the mid-west, living in New England for the last 60 years. Worked in tech. I've never heard the word used except as in Def. 1.
- irrational 6y agoI've never lived in the mid-west or the New England region of the USA. Maybe it's a regional usage (I've lived in Florida, Texas, California, Colorado, Utah, Oregon, and Washington). I'm not sure where I picked up my usage from. My dad is from Colorado and my mom from California. Maybe I picked it up from one of them ;-)
- Havoc 6y agoPotentially an AWS issue? Slack, notion and AWS all at same time seems unlikely https://downdetector.com/status/aws-amazon-web-services/ https://downdetector.com/status/aws-amazon-web-services/
- oriettaxx 6y agoI am so glad that at least today I do not hear that slak annoying sound. I do really think Slack is not helping me, at all, to concentrate on my job (system administrator): synchronous messages are really the worst, ever, while working: email is much much better
- AnIdiotOnTheNet 6y agoHonestly I'm fairly sure the vast majority of "technology" we've deployed, as an industry, in the past 10-15 years has actively made life worse. I don't know about anyone else, but that's the opposite of why I got into technology.
- not2b 6y agoI'm able to connect to Slack at the moment. My company doesn't use it, but a hobby group I belong to uses it for discussion forums and their instance is up and functional. So it isn't down completely as I write this.
- nicioan 6y agoLet’s take a moment and express solidarity towards the fellow engineers that are currently working like crazy under a lot of stress to fix this.
- dustymcp 6y agoWe got a discord server as a backup when teams is down, seems like it has gotten worse with days being down entirely, and we have to resort to discord voice which always seems to be up..
- paulcarroty 6y agoGood reason to try matrix or even setup a reserve (matrix) channel.
- Cuuugi 6y agoSeems to be back up for me? Status page has not updated yet though.
- killjoywashere 6y agoI noticed something about dead slack channels on the GCP console last night, which I thought was odd. Anyone see something similar?
- abanayev 6y agoI feel obligated to mention this, which was posted a mere 8 days ago. https://news.ycombinator.com/item?id=25550685 https://news.ycombinator.com/item?id=25550685
- thesuitonym 6y agoI'd like to take this moment to mention self-hosted, open source, and federated alternatives like XMPP and Matrix. I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all the Slack stuff--at least not that I'm aware of. I'd love to be proven wrong there. I know it can be done, but if it can't be deployed quickly by an already overstressed team member, what chance does it have?
- pachico 6y agoI can hardly see those as alternatives to Slack. Maybe https://mattermost.com/ https://mattermost.com/ is what you were thinking about?
- thesuitonym 6y agoMatrix with Element (Riot) as the front-end is pretty close. It does what slack does, it's just not very good. XMPP is arguable. It can be a Slack alternative, if you stitch enough other servers on top of it. Personally, I don't think XMPP will ever be more than chat, but some of its adherents believe differently. Mattermost is certainly not what I meant. That's just trading one Slack for another.
- PhilippGille 6y agoThis thread is about a Slack outage, which you have no control over. Mattermost and similar software is self-hosted, which of course doesn't mean you're getting 100% uptime, but you have (more) control over it.
- perryizgr8 6y agoIn practice self hosted usually translates to more downtime and slower performance when it works. Unless your org has more expertise running a chat service than Microsoft or slack, your self hosted alternative is always going to suck more.
- aledalgrande 6y agoI think this is due to AWS. Not only Slack is down (e.g. Notion). AWS status page didn't show anything yet, but wouldn't be the first time. The last Kinesis crisis didn't show up for hours.
- jll29 6y agoJust a note to say "thanks" to the Slack team for the uptime when Slack is not down, it's been incredibly useful as a tool to me when other enterprise systems (Teams, Outlook & co.) have been down over the last couple of years, and especially throughout 2020. Somehow Slack is very resilient in general. I also appreciate its UX/UI being far superior to Teams. Ultimately, the cloud is often a single point of failure that companies become over-dependent. So I'd favour a free (as in freedom) and open source self-hosted/deployed alternative if there was one (even if it was from Slack and for pay). I agree with most on here that there isn't such a thing yet - but it's well worth building! So those of you out there who are considering implementing "yet another text editor", maybe this is something to work on.
- dang 6y agoAll: large threads are paginated, especially today when our server is steaming. Click More at the bottom of the thread for more comments, or like this: https://news.ycombinator.com/item?id=25632346&p=2 https://news.ycombinator.com/item?id=25632346&p=2 https://news.ycombinator.com/item?id=25632346&p=3 https://news.ycombinator.com/item?id=25632346&p=3 (Yes, these comments are an annoying workaround. Their hidden agenda is to goad me into finishing some performance improvements we're badly in need of.)
- korethr 6y agoWell, they're mostly back up now. I'm curious to see what will come of the postmortem, and if that report will be made public, even if only in part.
- joshuaellinger 6y agoAnd... it is back (for us at least.)
- louiechristie 6y agohttps://twitter.com/louiechristie/status/1346213038924427265?s=20 https://twitter.com/louiechristie/status/1346213038924427265...
- louiechristie 6y agoBreaking news: Productivity hits sky high today as tech workers forced to work at home and not use Slack. https://twitter.com/louiechristie/status/1346213038924427265?s=20 https://twitter.com/louiechristie/status/1346213038924427265...
- holler 6y agoIf anyone is looking for an alternative way for fast and seamless chat with colleagues, friends, or strangers, you're welcome to check out Sqwok (https://sqwok.im https://sqwok.im) Although it's built as a live news discussion site versus a team messaging app, the topics can be about anything, are public, and inviting others is as simple as sharing the url of the post (mobile/desktop web). Example (reposted this hn post to sqwok): https://sqwok.im/p/Q3-1AZFLCSpjew https://sqwok.im/p/Q3-1AZFLCSpjew
- koreanguy 6y agowhat's the point of slack?, when you have irc
- tsjq 6y agoSo many large scale downtimes across multiple large companies in the past month or so. Is this for a bugfix deployment for the SolarWinds hack, or downtime caused by the hack itself ? Or some state-sponsored orgs installing upgraded eavesdropping stuff ?