7 ms·

At least Github's status reporting is accurate and generally truthful. At my current job, engineers are totally allergic to reporting incidents, especially to t

At least Github's status reporting is accurate and generally truthful. At my current job, engineers are totally allergic to reporting incidents, especially to the public status page, since doing so would put you in the spotlight of upper management and make you a prime candidate for a negative performance review in the next cycle. Happens time and time again. The result? Things break, no one takes responsibility, and customers complain while we lie and tell them everything's fine. Shrug.

stigi 3y ago
Wild guess: Are you working at Sendgrid?
- brodouevencode 3y ago
  AWS is also an acceptable guess
  - clnq 3y ago
    99.5% of other tech companies are also acceptable guesses. Some will only inform users of service disruption after the media picks it up.
whitemary 3y ago
It's almost like every company is its own little authoritarian regime. Hmmmmm, maybe Karl Marx was onto something...
- runlaszlorun 3y ago
  Ha, I’ve never read Marx myself but that’s a great viewpoint. And it’s on my list.
  - vorpalhex 3y ago
    Marx actually says the opposite. People form a dictatorship (dictatorship of the proletariat) to engage in a transition period away from Capitalism. Saying "the people own" is just a way of saying the state owns something. The state owns all the things under Communism. Every business is a state business.
    - FpUser 3y ago
      >"The state owns all the things under Communism." The state owns nothing under Communism as Communism assumes no state, social classes, private property and money. I do not think Communism can be achieved though, maybe in prehistoric society of docile gatherers of which we are not.
      - mordae 3y ago
        Perhaps eventually, after we learn to live better. But that's no reason to give up and make greed mandatory.
        FpUser 3y ago
        People are greedy to a various degree by nature. Same for violence and desire to control other people. This is in our genes. I have no idea how you could suppress it without The State (comprised of control freaks willing to use violence) being on top
        rambojohnson 3y ago
        are you a geneticist? got an sources to back up that claim, that we are all inherently selfish? take a critical look at the society you were raised in.
        FpUser 3y ago
        I am not and I do not have to be. One can see it plain all around. I am very critical of the society but it does not change a fucking thing.
- ALittleLight 3y ago
  Yes, if there's one thing Marxists are known for it is definitely avoiding authoritarian regimes. Oh, apart from literally every country that tried Marxism.
  - pedantismo 3y ago
    [flagged]
  - baremetal 3y ago
    Oh but "true Marxism has never been tried", they will retort.
    - andrewaylett 3y ago
      Well, it hasn't. Because humans aren't capable of true Marxism. (This is not a suggestion to try anyway, nor a suggestion to eliminate humans in order to further any kind of Marxist "ideal")
      - mordae 3y ago
        Maybe, but so they aren't capable of living in a 100% transactional capitalist dystopia. As evidenced by everyone popping neurol (or another coping mechanism) with ever increasing frequency.
        adventured 3y ago
        There is a huge difference still. At 3/4 Capitalist, your nation can prosper dramatically. At 3/4 Communist, you will know nothing but suffering and deprivation. Countries that have gotten closer to a free market orientation have prospered. Singapore, Hong Kong, the United States; all of the success stories of post USSR Europe are stories of market economics winning (including the dramatic example of the Baltics). All of Scandinavia operates via market based economic systems. Germany is a market based economy, not a Socialist economy, and they're the economic dynamo of Europe; the same goes for Britain, and even France (which certainly has a large welfare state, and is a mixed economy but not Socialist). There isn't a single prosperous Socialist nation in the world today. There are dozens of prosperous market based economic systems by contrast (the top several dozen most prosperous economies are dominated by market systems). Going the other direction, the closer you get to Socialism, the worse things get. And there are obviously limits to the market direction, anarchy does not work (although I'd argue that's not the market direction; market based economies in particular require the rule of law and a government to enforce it). The only country that can be argued to stand apart from this, even if temporarily, is China. However they're still drifting on the liberalizations of Deng and the benefits it spurred, that is grinding down under Xi. Their situation has worsened as they have headed back toward Socialism (and its inevitable increased authoritarianism) and away from market liberalization.
- bheadmaster 3y ago
  I don't think ownership (owning vs. working class) itself is the problem here. You can see similar behavior (hiding issues, lying about them) in any scenario in which punishment is tied to the failure to meet the metrics. When the metric becomes the goal, people will game the metric in order to avoid punishment and/or reap the rewards. The only kind of measurement that doesn't cause this behavior is the neutral measurement - measurement which is guaranteed not to change the decisions directly, but only serve as information which is later used in high-level optimization of systemic processes. In context of software companies, it would mean that status reports should only be reviewed in combination to other factors, such as tool used, features implemented, deadlines tightened, etc., and only to further understand how to optimize the process to avoid human error. It's risky to implement that, because you have to trust your employees enough to say "you won't be fired because of your metrics, ever", but I believe it's less worse than having people falsely report good metrics.
  - whitemary 3y ago
    In the real world, owners simply don't care about your enlightening idea. Why would they? And what are you gonna do about it when they don't?
    - latency-guy2 3y ago
      Make my own. Just a reminder for you lefties but people are capable of creating things, many times it really doesn't take that much to do so, nor does getting rich in the process. Besides, why should anyone trust a leftist in power? We already know your lot will immediately outlaw and jail the opposition and pretend their uni-party system is "democratic". So I don't think you actually care about owners and power, you're just mad that you aren't in power perpetually.
    - bheadmaster 3y ago
      I don't quite understand the context of this post. I was talking about metrics such as uptime percentage, not enlightening ideas. Can you explain what do you mean?
- za3faran 3y ago
  He wasn't.
- formerly_proven 3y ago
  Socialist and communist regimes are legendary around the world for lying to supervisors syndrome. The politburo always gets the numbers it wants.
  - mordae 3y ago
    Yeah. Almost like the capitalist pyramid shaped corporations while actually being... The same?! What a shock! I know you probably heard that one, but as long as there is a boss or a general, it's not communism. Also, democracy is vulnerable to this as well. No politician wants to appear incompetent, leading to lying about results and taking unnecessary risks.
    - nannal 3y ago
      > as long as there is a boss or a general, it's not communism. I believe you may be confusing communism with anarchy.
  - amw 3y ago
    They're legendary to people mostly exposed to capitalist-owned media, that is.
- rambojohnson 3y ago
  tech has been unraveling for years now, especially with constant layoffs despite reporting record profits. Marx was most certainly onto something.
  - bretticus 3y ago
    The murder and starvation of millions of people? Even the “communist” government still around like China just have capitalist economies in sheep’s clothing.
    - rambojohnson 3y ago
      the stockholm syndrome in tech / capitalist america is astounding. While it's true that some communist regimes have been responsible for the suffering and death of millions, it's important to recognize that this is a distortion of Marx's ideas, rather than a direct result of them. Additionally, many capitalist societies have also had their share of social injustices and economic inequalities. It's essential to separate the philosophy from its historical implementations and consider the context in which Marx's ideas were formed. In the case of tech companies reporting record profits amidst massive layoffs, it's worth exploring whether certain aspects of Marxist theory could shed light on the systemic issues within our current economic system, rather than dismissing them outright based on past examples of "communism"
- alphanullmeric 3y ago
  [flagged]
  - amw 3y ago
    These responses are the worst of bad-faith every time I see them. If you live in a place, with family and friends and social ties, but it has some stuff you want to change, why would the solution ever be "move away" rather than "stay and make it change", except that the former is more convenient for assholes who are fine with how things are? As far as getting "shot" in Cuba, you're more likely to get shot by a cop in the US than in Cuba for any particular reason. Stop just soaking in wild propaganda uncritically.
    - alphanullmeric 3y ago
      That’s right, collectivists never want to reside in collectivist countries, they insist on converting capitalist countries so that they can leech off of wealth and businesses that wouldn’t otherwise exist under their own system. If Cuba is so great, put your money where your mouth is and live there. I have always and will continue to always have absolutely zero issue when communists steal from each other, all I ask is that others be left out of it.
    - lemmiwinks 3y ago
      Talking about bad faith... you know you're replying in a thread about GitHub incidents where someone is blaming capitalism for that right?
donatj 3y ago
Their old pre-acquisition status page had automated up to the minute stats. Their new status page often takes hours to reflect when something is broken.
- manuelabeledo 3y ago
  > ... had automated up to the minute stats This sounds like a SRE's absolute nightmare.
  - capableweb 3y ago
    Unless you're actually a professional and don't like lying.
    - manuelabeledo 3y ago
      Not talking about lying, but these status pages are rarely fully automated, meaning that there will be at least one person hovering around and constantly asking "are we there yet?".
      - capableweb 3y ago
        In my experience, that happens regardless if the process of saying "we're down" is automatic or manual. I'm not sure I follow your logic here. People will ask SREs "are we up yet?" more because the status page automatically show downtime rather than manual, or what are you saying?
  - zhann_dc 3y ago
    Why?
cssanchez 3y ago
A prime example from the Systems Thinking post. Errors of commission are the only thing that matters for your company and it is allergic to change/disruption from top to bottom.
passwordoops 3y ago
At one place our CFO string armed the devs to use AWS status as our own. Logic was we use them, therefore we're justified. Everytime a client reported an outage he'd gleefully share the status page and say "must be on your end". It was almost always on our end. Dude was a lawyer too
- jwestbury 3y ago
  Yikes. The AWS status page is a joke -- it's even a joke internally at AWS. There used to be (might still be?) an extension which just replaces all the icons on the AWS status page with one level worse than actually reported.
  - joshmanders 3y ago
    When was it, like 2013 where they had a bad outage and because the status page was hosted on S3 and S3 was affected they couldn't update the status page, so it showed all green even though half the internet was down. Good times, Amazon. Good times.
CoastalCoder 3y ago
If a publicly traded company in the U.S. did this, I'm curious if it constitutes securities fraud. I.e., a material misrepresentation about the company's health.
- mejutoco 3y ago
  Matt Levine is right again: it seems everything is securities fraud.
  - DiabloD3 3y ago
    TL;DR of everything Matt Levine has said: Lying and misrepresenting anything at all is securities fraud. Makes it pretty simple to not commit it: DON'T LIE.
    - dangets 3y ago
      Unless not lying / over sharing is bad for your shareholders. Lawyers could argue that publicizing non-material bad news about your service health makes your stock price drop.

stigi 3y ago

Wild guess: Are you working at Sendgrid?

brodouevencode 3y ago
AWS is also an acceptable guess
- clnq 3y ago
  99.5% of other tech companies are also acceptable guesses. Some will only inform users of service disruption after the media picks it up.

whitemary 3y ago

It's almost like every company is its own little authoritarian regime. Hmmmmm, maybe Karl Marx was onto something...

runlaszlorun 3y ago
Ha, I’ve never read Marx myself but that’s a great viewpoint. And it’s on my list.
- vorpalhex 3y ago
  Marx actually says the opposite. People form a dictatorship (dictatorship of the proletariat) to engage in a transition period away from Capitalism. Saying "the people own" is just a way of saying the state owns something. The state owns all the things under Communism. Every business is a state business.
  - FpUser 3y ago
    >"The state owns all the things under Communism." The state owns nothing under Communism as Communism assumes no state, social classes, private property and money. I do not think Communism can be achieved though, maybe in prehistoric society of docile gatherers of which we are not.
    - mordae 3y ago
      Perhaps eventually, after we learn to live better. But that's no reason to give up and make greed mandatory.
      - FpUser 3y ago
        People are greedy to a various degree by nature. Same for violence and desire to control other people. This is in our genes. I have no idea how you could suppress it without The State (comprised of control freaks willing to use violence) being on top
        rambojohnson 3y ago
        are you a geneticist? got an sources to back up that claim, that we are all inherently selfish? take a critical look at the society you were raised in.
        FpUser 3y ago
        I am not and I do not have to be. One can see it plain all around. I am very critical of the society but it does not change a fucking thing.
ALittleLight 3y ago
Yes, if there's one thing Marxists are known for it is definitely avoiding authoritarian regimes. Oh, apart from literally every country that tried Marxism.
- pedantismo 3y ago
  [flagged]
- baremetal 3y ago
  Oh but "true Marxism has never been tried", they will retort.
  - andrewaylett 3y ago
    Well, it hasn't. Because humans aren't capable of true Marxism. (This is not a suggestion to try anyway, nor a suggestion to eliminate humans in order to further any kind of Marxist "ideal")
    - mordae 3y ago
      Maybe, but so they aren't capable of living in a 100% transactional capitalist dystopia. As evidenced by everyone popping neurol (or another coping mechanism) with ever increasing frequency.
      - adventured 3y ago
        There is a huge difference still. At 3/4 Capitalist, your nation can prosper dramatically. At 3/4 Communist, you will know nothing but suffering and deprivation. Countries that have gotten closer to a free market orientation have prospered. Singapore, Hong Kong, the United States; all of the success stories of post USSR Europe are stories of market economics winning (including the dramatic example of the Baltics). All of Scandinavia operates via market based economic systems. Germany is a market based economy, not a Socialist economy, and they're the economic dynamo of Europe; the same goes for Britain, and even France (which certainly has a large welfare state, and is a mixed economy but not Socialist). There isn't a single prosperous Socialist nation in the world today. There are dozens of prosperous market based economic systems by contrast (the top several dozen most prosperous economies are dominated by market systems). Going the other direction, the closer you get to Socialism, the worse things get. And there are obviously limits to the market direction, anarchy does not work (although I'd argue that's not the market direction; market based economies in particular require the rule of law and a government to enforce it). The only country that can be argued to stand apart from this, even if temporarily, is China. However they're still drifting on the liberalizations of Deng and the benefits it spurred, that is grinding down under Xi. Their situation has worsened as they have headed back toward Socialism (and its inevitable increased authoritarianism) and away from market liberalization.
bheadmaster 3y ago
I don't think ownership (owning vs. working class) itself is the problem here. You can see similar behavior (hiding issues, lying about them) in any scenario in which punishment is tied to the failure to meet the metrics. When the metric becomes the goal, people will game the metric in order to avoid punishment and/or reap the rewards. The only kind of measurement that doesn't cause this behavior is the neutral measurement - measurement which is guaranteed not to change the decisions directly, but only serve as information which is later used in high-level optimization of systemic processes. In context of software companies, it would mean that status reports should only be reviewed in combination to other factors, such as tool used, features implemented, deadlines tightened, etc., and only to further understand how to optimize the process to avoid human error. It's risky to implement that, because you have to trust your employees enough to say "you won't be fired because of your metrics, ever", but I believe it's less worse than having people falsely report good metrics.
- whitemary 3y ago
  In the real world, owners simply don't care about your enlightening idea. Why would they? And what are you gonna do about it when they don't?
  - latency-guy2 3y ago
    Make my own. Just a reminder for you lefties but people are capable of creating things, many times it really doesn't take that much to do so, nor does getting rich in the process. Besides, why should anyone trust a leftist in power? We already know your lot will immediately outlaw and jail the opposition and pretend their uni-party system is "democratic". So I don't think you actually care about owners and power, you're just mad that you aren't in power perpetually.
  - bheadmaster 3y ago
    I don't quite understand the context of this post. I was talking about metrics such as uptime percentage, not enlightening ideas. Can you explain what do you mean?
za3faran 3y ago
He wasn't.
formerly_proven 3y ago
Socialist and communist regimes are legendary around the world for lying to supervisors syndrome. The politburo always gets the numbers it wants.
- mordae 3y ago
  Yeah. Almost like the capitalist pyramid shaped corporations while actually being... The same?! What a shock! I know you probably heard that one, but as long as there is a boss or a general, it's not communism. Also, democracy is vulnerable to this as well. No politician wants to appear incompetent, leading to lying about results and taking unnecessary risks.
  - nannal 3y ago
    > as long as there is a boss or a general, it's not communism. I believe you may be confusing communism with anarchy.
- amw 3y ago
  They're legendary to people mostly exposed to capitalist-owned media, that is.
rambojohnson 3y ago
tech has been unraveling for years now, especially with constant layoffs despite reporting record profits. Marx was most certainly onto something.
- bretticus 3y ago
  The murder and starvation of millions of people? Even the “communist” government still around like China just have capitalist economies in sheep’s clothing.
  - rambojohnson 3y ago
    the stockholm syndrome in tech / capitalist america is astounding. While it's true that some communist regimes have been responsible for the suffering and death of millions, it's important to recognize that this is a distortion of Marx's ideas, rather than a direct result of them. Additionally, many capitalist societies have also had their share of social injustices and economic inequalities. It's essential to separate the philosophy from its historical implementations and consider the context in which Marx's ideas were formed. In the case of tech companies reporting record profits amidst massive layoffs, it's worth exploring whether certain aspects of Marxist theory could shed light on the systemic issues within our current economic system, rather than dismissing them outright based on past examples of "communism"
alphanullmeric 3y ago
[flagged]
- amw 3y ago
  These responses are the worst of bad-faith every time I see them. If you live in a place, with family and friends and social ties, but it has some stuff you want to change, why would the solution ever be "move away" rather than "stay and make it change", except that the former is more convenient for assholes who are fine with how things are? As far as getting "shot" in Cuba, you're more likely to get shot by a cop in the US than in Cuba for any particular reason. Stop just soaking in wild propaganda uncritically.
  - alphanullmeric 3y ago
    That’s right, collectivists never want to reside in collectivist countries, they insist on converting capitalist countries so that they can leech off of wealth and businesses that wouldn’t otherwise exist under their own system. If Cuba is so great, put your money where your mouth is and live there. I have always and will continue to always have absolutely zero issue when communists steal from each other, all I ask is that others be left out of it.
  - lemmiwinks 3y ago
    Talking about bad faith... you know you're replying in a thread about GitHub incidents where someone is blaming capitalism for that right?

donatj 3y ago

Their old pre-acquisition status page had automated up to the minute stats. Their new status page often takes hours to reflect when something is broken.

manuelabeledo 3y ago
> ... had automated up to the minute stats This sounds like a SRE's absolute nightmare.
- capableweb 3y ago
  Unless you're actually a professional and don't like lying.
  - manuelabeledo 3y ago
    Not talking about lying, but these status pages are rarely fully automated, meaning that there will be at least one person hovering around and constantly asking "are we there yet?".
    - capableweb 3y ago
      In my experience, that happens regardless if the process of saying "we're down" is automatic or manual. I'm not sure I follow your logic here. People will ask SREs "are we up yet?" more because the status page automatically show downtime rather than manual, or what are you saying?
- zhann_dc 3y ago
  Why?

cssanchez 3y ago

A prime example from the Systems Thinking post. Errors of commission are the only thing that matters for your company and it is allergic to change/disruption from top to bottom.

passwordoops 3y ago

At one place our CFO string armed the devs to use AWS status as our own. Logic was we use them, therefore we're justified. Everytime a client reported an outage he'd gleefully share the status page and say "must be on your end". It was almost always on our end. Dude was a lawyer too

jwestbury 3y ago
Yikes. The AWS status page is a joke -- it's even a joke internally at AWS. There used to be (might still be?) an extension which just replaces all the icons on the AWS status page with one level worse than actually reported.
- joshmanders 3y ago
  When was it, like 2013 where they had a bad outage and because the status page was hosted on S3 and S3 was affected they couldn't update the status page, so it showed all green even though half the internet was down. Good times, Amazon. Good times.

CoastalCoder 3y ago

If a publicly traded company in the U.S. did this, I'm curious if it constitutes securities fraud. I.e., a material misrepresentation about the company's health.

mejutoco 3y ago
Matt Levine is right again: it seems everything is securities fraud.
- DiabloD3 3y ago
  TL;DR of everything Matt Levine has said: Lying and misrepresenting anything at all is securities fraud. Makes it pretty simple to not commit it: DON'T LIE.
  - dangets 3y ago
    Unless not lying / over sharing is bad for your shareholders. Lawyers could argue that publicizing non-material bad news about your service health makes your stock price drop.