6 ms·

Kind of sad to see GitHub doesn't use GitHub itself to deploy and monitor their releases.

by aszen 6y ago

illnewsthat 6y ago
I was surprised to read that they are using Slack since it is such a competitor to Microsoft's Teams (parent company).
- kuschkufan 6y ago
  Are you expecting them to use Windows everywhere as well?
  - dubcanada 6y ago
    No but why would you use a product that is $7 or what ever times the number of employees (so let's say 200, so $1400 a month) when you can use a free one.
    - maccard 6y ago
      Speaking from experience, just because you work for a company doesn't mean you can use all of their products (or that you'll even get favorable pricing on them).
      - lostapathy 6y ago
        I'd love to hear this story! Seems crazy ... but we live in a crazy world.
        vulcan01 6y ago
        When I worked at MS Azure, we had to pay for Azure servers! (I believe our team had a $5k/month Azure bill.) It's part of internal budgeting, so that people within MS don't splurge on expensive things (because it does cost MS money for each person on Teams).
        names_are_hard 6y ago
        Did you drop a k? What can you do with 50 dollars?
        vulcan01 6y ago
        Yes, thank you, it should be $5k. Edited.
        scott_w 6y ago
        Unrelated to software but the company my dad works for (motor repair) has to buy all its parts from its own distribution arm, at the marked up price. He then has to turn a profit on those parts as well as pricing the labour. If cost price is £5 and the markup is 20%, he has to pay £6 to get the part, then charge £7.20 on the invoice to the customer. I’ll let you guess what that does to tender bids ;-)
        names_are_hard 6y ago
        At Microsoft if you build a product using Azure (and if you want to use the cloud you MUST use Azure, you're not going to get approval to write a check to AWS) the costs come out of your budget. And it's taken seriously, to the point where teams will very much emphasize managing costs (what will this new feature cost on our Azure bill? Can we build it more efficiently? Oh wow, that refactor saved us 100k/month in cloud costs, don't forget that when we start talking about promotions...)
        lostapathy 6y ago
        That makes sense since the amount you could use is variable. I was thinking more like somebody couldn't get a free word license at a MS subsidiary or something.
        josephg 6y ago
        My uncle used to work at Compaq (back before they got bought by HP). When their computers broke, his team had to pay their support staff to get them fixed. (Via internal budgeting). But the support team knew internal customers would call them anyway and it was still compaq’s money, so they charged several times more for internal support calls than normal support calls. My uncle’s team was having none of that, so they paid an external computer repair service to fix their computers. The external repair service subcontracted to compaq’s internal people anyway, so when their computers broke they called up (and paid) external consultants. Who in turn called compaq’s internal support team, who came downstairs and fixed their computers at a competitive price.
        maccard 6y ago
        Very mundane I'm afraid. Worked for a MS subsidiary, on an online game for xbox and PC. Developed on windows, using visual studio and deployed on azure, used TFS for bug tracking. All of the above costs were tracked rigorously, and charged to the project. Most frustrating was complying with the visual studio licenses across the board, with no assistance from the licensing team. We had an account manager for all of the above but my understanding is that he was more of an auditor than anything else.
      - theshrike79 6y ago
        On the other hand sometimes it means you MUST use the company products. Consulted for a sub-sub-sub-subsidiary of Toshiba. All computer equipment had to be from Toshiba - the closest place to get Toshiba laptops was two COUNTRIES over. They even had to tape over non-Toshiba branding from external displays that would be visible.
      - eyelidlessness 6y ago
        In fact, from what I've heard, Microsoft generally charges internally for usage of products across orgs.
        NordSteve 6y ago
        Services, yes. Code is shared across the board at no cost.
    - paxys 6y ago
      $1400 a month is less than a rounding error for a company that size. If you can get even the tiniest bit of extra developer productivity from the software then it is worth it. And Github will definitely still have to "pay" for Teams, whether that is internal accounting or actual money being exchanged.
  - names_are_hard 6y ago
    My understanding of Microsoft policy is that it's easier to buy macbooks for your developers than it is to buy Slack. Which makes sense, because they're currently doing head to head with slack for market share right now, while a few macbooks doesn't threaten their credibility when selling windows. My guess is that github was using slack before they were bought and inertia is a thing. I'm sure there are people within the parent company that would like to see them transition, but I'm sure there's a ton of resistance, especially "on the ground" at github. Buyouts are a delicate thing, they don't want to ruin github by trying to force it to change too quickly.
- dubcanada 6y ago
  Probably because Teams is the worst. More then likely it's because that's what they used before they got bought and haven't been forced to migrate over yet, they also seem to have bots, which are not really a direct copy and paste into MS Teams, and likely them converting over isn't a high priority.
  - jen20 6y ago
    IIRC GitHub used to use Campfire and it took a long time to switch to Slack - a switch to Teams would no doubt take a long time too!
- paxys 6y ago
  Easy to switch a chat application, hard to switch your entire chatops ecosystem. This blog post shows the perfect example of that.
WJW 6y ago
That seems like an extremely good idea actually, since if you dogfood your own releasing service then you can't fix it anymore if you accidentally bring down the service.
- xxpor 6y ago
  That's usually solved with a parallel stack deployment, use the other stack if something is broken
  - paxys 6y ago
    If the "other stack" isn't regularly used then you can assume it will be broken when needed
    - cpascal 6y ago
      You just run the previous version of the production stack in your "dogfood/operations" stack. Once you've fully rolled out production and have vetted it, you can upgrade the other one to match production.
      - epidemian 6y ago
        With large codebases maintained by many people, sometimes it's difficult to "just" do things like that. It's a bit weird to think that no one within that large group of professional developers hasn't thought of doing a simple obvious solution. It's probably not that simple.
        cutemonster 6y ago
        Lots of comments here around that say "just do it like this: ...", or "why don't you: ... it'd be good because ..."
      - marcinzm 6y ago
        And then you get hit with a subtle bug that utterly nukes your system when the year changes from 2020 to 2021. Now you can't deploy anything because both systems are down. Given Github's scale and number of engineers it's basically guaranteed that they'll hit some sort of bug like that eventually. Not all bugs act immediately.
jules2689 6y ago
There is some GitHub used, but as others stated we don't want to create a circular dependency on ourselves in case we deploy something that is broken.
- qppo 6y ago
  Compilers seem to manage just fine with bootstrapping themselves. The trick is not to overwrite yourself
  - StavrosK 6y ago
    You're suggesting they run a second Github somewhere?
    - mappu 6y ago
      I think GitHub Enterprise includes all the features they would need.
- cbb330 6y ago
  Was some mix of decoupling / parallelizing the Actions service considered? I believe the value of dogfooding would be immense. Not only could you become the customer (massive reduction to the deploy/measure feedback loop), but it would be a key marketing move. On top of that, the GUI that will now require critical care and development is essentially a clone of what is offered by Github Actions.
  - jules2689 6y ago
    Yes there was. The change would have been quite a lot to do at once, and we aren't ready to safely add that circular dependency yet. We aren't done on this path, and this certainly won't be the last iteration. I suspect that iteration will come at some point after we can figure out the circular dependency issue. It also makes sense internally. Our actions team is much larger than the team that manages the project I wrote/lead, so it also makes logistical sense IMO We also started before the CD product was ready on actions and we directly influenced it, so yea haha

Kind of sad to see GitHub doesn't use GitHub itself to deploy and monitor their releases.

by aszen 6y ago

Kind of sad to see GitHub doesn't use GitHub itself to deploy and monitor their releases.

illnewsthat 6y ago

I was surprised to read that they are using Slack since it is such a competitor to Microsoft's Teams (parent company).

kuschkufan 6y ago
Are you expecting them to use Windows everywhere as well?
- dubcanada 6y ago
  No but why would you use a product that is $7 or what ever times the number of employees (so let's say 200, so $1400 a month) when you can use a free one.
  - maccard 6y ago
    Speaking from experience, just because you work for a company doesn't mean you can use all of their products (or that you'll even get favorable pricing on them).
    - lostapathy 6y ago
      I'd love to hear this story! Seems crazy ... but we live in a crazy world.
      - vulcan01 6y ago
        When I worked at MS Azure, we had to pay for Azure servers! (I believe our team had a $5k/month Azure bill.) It's part of internal budgeting, so that people within MS don't splurge on expensive things (because it does cost MS money for each person on Teams).
        names_are_hard 6y ago
        Did you drop a k? What can you do with 50 dollars?
        vulcan01 6y ago
        Yes, thank you, it should be $5k. Edited.
      - scott_w 6y ago
        Unrelated to software but the company my dad works for (motor repair) has to buy all its parts from its own distribution arm, at the marked up price. He then has to turn a profit on those parts as well as pricing the labour. If cost price is £5 and the markup is 20%, he has to pay £6 to get the part, then charge £7.20 on the invoice to the customer. I’ll let you guess what that does to tender bids ;-)
      - names_are_hard 6y ago
        At Microsoft if you build a product using Azure (and if you want to use the cloud you MUST use Azure, you're not going to get approval to write a check to AWS) the costs come out of your budget. And it's taken seriously, to the point where teams will very much emphasize managing costs (what will this new feature cost on our Azure bill? Can we build it more efficiently? Oh wow, that refactor saved us 100k/month in cloud costs, don't forget that when we start talking about promotions...)
        lostapathy 6y ago
        That makes sense since the amount you could use is variable. I was thinking more like somebody couldn't get a free word license at a MS subsidiary or something.
      - josephg 6y ago
        My uncle used to work at Compaq (back before they got bought by HP). When their computers broke, his team had to pay their support staff to get them fixed. (Via internal budgeting). But the support team knew internal customers would call them anyway and it was still compaq’s money, so they charged several times more for internal support calls than normal support calls. My uncle’s team was having none of that, so they paid an external computer repair service to fix their computers. The external repair service subcontracted to compaq’s internal people anyway, so when their computers broke they called up (and paid) external consultants. Who in turn called compaq’s internal support team, who came downstairs and fixed their computers at a competitive price.
      - maccard 6y ago
        Very mundane I'm afraid. Worked for a MS subsidiary, on an online game for xbox and PC. Developed on windows, using visual studio and deployed on azure, used TFS for bug tracking. All of the above costs were tracked rigorously, and charged to the project. Most frustrating was complying with the visual studio licenses across the board, with no assistance from the licensing team. We had an account manager for all of the above but my understanding is that he was more of an auditor than anything else.
    - theshrike79 6y ago
      On the other hand sometimes it means you MUST use the company products. Consulted for a sub-sub-sub-subsidiary of Toshiba. All computer equipment had to be from Toshiba - the closest place to get Toshiba laptops was two COUNTRIES over. They even had to tape over non-Toshiba branding from external displays that would be visible.
    - eyelidlessness 6y ago
      In fact, from what I've heard, Microsoft generally charges internally for usage of products across orgs.
      - NordSteve 6y ago
        Services, yes. Code is shared across the board at no cost.
  - paxys 6y ago
    $1400 a month is less than a rounding error for a company that size. If you can get even the tiniest bit of extra developer productivity from the software then it is worth it. And Github will definitely still have to "pay" for Teams, whether that is internal accounting or actual money being exchanged.
- names_are_hard 6y ago
  My understanding of Microsoft policy is that it's easier to buy macbooks for your developers than it is to buy Slack. Which makes sense, because they're currently doing head to head with slack for market share right now, while a few macbooks doesn't threaten their credibility when selling windows. My guess is that github was using slack before they were bought and inertia is a thing. I'm sure there are people within the parent company that would like to see them transition, but I'm sure there's a ton of resistance, especially "on the ground" at github. Buyouts are a delicate thing, they don't want to ruin github by trying to force it to change too quickly.
dubcanada 6y ago
Probably because Teams is the worst. More then likely it's because that's what they used before they got bought and haven't been forced to migrate over yet, they also seem to have bots, which are not really a direct copy and paste into MS Teams, and likely them converting over isn't a high priority.
- jen20 6y ago
  IIRC GitHub used to use Campfire and it took a long time to switch to Slack - a switch to Teams would no doubt take a long time too!
paxys 6y ago
Easy to switch a chat application, hard to switch your entire chatops ecosystem. This blog post shows the perfect example of that.

WJW 6y ago

That seems like an extremely good idea actually, since if you dogfood your own releasing service then you can't fix it anymore if you accidentally bring down the service.

xxpor 6y ago
That's usually solved with a parallel stack deployment, use the other stack if something is broken
- paxys 6y ago
  If the "other stack" isn't regularly used then you can assume it will be broken when needed
  - cpascal 6y ago
    You just run the previous version of the production stack in your "dogfood/operations" stack. Once you've fully rolled out production and have vetted it, you can upgrade the other one to match production.
    - epidemian 6y ago
      With large codebases maintained by many people, sometimes it's difficult to "just" do things like that. It's a bit weird to think that no one within that large group of professional developers hasn't thought of doing a simple obvious solution. It's probably not that simple.
      - cutemonster 6y ago
        Lots of comments here around that say "just do it like this: ...", or "why don't you: ... it'd be good because ..."
    - marcinzm 6y ago
      And then you get hit with a subtle bug that utterly nukes your system when the year changes from 2020 to 2021. Now you can't deploy anything because both systems are down. Given Github's scale and number of engineers it's basically guaranteed that they'll hit some sort of bug like that eventually. Not all bugs act immediately.

jules2689 6y ago

There is some GitHub used, but as others stated we don't want to create a circular dependency on ourselves in case we deploy something that is broken.

qppo 6y ago
Compilers seem to manage just fine with bootstrapping themselves. The trick is not to overwrite yourself
- StavrosK 6y ago
  You're suggesting they run a second Github somewhere?
  - mappu 6y ago
    I think GitHub Enterprise includes all the features they would need.
cbb330 6y ago
Was some mix of decoupling / parallelizing the Actions service considered? I believe the value of dogfooding would be immense. Not only could you become the customer (massive reduction to the deploy/measure feedback loop), but it would be a key marketing move. On top of that, the GUI that will now require critical care and development is essentially a clone of what is offered by Github Actions.
- jules2689 6y ago
  Yes there was. The change would have been quite a lot to do at once, and we aren't ready to safely add that circular dependency yet. We aren't done on this path, and this certainly won't be the last iteration. I suspect that iteration will come at some point after we can figure out the circular dependency issue. It also makes sense internally. Our actions team is much larger than the team that manages the project I wrote/lead, so it also makes logistical sense IMO We also started before the CD product was ready on actions and we directly influenced it, so yea haha