8 ms·

This seems backwards. Why charge for me to run the thing myself instead of them?

by bdbdbdb 10mo ago

mfcl 10mo ago
They still run the whole orchestration. If you don't want to pay, you'd have to not use GitHub Actions at all, maybe by using their API to test new commits and PRs and mark them as failed or passed.
- nextaccountic 10mo ago
  Can someone share a Github bot that doesn't depend on actions? I mean maybe https://github.com/rust-lang/bors https://github.com/rust-lang/bors is enough to fully replace Github Actions? (not sure)
  - jjice 10mo ago
    We have internal integrations with GitHub webhooks that will hit our server to checkout a branch, run some compute, and then post a comment on the thread. Not sure if you can integrate something like that to help block a PR from being merged like Actions CI checks, but you can receive webhooks and make API calls for free (for now). Would definitely result in some extra overhead to implement outside of Actions for some tasks.
    - masklinn 10mo ago
      > Not sure if you can integrate something like that to help block a PR from being merged like Actions CI checks Post statuses, and add rulesets to require those statuses before a PR can be merged. The step after that is to lock out pushing to the branch entirely and perform the integration externally but that has its own challenges.
  - reissbaker 10mo ago
    You can use webhooks to replace Github Actions: https://docs.github.com/en/webhooks/about-webhooks https://docs.github.com/en/webhooks/about-webhooks Listen to webhooks for new commits + PRs, and then use the commit status API to push statuses: https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28 https://docs.github.com/en/rest/commits/statuses?apiVersion=...
    - masklinn 10mo ago
      Yep, this mostly works fine (and can be necessary already in some setups anyway), the main issues are that each status update requires an API call (over v3, AFAIK updating statuses was never added to v4) so if you have a lot of statuses and PR traffic you can hit rate limits annoyingly quickly, and github will regularly fail to deliver or forward webhooks (also no ordering guarantees).
    - nextaccountic 10mo ago
      I mean, is there some open source project that already uses webhooks to replace Github Actions? Rather than having to write some ad hoc code to do this
- bad_haircut72 10mo ago
  Everyone who has Actions built into their workflow now has to go change it. Microsoft just conned a bunch more people with the same classic tech lock-in strategy they've always pursued, people are right to be pissed. The only learning to take away is never ever use anything from the big tech companies, even if it seems easier or cheaper right now to do so, because they're just waiting for the right moment to try and claw it back from you.
  - baobun 10mo ago
    > Microsoft just conned a bunch more people with the same classic tech lock-in strategy they've always pursued, people are right to be pissed People would be better served by not expecting anything different from Microsoft. As you say yourself, this is how they roll. > The only learning to take away is never ever use anything from the big tech companies Do you even believe in this yourself? Not being dependent on them would be a good start.
- codeflo 10mo ago
  One problem is that GitHub Actions isn't good. It's not like you're happily paying for some top tier "orchestration". It's there and integrated, which does make it convenient, but any price on this piece of garbage makes switching/self-hosting something to seriously consider.
  - QuercusMax 10mo ago
    Yeah, it seems like a half-assed version of what Jenkins and other tools have been doing for ages. Not that Jenkins is some magical wonderful tool, but I still haven't found a reasonable way to test my actions outside of running them on real Github.
  - hadlock 10mo ago
    Github being a single pane of glass for developers with a single login is pretty powerful. Github hosting the runners is also pretty useful, ask anyone who has had to actually manage/scale them what their opinion is about Jenkins is. Being a "Jenkins Farmer" is a thankless job that means a lot of on-call work to fix the build system in the middle of the night at 2am on a Sunday. Paying a small monthly fee is absolutely worth it to rescue the morale of your infra/platform/devops/sre team. Nothing kills morale faster than wrenching on the unreliable piece of infrastructure everyone hates. Every time I see an alert in slack github is having issues with actions (again) all I think is, "I'm glad that isn't me" and go about my day
    - bigstrat2003 10mo ago
      I run Jenkins (have done so at multiple jobs) and it's totally fine. Jenkins, like other super customizable systems, is as reliable or crappy as you make it. It's decent out of the box, but if you load it down with a billion plugins and whatnot then yeah it's going to be a nightmare to maintain. It all comes down to whether you've done a good job setting it up, IMO.
      - hadlock 10mo ago
        Lots of systems are "fine" until they aren't. As you pointed out, Jenkins being super-customizable means it isn't strongly opinionated, and there is plenty of opportunity for a well-meaning developer to add several foot-guns, doing some simple point and click in the GUI. Or the worst case scenario: cleaning up someone elses' Jenkins mess after they leave the company. Contrast with a declarative system like github actions: "I would like an immutable environment like this, and then perform X actions and send the logs/report back to the centralized single pane of glass in github". Google's "cloud run" product is pretty good in this regard as well. Sure, developers can add foot guns to your GHA/Cloud Run workflow, but since it is inherently git-tracked, you can simply revert those atomically. I used Jenkins for 5-7 years across several jobs and I don't miss it at all.
larkost 10mo ago
GitHub has still been managing the orchestration and monitoring of runs that you run on your own (or other cloud) hardware. They have just decided that they are no longer going to do this for free. So the question becomes: is $0.002/minute a good price for this. I have never run GitHub Actions, so I am going to assume that experience on other, similar, systems applies. So if your job takes an hour to build and run though all tests (a bit on the long side, but I have some tests that run for days), then you are going to pay GitHub $.12 for that run. You are probably going to pay significantly more for the compute for running that (especially if you are running on multiple testers simultaneously). So this does not seem to be too bad. This is probably going to push a lot of people to invest more in parallelizing their workloads, and/or putting them on faster machines in order to reduce the number of minutes they are billed for. I should note that if you are doing something similar in AWS using SMS (Systems Management Service), that I found that if you are running small jobs on lots of system that the AWS charges can add up very quickly. I had to abandon a monitoring system idea I had for our fleet (~800 systems) because the per-hit cost of just a monitoring ping was $1.84 (I needed a small mount of data from an on-worker process). Running that every 10 minutes was going to be more than $250/day. Writing/running my own monitoring system was much cheaper.
- j45 10mo ago
  Additionally, they could just self-host their code since code is data is a moat.
- featherless 10mo ago
  As a solo Founder who recently invested in self-hosted build infrastructure because my company runs ~70,000 minutes/month, this change is going to add an extra $140/month for hardware I own. And that's just today; this number will only go up over time. I am not open to GitHub extracting usage-based rent for me using my own hardware. This is the first time in my 15+ years of using GitHub that I'm seriously evaluating alternative products to move my company to.
  - hugs 10mo ago
    feels like a new generation is learning what life is like when microsoft has a lot of power. (tl;dr: they try to use it.)
mindcrash 10mo ago
Because they know Forgejo is starting to get attention from major players and thus becoming competitive, and hosting your own CI infrastructure will make completely moving away from GitHub all that easier - If you don't really care about the metadata all it pretty much takes is moving git repositories with their history. Or shortly summarized: lock in through pricing. Pretty sure this will explode straight in their faces though. And pretty damn hard.
- sallveburrpi 10mo ago
  How can you lock in through charging money? Seems it’s like the opposite and they are charging because people are already locked in and they can or am I misreading your comment?
  - mindcrash 10mo ago
    Microsoft "suddenly" does not seem to want you to run your own CI, which is a key part of running your own SCM. And this decision miraculously happens the moment a lot of big orgs are looking at self-hosting a cost effective (because open source) near 1:1 alternative to GitHub (=Forgejo). So they make CI a bit cheaper but a future migration to Forgejo harder. In fact they could easily pull off some typical sleazy Microsoft bullshit and eventually make it a shit ton harder to migrate out of GitHub once you migrated back in.
  - Vegenoid 10mo ago
    The idea is that they let you stay locked in for free. They dissuade people from making their CI pipeline forge-agnostic by charging you if you if you take steps to not be dependent on them. This means they can keep charging in other areas, and keep people in GitHub so that it stays dominant. Dominance is something that can be used to keep people in the Microsoft ecosystem, keep GitHub as the place where code goes so they have training data for LLMs, and dominance can simply be cashed in down the line. I don’t know if that’s actually why they’re doing this, but it sounds plausible.
  - dragonwriter 10mo ago
    If you make running your own runners as expensive as running on Github's runners on top of the cost of actually hosting the runners, then if you are currently on Github and not able to migrate off immediately, the price conscious decision is to migrate runners into Github. But then, its even harder if you ever decide to migrate your whole operation out. Now, if you are already looking at migrating, its also potentially a kick in the butt to do it now. But if you aren’t, the path of least resistance—or at least, the path of least present recurring cost—is a path to a greater degree of lock-in.
gaigalas 10mo ago
I develop software, I also test and run it. All in my machines. But you (yes, you personally) have to collect the results and publish them to a webpage for me. For free. Would you make this deal?
- falsedan 10mo ago
  if you were paying me a monthly license fee for each developer working on your repos, I'd probably consider it
  - gaigalas 10mo ago
    What happens if I am, and now my developers suddenly start to produce changes much faster? Like, one developer now produces the volume of five. Would you keep charging the same rate per head?
    - falsedan 10mo ago
      no, I'd cut the monthly seat cost and grow my user base to include more low-volume devs but realistically, publishing a web page is practically free. you could be sending 100x as much data and I would still be laughing all the way to the bank
      - gaigalas 10mo ago
        Publishing the page is only the last step. It's orchestrating the stuff THEN publishing it. If you think that's easy, do it for me. I have some projects to migrate, give me the link of your service.
        janc_ 10mo ago
        There are several services I know who offer this for free for open source software, and I really doubt any commercial offerings of that software would charge you extra for what is basic API usage.
        10mo ago
naikrovek 10mo ago
Because they host the artifacts, logs, and schedule jobs which run on your runners, I assume.
- falsedan 10mo ago
  they charge you for artifacts and logs separately, already
  - naikrovek 10mo ago
    Yep and the sky is blue and GitHub can charge for that too if they want to. I don’t make policy at GitHub and I don’t work at GitHub so go ask GitHub why they charge for infrastructure costs like any other cloud service. It has to do with the queueing and assignment of jobs which is not free. Why do they charge per minute? I have no idea, maybe it was easiest to do that given the billing infrastructure they already have. Maybe they tried a million different ways and this was the most reasonable. Maybe it’s Microsoft and they’re giving us all the middle finger, who knows.
    - falsedan 10mo ago
      I don't think you're responsible for anything more than your own comments. I added some context that contradicts your assumption that the increased fees were to cover hosting/storage/scheduling costs.
- progval 10mo ago
  Then why do they charge by the minute instead of gigabytes and number of events?
  - naikrovek 10mo ago
    Ask them. I don’t set the policy at a company I don’t work at. Their announcement gives a clue, and it’s to do with job orchestration.
baq 10mo ago
The scheduler isn’t free, I always wondered how the financials work on this one. Turns out they didn’t ;) Anyway, GitHub actions is a dumpster fire even without this change.
vsl 10mo ago
Because GHA was stagnant and expensive and multiple services like https://www.warpbuild.com/ https://www.warpbuild.com/ popped up, with better performance and much lower price. Looks like they ate enough of GH’s lunch…
- suryao 10mo ago
  Hey, WarpBuild founder here. While it makes it harder for us to communicate this, we're still, we're still faster and cheaper even after the $0.002/min self hosting tax. Overall costs go up for everyone but we remain the better option.
IshKebab 10mo ago
Because they make money from charging way over cost price for per-minute CI runners, and they don't want people using much much cheaper alternative providers. They don't care about people actually self-hosting. They care about people "self hosting" with these guys: https://github.com/neysofu/awesome-github-actions-runners https://github.com/neysofu/awesome-github-actions-runners
vbezhenar 10mo ago
Because charging you brings more profits than not charging you.

This seems backwards. Why charge for me to run the thing myself instead of them?

by bdbdbdb 10mo ago

This seems backwards. Why charge for me to run the thing myself instead of them?

mfcl 10mo ago

They still run the whole orchestration. If you don't want to pay, you'd have to not use GitHub Actions at all, maybe by using their API to test new commits and PRs and mark them as failed or passed.

nextaccountic 10mo ago
Can someone share a Github bot that doesn't depend on actions? I mean maybe https://github.com/rust-lang/bors https://github.com/rust-lang/bors is enough to fully replace Github Actions? (not sure)
- jjice 10mo ago
  We have internal integrations with GitHub webhooks that will hit our server to checkout a branch, run some compute, and then post a comment on the thread. Not sure if you can integrate something like that to help block a PR from being merged like Actions CI checks, but you can receive webhooks and make API calls for free (for now). Would definitely result in some extra overhead to implement outside of Actions for some tasks.
  - masklinn 10mo ago
    > Not sure if you can integrate something like that to help block a PR from being merged like Actions CI checks Post statuses, and add rulesets to require those statuses before a PR can be merged. The step after that is to lock out pushing to the branch entirely and perform the integration externally but that has its own challenges.
- reissbaker 10mo ago
  You can use webhooks to replace Github Actions: https://docs.github.com/en/webhooks/about-webhooks https://docs.github.com/en/webhooks/about-webhooks Listen to webhooks for new commits + PRs, and then use the commit status API to push statuses: https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28 https://docs.github.com/en/rest/commits/statuses?apiVersion=...
  - masklinn 10mo ago
    Yep, this mostly works fine (and can be necessary already in some setups anyway), the main issues are that each status update requires an API call (over v3, AFAIK updating statuses was never added to v4) so if you have a lot of statuses and PR traffic you can hit rate limits annoyingly quickly, and github will regularly fail to deliver or forward webhooks (also no ordering guarantees).
  - nextaccountic 10mo ago
    I mean, is there some open source project that already uses webhooks to replace Github Actions? Rather than having to write some ad hoc code to do this
bad_haircut72 10mo ago
Everyone who has Actions built into their workflow now has to go change it. Microsoft just conned a bunch more people with the same classic tech lock-in strategy they've always pursued, people are right to be pissed. The only learning to take away is never ever use anything from the big tech companies, even if it seems easier or cheaper right now to do so, because they're just waiting for the right moment to try and claw it back from you.
- baobun 10mo ago
  > Microsoft just conned a bunch more people with the same classic tech lock-in strategy they've always pursued, people are right to be pissed People would be better served by not expecting anything different from Microsoft. As you say yourself, this is how they roll. > The only learning to take away is never ever use anything from the big tech companies Do you even believe in this yourself? Not being dependent on them would be a good start.
codeflo 10mo ago
One problem is that GitHub Actions isn't good. It's not like you're happily paying for some top tier "orchestration". It's there and integrated, which does make it convenient, but any price on this piece of garbage makes switching/self-hosting something to seriously consider.
- QuercusMax 10mo ago
  Yeah, it seems like a half-assed version of what Jenkins and other tools have been doing for ages. Not that Jenkins is some magical wonderful tool, but I still haven't found a reasonable way to test my actions outside of running them on real Github.
- hadlock 10mo ago
  Github being a single pane of glass for developers with a single login is pretty powerful. Github hosting the runners is also pretty useful, ask anyone who has had to actually manage/scale them what their opinion is about Jenkins is. Being a "Jenkins Farmer" is a thankless job that means a lot of on-call work to fix the build system in the middle of the night at 2am on a Sunday. Paying a small monthly fee is absolutely worth it to rescue the morale of your infra/platform/devops/sre team. Nothing kills morale faster than wrenching on the unreliable piece of infrastructure everyone hates. Every time I see an alert in slack github is having issues with actions (again) all I think is, "I'm glad that isn't me" and go about my day
  - bigstrat2003 10mo ago
    I run Jenkins (have done so at multiple jobs) and it's totally fine. Jenkins, like other super customizable systems, is as reliable or crappy as you make it. It's decent out of the box, but if you load it down with a billion plugins and whatnot then yeah it's going to be a nightmare to maintain. It all comes down to whether you've done a good job setting it up, IMO.
    - hadlock 10mo ago
      Lots of systems are "fine" until they aren't. As you pointed out, Jenkins being super-customizable means it isn't strongly opinionated, and there is plenty of opportunity for a well-meaning developer to add several foot-guns, doing some simple point and click in the GUI. Or the worst case scenario: cleaning up someone elses' Jenkins mess after they leave the company. Contrast with a declarative system like github actions: "I would like an immutable environment like this, and then perform X actions and send the logs/report back to the centralized single pane of glass in github". Google's "cloud run" product is pretty good in this regard as well. Sure, developers can add foot guns to your GHA/Cloud Run workflow, but since it is inherently git-tracked, you can simply revert those atomically. I used Jenkins for 5-7 years across several jobs and I don't miss it at all.

larkost 10mo ago

GitHub has still been managing the orchestration and monitoring of runs that you run on your own (or other cloud) hardware. They have just decided that they are no longer going to do this for free. So the question becomes: is $0.002/minute a good price for this. I have never run GitHub Actions, so I am going to assume that experience on other, similar, systems applies. So if your job takes an hour to build and run though all tests (a bit on the long side, but I have some tests that run for days), then you are going to pay GitHub $.12 for that run. You are probably going to pay significantly more for the compute for running that (especially if you are running on multiple testers simultaneously). So this does not seem to be too bad. This is probably going to push a lot of people to invest more in parallelizing their workloads, and/or putting them on faster machines in order to reduce the number of minutes they are billed for. I should note that if you are doing something similar in AWS using SMS (Systems Management Service), that I found that if you are running small jobs on lots of system that the AWS charges can add up very quickly. I had to abandon a monitoring system idea I had for our fleet (~800 systems) because the per-hit cost of just a monitoring ping was $1.84 (I needed a small mount of data from an on-worker process). Running that every 10 minutes was going to be more than $250/day. Writing/running my own monitoring system was much cheaper.

j45 10mo ago
Additionally, they could just self-host their code since code is data is a moat.
featherless 10mo ago
As a solo Founder who recently invested in self-hosted build infrastructure because my company runs ~70,000 minutes/month, this change is going to add an extra $140/month for hardware I own. And that's just today; this number will only go up over time. I am not open to GitHub extracting usage-based rent for me using my own hardware. This is the first time in my 15+ years of using GitHub that I'm seriously evaluating alternative products to move my company to.
- hugs 10mo ago
  feels like a new generation is learning what life is like when microsoft has a lot of power. (tl;dr: they try to use it.)

mindcrash 10mo ago

Because they know Forgejo is starting to get attention from major players and thus becoming competitive, and hosting your own CI infrastructure will make completely moving away from GitHub all that easier - If you don't really care about the metadata all it pretty much takes is moving git repositories with their history. Or shortly summarized: lock in through pricing. Pretty sure this will explode straight in their faces though. And pretty damn hard.

sallveburrpi 10mo ago
How can you lock in through charging money? Seems it’s like the opposite and they are charging because people are already locked in and they can or am I misreading your comment?
- mindcrash 10mo ago
  Microsoft "suddenly" does not seem to want you to run your own CI, which is a key part of running your own SCM. And this decision miraculously happens the moment a lot of big orgs are looking at self-hosting a cost effective (because open source) near 1:1 alternative to GitHub (=Forgejo). So they make CI a bit cheaper but a future migration to Forgejo harder. In fact they could easily pull off some typical sleazy Microsoft bullshit and eventually make it a shit ton harder to migrate out of GitHub once you migrated back in.
- Vegenoid 10mo ago
  The idea is that they let you stay locked in for free. They dissuade people from making their CI pipeline forge-agnostic by charging you if you if you take steps to not be dependent on them. This means they can keep charging in other areas, and keep people in GitHub so that it stays dominant. Dominance is something that can be used to keep people in the Microsoft ecosystem, keep GitHub as the place where code goes so they have training data for LLMs, and dominance can simply be cashed in down the line. I don’t know if that’s actually why they’re doing this, but it sounds plausible.
- dragonwriter 10mo ago
  If you make running your own runners as expensive as running on Github's runners on top of the cost of actually hosting the runners, then if you are currently on Github and not able to migrate off immediately, the price conscious decision is to migrate runners into Github. But then, its even harder if you ever decide to migrate your whole operation out. Now, if you are already looking at migrating, its also potentially a kick in the butt to do it now. But if you aren’t, the path of least resistance—or at least, the path of least present recurring cost—is a path to a greater degree of lock-in.

gaigalas 10mo ago

I develop software, I also test and run it. All in my machines. But you (yes, you personally) have to collect the results and publish them to a webpage for me. For free. Would you make this deal?

falsedan 10mo ago
if you were paying me a monthly license fee for each developer working on your repos, I'd probably consider it
- gaigalas 10mo ago
  What happens if I am, and now my developers suddenly start to produce changes much faster? Like, one developer now produces the volume of five. Would you keep charging the same rate per head?
  - falsedan 10mo ago
    no, I'd cut the monthly seat cost and grow my user base to include more low-volume devs but realistically, publishing a web page is practically free. you could be sending 100x as much data and I would still be laughing all the way to the bank
    - gaigalas 10mo ago
      Publishing the page is only the last step. It's orchestrating the stuff THEN publishing it. If you think that's easy, do it for me. I have some projects to migrate, give me the link of your service.
      - janc_ 10mo ago
        There are several services I know who offer this for free for open source software, and I really doubt any commercial offerings of that software would charge you extra for what is basic API usage.
      - 10mo ago

naikrovek 10mo ago

Because they host the artifacts, logs, and schedule jobs which run on your runners, I assume.

falsedan 10mo ago
they charge you for artifacts and logs separately, already
- naikrovek 10mo ago
  Yep and the sky is blue and GitHub can charge for that too if they want to. I don’t make policy at GitHub and I don’t work at GitHub so go ask GitHub why they charge for infrastructure costs like any other cloud service. It has to do with the queueing and assignment of jobs which is not free. Why do they charge per minute? I have no idea, maybe it was easiest to do that given the billing infrastructure they already have. Maybe they tried a million different ways and this was the most reasonable. Maybe it’s Microsoft and they’re giving us all the middle finger, who knows.
  - falsedan 10mo ago
    I don't think you're responsible for anything more than your own comments. I added some context that contradicts your assumption that the increased fees were to cover hosting/storage/scheduling costs.
progval 10mo ago
Then why do they charge by the minute instead of gigabytes and number of events?
- naikrovek 10mo ago
  Ask them. I don’t set the policy at a company I don’t work at. Their announcement gives a clue, and it’s to do with job orchestration.

baq 10mo ago

The scheduler isn’t free, I always wondered how the financials work on this one. Turns out they didn’t ;) Anyway, GitHub actions is a dumpster fire even without this change.

vsl 10mo ago

Because GHA was stagnant and expensive and multiple services like https://www.warpbuild.com/ https://www.warpbuild.com/ popped up, with better performance and much lower price. Looks like they ate enough of GH’s lunch…

suryao 10mo ago
Hey, WarpBuild founder here. While it makes it harder for us to communicate this, we're still, we're still faster and cheaper even after the $0.002/min self hosting tax. Overall costs go up for everyone but we remain the better option.

IshKebab 10mo ago

Because they make money from charging way over cost price for per-minute CI runners, and they don't want people using much much cheaper alternative providers. They don't care about people actually self-hosting. They care about people "self hosting" with these guys: https://github.com/neysofu/awesome-github-actions-runners https://github.com/neysofu/awesome-github-actions-runners

vbezhenar 10mo ago

Because charging you brings more profits than not charging you.