10 ms·

Arm64 on GitHub Actions

247 points by sonichigo 2y ago

obviyus 2y ago
Nice! I had been using self-hosted arm runners for my projects lately (which GitHub makes surprisingly easy to do): https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners https://docs.github.com/en/actions/hosting-your-own-runners/... Considering how compiling a couple of these projects on my tiny arm VPS slows down everything else, this is a welcome change!
- sofixa 2y ago
  > which GitHub makes surprisingly easy to do Why surprisingly? Are there any CI/CD, let alone SaaS CI/CD providers, that don't make it easy?
  - obviyus 2y ago
    For some reason I expected a "not-first-class" experience while using my own hosted runner but that was not the case. It works and behaves exactly as their own runners do. Maybe s/surprisingly/extremely would be more what I meant. Just wanted to add, I loved your post on logging Nomad allocs using Loki. Was a huge help while I was setting up our clusters at $CURRENT_JOB
    - sofixa 2y ago
      > For some reason I expected a "not-first-class" experience while using my own hosted runner but that was not the case. It works and behaves exactly as their own runners do. Interesting, I've used GitLab extensively and hosted/self-hosted runners were trivial to use/run, so I would have expected nothing less from GitHub. They probably make a decent margin on the hosted runners at the cost of significant complexity, so if they can give that away to the orgs that will have their own (virtual) hardware anyway, with their own contracts and networking and etc., why not? > Just wanted to add, I loved your post on logging Nomad allocs using Loki. Was a huge help while I was setting up our clusters at $CURRENT_JOB Glad you found it helpful! If you ever want to chat about it, feel free to hit me up (contacts are on my website available in my profile).
saagarjha 2y ago
I mean they did before, but they were running macOS ;)
drcongo 2y ago
Digital Ocean are going to be dead last in the transition to ARM.
- diggan 2y ago
  Are GitHub and Digital Ocean competitors nowadays? Or you're just referring to anything that offers "run code on someone elses computer, somehow"?
  - drcongo 2y ago
    Sorry, my post was rather lacking context wasn't it. What I meant was that the vast majority of my work is now done end to end on ARM CPUs and even the bits that haven't been, like GitHub runners, are starting to make ARM an option. Customers have been asking Digital Ocean for ARM servers for years [0], and every time they do, DO just tell you to upvote the suggestion on their feedback site, but you can't, because they closed it [1]. [0] https://www.digitalocean.com/community/questions/feature-request-any-chance-of-arm-instances?comment=119804 https://www.digitalocean.com/community/questions/feature-req... [1] https://ideas.digitalocean.com/core-compute-platform/p/arm-based-droplets https://ideas.digitalocean.com/core-compute-platform/p/arm-b... edit for additional context: An ARM server on Hetzner costs roughly 1/10th that of an x86 one on DO with the same core count and memory, and in my tests out-performs it too.
- bdcravens 2y ago
  AWS still only supports X64 on Fargate Spot instances (they do support ARM on the non-spot instances)
  - trueismywork 2y ago
    T4g is available as spot
    - endgame 2y ago
      GP's talking about Fargate, which is their "serverless" container runtime offering.
shpx 2y ago
It's not free. > Larger runners are only available for organizations and enterprises using the GitHub Team [$4 per user/month] or GitHub Enterprise Cloud [$21 per user/month] plans.
- sunaookami 2y ago
  Seems like normal users have to wait: > We expect to begin offering Arm runners for open source projects by the end of the year side note: I find it mildly annoying that it's no longer "ARM" but "Arm"
  - VWWHFSfQ 2y ago
    Yeah it's confusing because the company is branded like "Arm" but they still call the architecture "ARM"
joshstrange 2y ago
GitHub’s runners are a joke. Switching to WarpBuild was the best decision I made for my CI/CD. I halved my build time and got cheaper minutes. GH’s macOS runners particularly were complete trash.
- deleted 2y ago
  [deleted]
- 999900000999 2y ago
  I understand the Mac builds, but the entire appeal of GitHub runners is it's literally built in. I can just add a YAML file, and we have CICD!
  - joshstrange 2y ago
    I get that, but you can run GH runners on your own hardware or someone else's. Switching to WarpBuild was literally as easy as linking my GH account and changing the running image from `macOS-latest` to `warp-macos-14-arm64-6x` and everything worked. Secrets and everything.
    - palata 2y ago
      > Secrets and everything. I guess you had to import the secrets, too...
  - deleted 2y ago
    [deleted]
- yohannparis 2y ago
  Different needs, different tools! I work on a open-source project, and not having to pay for CI/CD with a "simple" yml file configuration is simpler and easy to manage.
  - 01HNNWZ0MV43FF 2y ago
    It's only free if your time is worthless :P every couple weeks I have to debug something that I can't replicate locally because I don't have local CI
deivid 2y ago
I wonder if they'll start supporting risc-v soon. I've been using github-act-runner on top of a local Nomad cluster to run some of my builds on risc-v, arm64 & x86-64, it's fine, but a bit of a hassle to set up
- compsciphd 2y ago
  would only happen after azure supports risc-v VMs.
- sapiogram 2y ago
  Given how long arm64 took them, my guess is "3 years after major cloud vendors start offering risc-v VMs".
  - deivid 2y ago
    I'd kinda hoped that they took so long to go arm64 because they generalized "1 architecture -> N architectures"
crohr 2y ago
Much welcome addition, although not available (yet) on free plans. I benchmarked them and they are ~15-20% slower than AWS Graviton processors: https://runs-on.com/benchmarks/github-actions-runners/#arm64-runners https://runs-on.com/benchmarks/github-actions-runners/#arm64... But at 37% cheaper than x64 it's quite a better deal than what the outdated x64 CPUs.
- crohr 2y ago
  Also a bit weird that they now seem to delegate the image building to third-party vendors (first GPU images, now ARM). Would be nice to know how they validate base images before distributing them to everyone.
- ozgune 2y ago
  (Ozgun from Ubicloud) Looks like every GitHub Actions provider is on this thread. :) I also wanted to chime in with two thoughts. First, we recently wrote about how we enabled arm64 VMs at Ubicloud. Any feedback on the technical details is more than welcome: https://news.ycombinator.com/item?id=40491377 https://news.ycombinator.com/item?id=40491377 Second, I feel runs-on's analysis is a bit unfair to us. Ubicloud's x64 / arm64 performance and queue times above are as good as any (we deprecated the AMD EPYC 7502 line). For example, RunsOn arm64 queue times are 31|42 seconds and Ubicloud queue times are 17|24 seconds. But the analysis says the following, "Be aware that Ubicloud has low CPU speeds and high queue times, which means they won’t necessarily end up being cheaper than competitors." I fail to see this conclusion from the numbers above. What am I missing?
  - crohr 2y ago
    The queue time for x64 appears to vary quite a bit still. But you are correct that it is much better than what it was a few weeks back, when the analysis was written (it was not unusual to see 60s+ spikes). Also note that I specifically mentioned that "all third-parties are good on that front, expected [sic] Ubicloud (but that may change)." Also agree that now that you removed the outdated CPUs (was still active end of May), the analysis should mention that you have OK CPUs by now. I will fix it. For providers that are on this page, do not hesitate to write to me if you find the analysis outdated for your specific service. [edit] page is now up to date with new analysis
suryao 2y ago
I'm so glad that GitHub finally offers this. They're just more efficient for certain kinds of workflows - both cheaper and faster. If you're building for arm64 targets or android, this is a no brainer because it sidesteps the emulation requirements. We've been offering arm64 runners for a few months now (up to 32 vcpu) with WarpBuild and our users love it. In general, there are lots of inefficiencies in CI starting from the lack of visibility, debugging ease, performance, runner config customization, persistence, etc. We're out to solve it and make the world's fastest CI cloud ecosystem on top of existing CI providers with WarpBuild.
- christina97 2y ago
  Does WarpBuild offer discounts for nonprofits/FOSS?
  - suryao 2y ago
    Not at this time. We're a small team trying to build a sustainably growing business with server bills to pay.
  - vessy-st6io 2y ago
    We at FlyCI offer macOS runners of GitHub Actions and have free 500 minutes per month for public repositories. Also, we are about to release FlyCI Wingman - an AI agent that analyzes and fixes failing builds. Feel free to contact us at contact@flyci.net and tell us more about your needs. We'll be happy to help you with a solution. FlyCI website: https://www.flyci.net/ https://www.flyci.net/ Discord server: https://discord.gg/JyCjh439da https://discord.gg/JyCjh439da
bhouston 2y ago
Hmm... this is neat. I develop on MacBook so ARM64, and I do use GitHub Actions, which can not be ARM64, but unfortunately Google Cloud Run doesn't support ARM64 images yet: https://issuetracker.google.com/issues/303743857 https://issuetracker.google.com/issues/303743857
- stefantalpalaru 2y ago
  [dead]
- david_allison 2y ago
  Excuse my ignorance, isn't `runs-on: macos-14` ARM64-based (M1) https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/ https://github.blog/changelog/2024-01-30-github-actions-maco...
  - bhouston 2y ago
    I am looking for Google Cloud Run arm64 support.
booleanbetrayal 2y ago
It's nice to know that we have an ARM64 fallback if our perfectly robust and cheaper self-hosting of GHA (gha-runner-scale-set* components) were ever to go down, I suppose, but I can't help but feel like I am part of a huge market segment that GitHub lost during the ARM64 runner absence. Seems a little too late given they have natively supported action-runners images in ARM64 for some time.
dedene 2y ago
Can it be used to efficiently build multi-arch docker images? Both armv8/x64
- caleblloyd 2y ago
  Maybe but I’d recommend checking out Depot.Dev for that, they expose Arm and x86-64 runners to buildkit/buildx. Not associated with them, just a happy customer.
- pl4nty 2y ago
  it's a pain, you'll need to build the images on separate jobs then merge them into a multi-arch manifest. I moved to namespace.so for remote BuildKit builds a while ago and haven't looked back. depot.dev is also good, but pretty expensive
rahkiin 2y ago
Are there third party providers for Gitlab runners? I’d love windows and mac runners for my self hosted gitlab.
frankdejonge 2y ago
One thing that consistently bugs me is how upgrading from the lowest type of runners to _literally_ anything else renders your paid for included minutes useless. I do not understand who those wouldn't just be multiples of the base runner, much like how Mac and Windows runners are. Seems like the crediting logic is there, but knowing software, there is probably some accidental complexity causing this part not to leverage it. That said, it's very frustrating from a paying customer perspective. The same goes for minute crediting, where splitting things up to run concurrently is actively discouraged because they individually all round up to the next minute. For example; have 3 concurrent tasks that run 1m10s? You're billed 6 minutes. I get that a run would be rounded up, but come on.
deleted 2y ago
[deleted]
AndyKelley 2y ago
At Zig Software Foundation we rent an aarch64 Hetzner server for about 100 euros a month and it has 64 processors. It can handle a lot more CI work than the GitHub provided runners for a fraction of the cost.
qwertox 2y ago
I had an issue where a Rust application, which I had compiled on a Radxa ROCK 5B, would nor run on a Raspberry Pi 4; I had to compile it again for it. Both were running Ubuntu 22.04, 64 bit. Why would that be and how would this affect these Arm64 compiles on GitHub? If you compile a normal application for amd64, it will work on any 64 bit Intel and AMD machine, unless you start using specialized features, which wasn't the case in the above mentioned case.
- written-beyond 2y ago
  What was the error you were getting when you tried to run on the RPI?
- rami3l 2y ago
  This reminds me of one issue that I've encountered at Rustup. Early versions of ARM64 Pi OS actually use a 32-bit userland. As a result, this has caused user complaints since our installer sees a 64-bit kernel and will happily download a 64-bit binary (which will then refuse to run): https://github.com/rust-lang/rustup/issues/3307#issuecomment-1732175383 https://github.com/rust-lang/rustup/issues/3307#issuecomment...

247 points by sonichigo 2y ago

Nice! I had been using self-hosted arm runners for my projects lately (which GitHub makes surprisingly easy to do): https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners https://docs.github.com/en/actions/hosting-your-own-runners/... Considering how compiling a couple of these projects on my tiny arm VPS slows down everything else, this is a welcome change!

sofixa 2y ago
> which GitHub makes surprisingly easy to do Why surprisingly? Are there any CI/CD, let alone SaaS CI/CD providers, that don't make it easy?
- obviyus 2y ago
  For some reason I expected a "not-first-class" experience while using my own hosted runner but that was not the case. It works and behaves exactly as their own runners do. Maybe s/surprisingly/extremely would be more what I meant. Just wanted to add, I loved your post on logging Nomad allocs using Loki. Was a huge help while I was setting up our clusters at $CURRENT_JOB
  - sofixa 2y ago
    > For some reason I expected a "not-first-class" experience while using my own hosted runner but that was not the case. It works and behaves exactly as their own runners do. Interesting, I've used GitLab extensively and hosted/self-hosted runners were trivial to use/run, so I would have expected nothing less from GitHub. They probably make a decent margin on the hosted runners at the cost of significant complexity, so if they can give that away to the orgs that will have their own (virtual) hardware anyway, with their own contracts and networking and etc., why not? > Just wanted to add, I loved your post on logging Nomad allocs using Loki. Was a huge help while I was setting up our clusters at $CURRENT_JOB Glad you found it helpful! If you ever want to chat about it, feel free to hit me up (contacts are on my website available in my profile).

saagarjha 2y ago

I mean they did before, but they were running macOS ;)

drcongo 2y ago

Digital Ocean are going to be dead last in the transition to ARM.

diggan 2y ago
Are GitHub and Digital Ocean competitors nowadays? Or you're just referring to anything that offers "run code on someone elses computer, somehow"?
- drcongo 2y ago
  Sorry, my post was rather lacking context wasn't it. What I meant was that the vast majority of my work is now done end to end on ARM CPUs and even the bits that haven't been, like GitHub runners, are starting to make ARM an option. Customers have been asking Digital Ocean for ARM servers for years [0], and every time they do, DO just tell you to upvote the suggestion on their feedback site, but you can't, because they closed it [1]. [0] https://www.digitalocean.com/community/questions/feature-request-any-chance-of-arm-instances?comment=119804 https://www.digitalocean.com/community/questions/feature-req... [1] https://ideas.digitalocean.com/core-compute-platform/p/arm-based-droplets https://ideas.digitalocean.com/core-compute-platform/p/arm-b... edit for additional context: An ARM server on Hetzner costs roughly 1/10th that of an x86 one on DO with the same core count and memory, and in my tests out-performs it too.
bdcravens 2y ago
AWS still only supports X64 on Fargate Spot instances (they do support ARM on the non-spot instances)
- trueismywork 2y ago
  T4g is available as spot
  - endgame 2y ago
    GP's talking about Fargate, which is their "serverless" container runtime offering.

shpx 2y ago

It's not free. > Larger runners are only available for organizations and enterprises using the GitHub Team [$4 per user/month] or GitHub Enterprise Cloud [$21 per user/month] plans.

sunaookami 2y ago
Seems like normal users have to wait: > We expect to begin offering Arm runners for open source projects by the end of the year side note: I find it mildly annoying that it's no longer "ARM" but "Arm"
- VWWHFSfQ 2y ago
  Yeah it's confusing because the company is branded like "Arm" but they still call the architecture "ARM"

joshstrange 2y ago

GitHub’s runners are a joke. Switching to WarpBuild was the best decision I made for my CI/CD. I halved my build time and got cheaper minutes. GH’s macOS runners particularly were complete trash.

deleted 2y ago
[deleted]
999900000999 2y ago
I understand the Mac builds, but the entire appeal of GitHub runners is it's literally built in. I can just add a YAML file, and we have CICD!
- joshstrange 2y ago
  I get that, but you can run GH runners on your own hardware or someone else's. Switching to WarpBuild was literally as easy as linking my GH account and changing the running image from `macOS-latest` to `warp-macos-14-arm64-6x` and everything worked. Secrets and everything.
  - palata 2y ago
    > Secrets and everything. I guess you had to import the secrets, too...
- deleted 2y ago
  [deleted]
yohannparis 2y ago
Different needs, different tools! I work on a open-source project, and not having to pay for CI/CD with a "simple" yml file configuration is simpler and easy to manage.
- 01HNNWZ0MV43FF 2y ago
  It's only free if your time is worthless :P every couple weeks I have to debug something that I can't replicate locally because I don't have local CI

deivid 2y ago

I wonder if they'll start supporting risc-v soon. I've been using github-act-runner on top of a local Nomad cluster to run some of my builds on risc-v, arm64 & x86-64, it's fine, but a bit of a hassle to set up

compsciphd 2y ago
would only happen after azure supports risc-v VMs.
sapiogram 2y ago
Given how long arm64 took them, my guess is "3 years after major cloud vendors start offering risc-v VMs".
- deivid 2y ago
  I'd kinda hoped that they took so long to go arm64 because they generalized "1 architecture -> N architectures"

crohr 2y ago

Much welcome addition, although not available (yet) on free plans. I benchmarked them and they are ~15-20% slower than AWS Graviton processors: https://runs-on.com/benchmarks/github-actions-runners/#arm64-runners https://runs-on.com/benchmarks/github-actions-runners/#arm64... But at 37% cheaper than x64 it's quite a better deal than what the outdated x64 CPUs.

crohr 2y ago
Also a bit weird that they now seem to delegate the image building to third-party vendors (first GPU images, now ARM). Would be nice to know how they validate base images before distributing them to everyone.
ozgune 2y ago
(Ozgun from Ubicloud) Looks like every GitHub Actions provider is on this thread. :) I also wanted to chime in with two thoughts. First, we recently wrote about how we enabled arm64 VMs at Ubicloud. Any feedback on the technical details is more than welcome: https://news.ycombinator.com/item?id=40491377 https://news.ycombinator.com/item?id=40491377 Second, I feel runs-on's analysis is a bit unfair to us. Ubicloud's x64 / arm64 performance and queue times above are as good as any (we deprecated the AMD EPYC 7502 line). For example, RunsOn arm64 queue times are 31|42 seconds and Ubicloud queue times are 17|24 seconds. But the analysis says the following, "Be aware that Ubicloud has low CPU speeds and high queue times, which means they won’t necessarily end up being cheaper than competitors." I fail to see this conclusion from the numbers above. What am I missing?
- crohr 2y ago
  The queue time for x64 appears to vary quite a bit still. But you are correct that it is much better than what it was a few weeks back, when the analysis was written (it was not unusual to see 60s+ spikes). Also note that I specifically mentioned that "all third-parties are good on that front, expected [sic] Ubicloud (but that may change)." Also agree that now that you removed the outdated CPUs (was still active end of May), the analysis should mention that you have OK CPUs by now. I will fix it. For providers that are on this page, do not hesitate to write to me if you find the analysis outdated for your specific service. [edit] page is now up to date with new analysis

suryao 2y ago

I'm so glad that GitHub finally offers this. They're just more efficient for certain kinds of workflows - both cheaper and faster. If you're building for arm64 targets or android, this is a no brainer because it sidesteps the emulation requirements. We've been offering arm64 runners for a few months now (up to 32 vcpu) with WarpBuild and our users love it. In general, there are lots of inefficiencies in CI starting from the lack of visibility, debugging ease, performance, runner config customization, persistence, etc. We're out to solve it and make the world's fastest CI cloud ecosystem on top of existing CI providers with WarpBuild.

christina97 2y ago
Does WarpBuild offer discounts for nonprofits/FOSS?
- suryao 2y ago
  Not at this time. We're a small team trying to build a sustainably growing business with server bills to pay.
- vessy-st6io 2y ago
  We at FlyCI offer macOS runners of GitHub Actions and have free 500 minutes per month for public repositories. Also, we are about to release FlyCI Wingman - an AI agent that analyzes and fixes failing builds. Feel free to contact us at contact@flyci.net and tell us more about your needs. We'll be happy to help you with a solution. FlyCI website: https://www.flyci.net/ https://www.flyci.net/ Discord server: https://discord.gg/JyCjh439da https://discord.gg/JyCjh439da

bhouston 2y ago

Hmm... this is neat. I develop on MacBook so ARM64, and I do use GitHub Actions, which can not be ARM64, but unfortunately Google Cloud Run doesn't support ARM64 images yet: https://issuetracker.google.com/issues/303743857 https://issuetracker.google.com/issues/303743857

stefantalpalaru 2y ago
[dead]
david_allison 2y ago
Excuse my ignorance, isn't `runs-on: macos-14` ARM64-based (M1) https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/ https://github.blog/changelog/2024-01-30-github-actions-maco...
- bhouston 2y ago
  I am looking for Google Cloud Run arm64 support.

booleanbetrayal 2y ago

It's nice to know that we have an ARM64 fallback if our perfectly robust and cheaper self-hosting of GHA (gha-runner-scale-set* components) were ever to go down, I suppose, but I can't help but feel like I am part of a huge market segment that GitHub lost during the ARM64 runner absence. Seems a little too late given they have natively supported action-runners images in ARM64 for some time.

dedene 2y ago

Can it be used to efficiently build multi-arch docker images? Both armv8/x64

caleblloyd 2y ago
Maybe but I’d recommend checking out Depot.Dev for that, they expose Arm and x86-64 runners to buildkit/buildx. Not associated with them, just a happy customer.
pl4nty 2y ago
it's a pain, you'll need to build the images on separate jobs then merge them into a multi-arch manifest. I moved to namespace.so for remote BuildKit builds a while ago and haven't looked back. depot.dev is also good, but pretty expensive

rahkiin 2y ago

Are there third party providers for Gitlab runners? I’d love windows and mac runners for my self hosted gitlab.

frankdejonge 2y ago

One thing that consistently bugs me is how upgrading from the lowest type of runners to _literally_ anything else renders your paid for included minutes useless. I do not understand who those wouldn't just be multiples of the base runner, much like how Mac and Windows runners are. Seems like the crediting logic is there, but knowing software, there is probably some accidental complexity causing this part not to leverage it. That said, it's very frustrating from a paying customer perspective. The same goes for minute crediting, where splitting things up to run concurrently is actively discouraged because they individually all round up to the next minute. For example; have 3 concurrent tasks that run 1m10s? You're billed 6 minutes. I get that a run would be rounded up, but come on.

deleted 2y ago

[deleted]

AndyKelley 2y ago

At Zig Software Foundation we rent an aarch64 Hetzner server for about 100 euros a month and it has 64 processors. It can handle a lot more CI work than the GitHub provided runners for a fraction of the cost.

qwertox 2y ago

I had an issue where a Rust application, which I had compiled on a Radxa ROCK 5B, would nor run on a Raspberry Pi 4; I had to compile it again for it. Both were running Ubuntu 22.04, 64 bit. Why would that be and how would this affect these Arm64 compiles on GitHub? If you compile a normal application for amd64, it will work on any 64 bit Intel and AMD machine, unless you start using specialized features, which wasn't the case in the above mentioned case.

written-beyond 2y ago
What was the error you were getting when you tried to run on the RPI?
rami3l 2y ago
This reminds me of one issue that I've encountered at Rustup. Early versions of ARM64 Pi OS actually use a 32-bit userland. As a result, this has caused user complaints since our installer sees a 64-bit kernel and will happily download a 64-bit binary (which will then refuse to run): https://github.com/rust-lang/rustup/issues/3307#issuecomment-1732175383 https://github.com/rust-lang/rustup/issues/3307#issuecomment...