3 ms·

Building and deploying a custom site using GitHub Actions and GitHub Pages

54 points by linsomniac 2y ago

linsomniac 2y ago
I just set up an apt repo of uv packages using GitHub Pages via Actions and wrote up some notes here: https://linsomniac.com/post/2025-03-18-building_and_publishing_apt_repos_to_github_pages/ https://linsomniac.com/post/2025-03-18-building_and_publishi... Looks like Simon and I were working on very similar things simultaneously.
- westurner 2y ago
  There should be a "Verify signature of checksums and Verify checksums" at "Download latest uv binary". GitHub has package repos; "GitHub Packages" for a few package formats, and OCI artifacts. https://docs.github.com/en/packages/working-with-a-github-packages-registry https://docs.github.com/en/packages/working-with-a-github-pa... OCI Artifacts with labels include signed Container images and signed Packages. Packages can be hosted as OCI image repository artifacts with signatures; but package managers don't natively support OCI image stores, though many can install packages hosted at GitHub Packages URLs which are referenced by a signed package repository Manifest on a GitHub Pages or GitLab Pages site (e.g. with CF DNS and/or CloudFlare Pages in front)
  - linsomniac 2y ago
    >There should be a "Verify signature of checksums and Verify checksums" I get it, but verifying checksums of downloads via https from github releases, downloaded across github's architecture (admittedly Azure) to githubs's runners seems like overkill. Especially as I don't see any signatures of the checksums, so the checksums are being retrieved via that same infrastructure with the same security guarantees. What am I missing?
    - westurner 2y ago
      If downloading over https and installing were enough, we wouldn't need SLSA, TUF, or sigstore. .deb and .rpm Package managers typically reference a .tar.gz with a checksum; commit that to git signed with a GPG key; and then the package is signed by a (reproducible) CI build with the signing key for that repo's packages. conda-forge can automatically send a Pull Request to update the upstream archive URL and cached checksum in the package manifest when there is a new upstream package.
      - westurner 2y ago
        Actually, the conda-forge bot does more than that; From https://github.com/regro/cf-scripts/issues/3920#issuecomment-2746675003 https://github.com/regro/cf-scripts/issues/3920#issuecomment... re: (now github) dependabot maybe someday scanning conda environment.yml, conda-lock.yml, and/or feedstock meta.yml: > So the bot here does a fair bit more than update dependencies and/or versions. > We have globally pinned ABIs which have to migrated in a specific order. We also use the bot here to start the migrations, produce progress / error outputs on the status page, and then close the migrations. The bot here also does specific migrations of dependencies that have been renamed, recipe maintenance, and more. Dependabot scans for vulnerable software package versions in software dependency specification documents for a number of languages and package formats, when you git push to a repo which has dependabot configured in their dependabot.yml: From the dependabot.yml docs: https://docs.github.com/en/code-security/dependabot/working-with-dependabot/dependabot-options-reference#package-ecosystem- https://docs.github.com/en/code-security/dependabot/working-... : > Define one package-ecosystem element for each package manager that you want Dependabot to monitor for new versions Dependabot supports e.g. npm, pip, gomod, cargo, docker, github-actions, devcontainers, SBOM tools have additional methods of determining which packages are installed on a specific server or container instance. Static sites are sometimes build and forget projects that also need regular review of the statically-compiled-in dependencies for known vulns. E.g. jupyterlite-xeus builds WASM static sites from environment.yml; though Hugo is probably much faster.
onli 2y ago
The simplicity of the example might give the wrong impression here. A Github action that produces some static HTML is of course without value, Github pages can host HTML files without needing any workflow action. The idea of the article seems to be that the Github action file can do something interesting regularly, like downloading a json file from somewhere, then produce a page and publish it. That is kind of neat then, as you can have your static site generator offloaded to Github's architecture - instead of needing to run that somewhere else, like your own PC or a dedicated server.
- 0xbadcafebee 2y ago
  Actually it does have value, since I literally do what you describe, I just haven't plumbed the GitHub Pages part yet. Here[1] I keep docs about the truck camper I'm building. I use one GHA[2] to generate the TOC, and another GHA[3] to run make in the root directory and then commit and push modified files (to a PR branch). The make process uses scripts to generate new Markdown pages based on a template, auto-generating Markdown tables from CSV files and inserting them into the final Markdown. By keeping this all as a GHA, I don't have to do anything manual to generate the new site. I just open a PR with my new content and the actions fix up everything. It's basically a janky static site editor but using simpler tools I'm familiar with and can customize easier. I've been wanting to host it as GH Pages but haven't gotten around to it. Partly I need to do this because my 3d models run into GitHub's LFS bandwidth limits and hopefully Pages won't (if I do run into Pages' bandwidth limit, I'll need to find some other cheap hosting, or not share my models anymore =/ ) [1] https://github.com/peterwwillis/truckcamper https://github.com/peterwwillis/truckcamper [2] https://github.com/peterwwillis/truckcamper/blob/main/.github/workflows/toc-generator.yml https://github.com/peterwwillis/truckcamper/blob/main/.githu... [3] https://github.com/peterwwillis/truckcamper/blob/main/.github/workflows/template-generator.yml https://github.com/peterwwillis/truckcamper/blob/main/.githu...
  - onli 2y ago
    Do what works for you of course, but you could also just run your makefile on our PC and then push the generated HTML to the gh-pages branch/repo. I don't see (much? any?) value in running that on a server, if it's not triggered regularly for some external source. There are no simpler tools here, just the same with the gh layer on top.
rcarmo 2y ago
The thing about GitHub Pages is that it doesn't handle redirects and index pages the way I like, besides being quite a bit slow when you have thousands of static items (I ended up building my own custom uploader to Azure storage, which only rebuilds the pages I update and figures out all the asset dependencies, so it also only uploads the bits that need updating).
- simonw 2y ago
  Yeah, the lack of support for redirects is really frustrating - it's the single feature I would most like to see them add. I spent a while exploring how index pages a few years ago and figured out the rules: https://til.simonwillison.net/github/github-pages#user-content-foo-will-serve-content-from-foohtml-if-it-exists https://til.simonwillison.net/github/github-pages#user-conte...
diggan 2y ago
I did something similar recently for deploying some Bevy examples to a webpage: https://github.com/victorb/dogoap/blob/3505cb42784381b696ce5a3f749b6c095ae320b6/.github/workflows/wasm-examples.yml https://github.com/victorb/dogoap/blob/3505cb42784381b696ce5... Probably not a problem unless you work with others or have a tendency to push commits fast/have a slow pipeline (building WASM from Rust is really slow), but it seems you're missing to limit the concurrency, which can lead to old commit being deployed over a newer commit, depending on which finishes first. Also limiting it to your main/master branch makes sense in case you use branches, otherwise WIP branches might deploy over your "production" one :) One final word of advice, try to do as little as possible in the .yml configuration itself (I see Simon is "emulating" the build process inside of the CI config), as testing changes becomes a huge hassle otherwise. Instead, extract it into a shell-script/Makefile/Justfile or something, then call that from your CI config, so you can also run the same code locally.
mclau156 2y ago
https://codename-recon.github.io/Gridlock/ https://codename-recon.github.io/Gridlock/
deleted 2y ago
[deleted]
1ewish 2y ago
I've used GitHub Pages for a long time to serve sites for free, notably https://asciiflow.com https://asciiflow.com, however I have recently moved it and other sites to CloudFlare Pages - it's also got a very generous free tier and many other useful features that GH pages lacks. Setup is simple - https://github.com/lewish/asciiflow/blob/main/.github%2Fworkflows%2Fdeploy.yaml https://github.com/lewish/asciiflow/blob/main/.github%2Fwork... GitHub Actions is just great, and I think people overlook again how much you can do for free! As the author highlights, I use it for running workflows on a schedule, which you can use to rebuild static sites hourly, so they aren't so static after all.
- simonw 2y ago
  I need to spend more time with Cloudflare Pages - I get the impression deploys are faster and they have support for redirects! https://developers.cloudflare.com/pages/configuration/redirects/ https://developers.cloudflare.com/pages/configuration/redire...

54 points by linsomniac 2y ago

I just set up an apt repo of uv packages using GitHub Pages via Actions and wrote up some notes here: https://linsomniac.com/post/2025-03-18-building_and_publishing_apt_repos_to_github_pages/ https://linsomniac.com/post/2025-03-18-building_and_publishi... Looks like Simon and I were working on very similar things simultaneously.

westurner 2y ago
There should be a "Verify signature of checksums and Verify checksums" at "Download latest uv binary". GitHub has package repos; "GitHub Packages" for a few package formats, and OCI artifacts. https://docs.github.com/en/packages/working-with-a-github-packages-registry https://docs.github.com/en/packages/working-with-a-github-pa... OCI Artifacts with labels include signed Container images and signed Packages. Packages can be hosted as OCI image repository artifacts with signatures; but package managers don't natively support OCI image stores, though many can install packages hosted at GitHub Packages URLs which are referenced by a signed package repository Manifest on a GitHub Pages or GitLab Pages site (e.g. with CF DNS and/or CloudFlare Pages in front)
- linsomniac 2y ago
  >There should be a "Verify signature of checksums and Verify checksums" I get it, but verifying checksums of downloads via https from github releases, downloaded across github's architecture (admittedly Azure) to githubs's runners seems like overkill. Especially as I don't see any signatures of the checksums, so the checksums are being retrieved via that same infrastructure with the same security guarantees. What am I missing?
  - westurner 2y ago
    If downloading over https and installing were enough, we wouldn't need SLSA, TUF, or sigstore. .deb and .rpm Package managers typically reference a .tar.gz with a checksum; commit that to git signed with a GPG key; and then the package is signed by a (reproducible) CI build with the signing key for that repo's packages. conda-forge can automatically send a Pull Request to update the upstream archive URL and cached checksum in the package manifest when there is a new upstream package.
    - westurner 2y ago
      Actually, the conda-forge bot does more than that; From https://github.com/regro/cf-scripts/issues/3920#issuecomment-2746675003 https://github.com/regro/cf-scripts/issues/3920#issuecomment... re: (now github) dependabot maybe someday scanning conda environment.yml, conda-lock.yml, and/or feedstock meta.yml: > So the bot here does a fair bit more than update dependencies and/or versions. > We have globally pinned ABIs which have to migrated in a specific order. We also use the bot here to start the migrations, produce progress / error outputs on the status page, and then close the migrations. The bot here also does specific migrations of dependencies that have been renamed, recipe maintenance, and more. Dependabot scans for vulnerable software package versions in software dependency specification documents for a number of languages and package formats, when you git push to a repo which has dependabot configured in their dependabot.yml: From the dependabot.yml docs: https://docs.github.com/en/code-security/dependabot/working-with-dependabot/dependabot-options-reference#package-ecosystem- https://docs.github.com/en/code-security/dependabot/working-... : > Define one package-ecosystem element for each package manager that you want Dependabot to monitor for new versions Dependabot supports e.g. npm, pip, gomod, cargo, docker, github-actions, devcontainers, SBOM tools have additional methods of determining which packages are installed on a specific server or container instance. Static sites are sometimes build and forget projects that also need regular review of the statically-compiled-in dependencies for known vulns. E.g. jupyterlite-xeus builds WASM static sites from environment.yml; though Hugo is probably much faster.

onli 2y ago

The simplicity of the example might give the wrong impression here. A Github action that produces some static HTML is of course without value, Github pages can host HTML files without needing any workflow action. The idea of the article seems to be that the Github action file can do something interesting regularly, like downloading a json file from somewhere, then produce a page and publish it. That is kind of neat then, as you can have your static site generator offloaded to Github's architecture - instead of needing to run that somewhere else, like your own PC or a dedicated server.

0xbadcafebee 2y ago
Actually it does have value, since I literally do what you describe, I just haven't plumbed the GitHub Pages part yet. Here[1] I keep docs about the truck camper I'm building. I use one GHA[2] to generate the TOC, and another GHA[3] to run make in the root directory and then commit and push modified files (to a PR branch). The make process uses scripts to generate new Markdown pages based on a template, auto-generating Markdown tables from CSV files and inserting them into the final Markdown. By keeping this all as a GHA, I don't have to do anything manual to generate the new site. I just open a PR with my new content and the actions fix up everything. It's basically a janky static site editor but using simpler tools I'm familiar with and can customize easier. I've been wanting to host it as GH Pages but haven't gotten around to it. Partly I need to do this because my 3d models run into GitHub's LFS bandwidth limits and hopefully Pages won't (if I do run into Pages' bandwidth limit, I'll need to find some other cheap hosting, or not share my models anymore =/ ) [1] https://github.com/peterwwillis/truckcamper https://github.com/peterwwillis/truckcamper [2] https://github.com/peterwwillis/truckcamper/blob/main/.github/workflows/toc-generator.yml https://github.com/peterwwillis/truckcamper/blob/main/.githu... [3] https://github.com/peterwwillis/truckcamper/blob/main/.github/workflows/template-generator.yml https://github.com/peterwwillis/truckcamper/blob/main/.githu...
- onli 2y ago
  Do what works for you of course, but you could also just run your makefile on our PC and then push the generated HTML to the gh-pages branch/repo. I don't see (much? any?) value in running that on a server, if it's not triggered regularly for some external source. There are no simpler tools here, just the same with the gh layer on top.

rcarmo 2y ago

The thing about GitHub Pages is that it doesn't handle redirects and index pages the way I like, besides being quite a bit slow when you have thousands of static items (I ended up building my own custom uploader to Azure storage, which only rebuilds the pages I update and figures out all the asset dependencies, so it also only uploads the bits that need updating).

simonw 2y ago
Yeah, the lack of support for redirects is really frustrating - it's the single feature I would most like to see them add. I spent a while exploring how index pages a few years ago and figured out the rules: https://til.simonwillison.net/github/github-pages#user-content-foo-will-serve-content-from-foohtml-if-it-exists https://til.simonwillison.net/github/github-pages#user-conte...

diggan 2y ago

I did something similar recently for deploying some Bevy examples to a webpage: https://github.com/victorb/dogoap/blob/3505cb42784381b696ce5a3f749b6c095ae320b6/.github/workflows/wasm-examples.yml https://github.com/victorb/dogoap/blob/3505cb42784381b696ce5... Probably not a problem unless you work with others or have a tendency to push commits fast/have a slow pipeline (building WASM from Rust is really slow), but it seems you're missing to limit the concurrency, which can lead to old commit being deployed over a newer commit, depending on which finishes first. Also limiting it to your main/master branch makes sense in case you use branches, otherwise WIP branches might deploy over your "production" one :) One final word of advice, try to do as little as possible in the .yml configuration itself (I see Simon is "emulating" the build process inside of the CI config), as testing changes becomes a huge hassle otherwise. Instead, extract it into a shell-script/Makefile/Justfile or something, then call that from your CI config, so you can also run the same code locally.

mclau156 2y ago

https://codename-recon.github.io/Gridlock/ https://codename-recon.github.io/Gridlock/

deleted 2y ago

[deleted]

1ewish 2y ago

I've used GitHub Pages for a long time to serve sites for free, notably https://asciiflow.com https://asciiflow.com, however I have recently moved it and other sites to CloudFlare Pages - it's also got a very generous free tier and many other useful features that GH pages lacks. Setup is simple - https://github.com/lewish/asciiflow/blob/main/.github%2Fworkflows%2Fdeploy.yaml https://github.com/lewish/asciiflow/blob/main/.github%2Fwork... GitHub Actions is just great, and I think people overlook again how much you can do for free! As the author highlights, I use it for running workflows on a schedule, which you can use to rebuild static sites hourly, so they aren't so static after all.

simonw 2y ago
I need to spend more time with Cloudflare Pages - I get the impression deploys are faster and they have support for redirects! https://developers.cloudflare.com/pages/configuration/redirects/ https://developers.cloudflare.com/pages/configuration/redire...