5 ms·

Creating signed GitHub releases

111 points by ashitlerferad 11y ago

creshal 11y ago
Or, alternatively: 1. Tag your release with `git tag -s` 2. `git push --follow-tags` (or `git push remotename tagname` in git <1.8.3) 3. Upload your pubkey to github. Much less effort…
- dsp1234 11y ago
  Do the above steps generate an actual signed release, or just a signed tag?
  - creshal 11y ago
    A tag, not a release. I didn't realize there was a difference, but in hindsight, I'd argue it's superior anyway: I'm much more comfortable to sign my local repository copy than signing an archive generated by github in a non-reproducible way. I can verify the contents of github's archives, but I cannot, in good faith, say I've verified that the archives do not contain malicious code that triggers vulnerabilities in zip/tar/gzip.
    - chungy 11y ago
      Reproducing GitHub's archives locally is trivial. They just use a standard git command to do it anyway. git archive -o project-0.5.tar.gz --prefix project-0.5/ v0.5
      - creshal 11y ago
        I think we have different standards for "reproducible". You can replicate the archive structure and naming, but the archive metadata – and thus checksums – will be different: $ curl -sL https://github.com/creshal/yspave/archive/0.1.0.zip https://github.com/creshal/yspave/archive/0.1.0.zip | sha512sum 375564f127b46f3d1208c30b4eb5c39725985ca09d7bacf940d346727e8bcf9362966f8523c39e6c4f6f1a86b52ceba8823c2f4b016fdedb5947cd6a6bafc930 - $ git clone git@github.com:creshal/yspave.git … $ cd yspave $ git archive -o 0.1.0.zip --prefix yspave-0.1.0/ 0.1.0 $ sha512sum 0.1.0.zip aca1163d5270e8ad6796f10511f49b0cf5fe61007fed0786a6643ae522e59a6aa7d6b0cddf4e5b94a435da8d99aa09b2f2ebef7788a82d4c2e7d8d4ad79e0634 0.1.0.zip ––––– So, I can verify that the contents weren't tampered with, but I cannot verify the archives themselves unless I happen to understand the zip/tar/gz… specifications better than most people implementing it.
        chungy 11y ago
        That's interesting. I'm not able to replicate your results. Cloning that respository and doing the same git archive command produces an identical file to what GitHub serves. I'm on a fully updated Arch Linux with the latest Git. If you have an older version, that might be making the difference.
        Xylakant 11y ago
        I can reproduce the result: The content of the file is identical, the checksum differs.
        deleted 11y ago
        [deleted]
        creshal 11y ago
        > I'm on a fully updated Arch Linux with the latest Git. Same, actually.
        Xylakant 11y ago
        you can verify by downloading the archive and comparing the content of the archive with the content of the archive produced by git archive. The issue seems to be that the zip file contains the version of the zip utility used to create the file, so you'd need the exact same version to make a reproducible build.
        yrro 11y ago
        According to diffoscope, the (only) differences are in the archive member modification times: --- ../0.1.0.zip +++ 0.1.0.zip ├── zipinfo -v {} │ @@ -28,15 +28,15 @@ │ file system or operating system of origin: MS-DOS, OS/2 or NT FAT │ version of encoding software: 0.0 │ minimum file system compatibility required: MS-DOS, OS/2 or NT FAT │ minimum software version required to extract: 1.0 │ compression method: none (stored) │ file security status: not encrypted │ extended local header: no │ - file last modified on (DOS date/time): 2016 Apr 8 06:33:12 │ + file last modified on (DOS date/time): 2016 Apr 8 14:33:12 │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 local │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 UTC │ 32-bit CRC value (hex): 00000000 │ compressed size: 0 bytes │ uncompressed size: 0 bytes │ length of filename: 13 characters │ length of extra field: 9 bytes │ @@ -63,15 +63,15 @@ │ version of encoding software: 0.0 │ minimum file system compatibility required: MS-DOS, OS/2 or NT FAT │ minimum software version required to extract: 1.0 │ compression method: deflated │ compression sub-type (deflation): normal │ file security status: not encrypted │ extended local header: no │ - file last modified on (DOS date/time): 2016 Apr 8 06:33:12 │ + file last modified on (DOS date/time): 2016 Apr 8 14:33:12 │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 local │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 UTC │ 32-bit CRC value (hex): 570b4391 │ compressed size: 52 bytes │ uncompressed size: 78 bytes │ length of filename: 23 characters │ length of extra field: 9 bytes │ @@ -97,15 +97,15 @@ │ file system or operating system of origin: Unix │ version of encoding software: 2.3 │ minimum file system compatibility required: MS-DOS, OS/2 or NT FAT │ minimum software version required to extract: 1.0 │ compression method: none (stored) │ file security status: not encrypted │ extended local header: no │ - file last modified on (DOS date/time): 2016 Apr 8 06:33:12 │ + file last modified on (DOS date/time): 2016 Apr 8 14:33:12 │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 local │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 UTC │ 32-bit CRC value (hex): bc890775 │ compressed size: 59 bytes │ uncompressed size: 59 bytes │ length of filename: 20 characters │ length of extra field: 9 bytes (rest of output omitted as it's too long for an HN comment; the full output can be found at http://paste.debian.net/432708/ http://paste.debian.net/432708/)
      - creshal 11y ago
        Ahh, never mind. The tar.gz is indeed identical, "only" the zip differs.
    - dsp1234 11y ago
      I'm much more comfortable to sign my local repository copy than signing an archive generated by github Both can be part of the process. However, signing the tag doesn't really help someone who isn't cloning the repository. For those people that are just downloading the source tarball directly, having a GPG signature for the release can ensure that github(or other attacker) hasn't modified the contents of the tarball.
      - ownagefool 11y ago
        Sort of. I mean if github wants to modify the contents of the tarball, they'll also modify the signature. So unless you're also confirming that it's the right signature, and not just that it's signed, what exactly are you doing?
        ashitlerferad 11y ago
        That is where signing other people's OpenPGP keys comes in, and the web of trust for those who haven't met their upstreams. github can also modify signed tags.
- Ruud-v-A 11y ago
  Exactly. It has the additional advantage of signing the history, not just one source dump.
- FiloSottile 11y ago
  That does not nearly have the same security level. Signing a tag in git is like signing a commit. A commit is nothing else than a reference to a tree SHA1 (and parent, message and author). That tree SHA1 contains SHA1 of the files and trees in the directory. Even assuming your client checks all the SHA1 and does not just trust the server (which is not the default), and that you are verifying the git tag signature correctly (which is harder than just checking a tarball), SHA1 collisions are considered practical.
  - brillenfux 11y ago
    Git is essentially a Merkle DAG, thus a single reference to the root object suffices. Also collisions are only practical with large amounts of padding which would be visible. That's why nobody really cares too much about them for source control systems.
    - deleted 11y ago
      [deleted]
embik 11y ago
> 4. Go back to your "Releases" section and download the tarball mysoftware-0.4.tar.gz automatically generated by GitHub. If only. That's by far my biggest irk with GitHub releases because they only use the tag as name for the tarball (e.g. v0.4.tar.gz). Building RPM packages with those tarballs is a bit of a pain because RPM building tools expect all tarballs in one directory and you don't know which tarball belongs to which software. Maybe I'm just being ignorant here.
- creshal 11y ago
  It's not github's fault that RPM's tooling is stuck in the 90s, to be honest. makepkg and others support renaming tarballs during download to cover this use case.
  - embik 11y ago
    I don't disagree, but it just feels off to tell a packaging tool to rename a tarball because the default name is not verbose enough to even identify the tarball.
    - justincormack 11y ago
      But even so it is designed to be likely to get accidental collisions.
- jsizz 11y ago
  Yes, you're being ignorant. Use '--content-disposition' with wget, or '--remote-name' in curl. The top level directory in the tarball corresponds to this name and everything works out as expected.
  - embik 11y ago
    It's not only the name of the tarball after downloading it. RPM SPEC files require me to pass a full URL to the tarball's upstream source[1]: > Source0: The full URL for the compressed archive containing the (original) pristine source code, as upstream released it. "Source" is synonymous with "Source0". If you give a full URL (and you should), its basename will be used when looking in the SOURCES directory. But the only URL that is available is <Project URL>/archive/v0.1.1.tar.gz. The SPEC file is therefore defining a file named v0.1.1.tar.gz although I downloaded my tarball as foo-0.1.1.tar.gz. The packaging process looks for v0.1.1.tar.gz, fails to find it in my SOURCES directory and aborts. I'm not blaming GitHub here in any way, I just wish I had a bit more flexibility for the tarball URL. Would be neat. [1] https://fedoraproject.org/wiki/How_to_create_an_RPM_package#SPEC_file_overview https://fedoraproject.org/wiki/How_to_create_an_RPM_package#...
rickycook 11y ago
They missed the bit about verifying the validity of the content of the tarball... Given that you're essentially guarding against GitHub going rogue, that's half the point right?
- creshal 11y ago
  Verifying the content is one thing, what about making sure the archive itself isn't malicious and triggering tar/gzip/zip vulnerabilities? In my experience, github's archives are not generated in a reproducible way.
  - creshal 11y ago
    Never mind that comment, as established in a different comment chain, it's an inherent limitation of zip archives. .tar.gz archives are fully reproducible.
  - ashitlerferad 11y ago
    github archives are definitely generated in a reproducible way.
    - LukeShu 11y ago
      They are now. They weren't 2 or 3 years ago.
richardwhiuk 11y ago
What's the threat model you are defending against here? GitHub changing the binary being downloaded? They could serve a different signature. Another committer changing the tag? They could equally change the signature....
- lmm 11y ago
  > GitHub changing the binary being downloaded? They could serve a different signature. > Another committer changing the tag? They could equally change the signature.... No they couldn't, not unless they have your private key. (Debian does actually require all keys used to sign debian packages to have a trust path back to them - if you want to become Debian maintainer for a project you have to meet up with a Debian developer and have them sign your GPG key - so GitHub or another committer couldn't just use a different key to sign with).
- edwintorok 11y ago
  Github doesn't have your private key, so they won't be able to serve a valid signature. The only thing they could do is to substitute an older signed version when you request a newer version.
  - ashitlerferad 11y ago
    They could create a new private key using your name. Most people will trust 'Verified' on a web page more than the OpenPGP web of trust.
    - sneak 11y ago
      Not sure why you're being downvoted, this "what is the threat model here" was my first thought, too, and this comment is also correct, as well.
      - ashitlerferad 11y ago
        Probably my "Most people" quip.
dmvaldman 11y ago
There appear to be a lot of shots taken behind the basket - even as far back at the 3 pt line. Is this a mistake in measurement?
- sb8244 11y ago
  https://news.ycombinator.com/item?id=11495374 https://news.ycombinator.com/item?id=11495374
cdnsteve 11y ago
Signed releases should be required for any package management system. I'm finding Ruby to be lacking in this area: `bundle install --trust-policy HighSecurity`

111 points by ashitlerferad 11y ago

Or, alternatively: 1. Tag your release with `git tag -s` 2. `git push --follow-tags` (or `git push remotename tagname` in git <1.8.3) 3. Upload your pubkey to github. Much less effort…

dsp1234 11y ago
Do the above steps generate an actual signed release, or just a signed tag?
- creshal 11y ago
  A tag, not a release. I didn't realize there was a difference, but in hindsight, I'd argue it's superior anyway: I'm much more comfortable to sign my local repository copy than signing an archive generated by github in a non-reproducible way. I can verify the contents of github's archives, but I cannot, in good faith, say I've verified that the archives do not contain malicious code that triggers vulnerabilities in zip/tar/gzip.
  - chungy 11y ago
    Reproducing GitHub's archives locally is trivial. They just use a standard git command to do it anyway. git archive -o project-0.5.tar.gz --prefix project-0.5/ v0.5
    - creshal 11y ago
      I think we have different standards for "reproducible". You can replicate the archive structure and naming, but the archive metadata – and thus checksums – will be different: $ curl -sL https://github.com/creshal/yspave/archive/0.1.0.zip https://github.com/creshal/yspave/archive/0.1.0.zip | sha512sum 375564f127b46f3d1208c30b4eb5c39725985ca09d7bacf940d346727e8bcf9362966f8523c39e6c4f6f1a86b52ceba8823c2f4b016fdedb5947cd6a6bafc930 - $ git clone git@github.com:creshal/yspave.git … $ cd yspave $ git archive -o 0.1.0.zip --prefix yspave-0.1.0/ 0.1.0 $ sha512sum 0.1.0.zip aca1163d5270e8ad6796f10511f49b0cf5fe61007fed0786a6643ae522e59a6aa7d6b0cddf4e5b94a435da8d99aa09b2f2ebef7788a82d4c2e7d8d4ad79e0634 0.1.0.zip ––––– So, I can verify that the contents weren't tampered with, but I cannot verify the archives themselves unless I happen to understand the zip/tar/gz… specifications better than most people implementing it.
      - chungy 11y ago
        That's interesting. I'm not able to replicate your results. Cloning that respository and doing the same git archive command produces an identical file to what GitHub serves. I'm on a fully updated Arch Linux with the latest Git. If you have an older version, that might be making the difference.
        Xylakant 11y ago
        I can reproduce the result: The content of the file is identical, the checksum differs.
        deleted 11y ago
        [deleted]
        creshal 11y ago
        > I'm on a fully updated Arch Linux with the latest Git. Same, actually.
      - Xylakant 11y ago
        you can verify by downloading the archive and comparing the content of the archive with the content of the archive produced by git archive. The issue seems to be that the zip file contains the version of the zip utility used to create the file, so you'd need the exact same version to make a reproducible build.
      - yrro 11y ago
        According to diffoscope, the (only) differences are in the archive member modification times: --- ../0.1.0.zip +++ 0.1.0.zip ├── zipinfo -v {} │ @@ -28,15 +28,15 @@ │ file system or operating system of origin: MS-DOS, OS/2 or NT FAT │ version of encoding software: 0.0 │ minimum file system compatibility required: MS-DOS, OS/2 or NT FAT │ minimum software version required to extract: 1.0 │ compression method: none (stored) │ file security status: not encrypted │ extended local header: no │ - file last modified on (DOS date/time): 2016 Apr 8 06:33:12 │ + file last modified on (DOS date/time): 2016 Apr 8 14:33:12 │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 local │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 UTC │ 32-bit CRC value (hex): 00000000 │ compressed size: 0 bytes │ uncompressed size: 0 bytes │ length of filename: 13 characters │ length of extra field: 9 bytes │ @@ -63,15 +63,15 @@ │ version of encoding software: 0.0 │ minimum file system compatibility required: MS-DOS, OS/2 or NT FAT │ minimum software version required to extract: 1.0 │ compression method: deflated │ compression sub-type (deflation): normal │ file security status: not encrypted │ extended local header: no │ - file last modified on (DOS date/time): 2016 Apr 8 06:33:12 │ + file last modified on (DOS date/time): 2016 Apr 8 14:33:12 │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 local │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 UTC │ 32-bit CRC value (hex): 570b4391 │ compressed size: 52 bytes │ uncompressed size: 78 bytes │ length of filename: 23 characters │ length of extra field: 9 bytes │ @@ -97,15 +97,15 @@ │ file system or operating system of origin: Unix │ version of encoding software: 2.3 │ minimum file system compatibility required: MS-DOS, OS/2 or NT FAT │ minimum software version required to extract: 1.0 │ compression method: none (stored) │ file security status: not encrypted │ extended local header: no │ - file last modified on (DOS date/time): 2016 Apr 8 06:33:12 │ + file last modified on (DOS date/time): 2016 Apr 8 14:33:12 │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 local │ file last modified on (UT extra field modtime): 2016 Apr 8 13:33:13 UTC │ 32-bit CRC value (hex): bc890775 │ compressed size: 59 bytes │ uncompressed size: 59 bytes │ length of filename: 20 characters │ length of extra field: 9 bytes (rest of output omitted as it's too long for an HN comment; the full output can be found at http://paste.debian.net/432708/ http://paste.debian.net/432708/)
    - creshal 11y ago
      Ahh, never mind. The tar.gz is indeed identical, "only" the zip differs.
  - dsp1234 11y ago
    I'm much more comfortable to sign my local repository copy than signing an archive generated by github Both can be part of the process. However, signing the tag doesn't really help someone who isn't cloning the repository. For those people that are just downloading the source tarball directly, having a GPG signature for the release can ensure that github(or other attacker) hasn't modified the contents of the tarball.
    - ownagefool 11y ago
      Sort of. I mean if github wants to modify the contents of the tarball, they'll also modify the signature. So unless you're also confirming that it's the right signature, and not just that it's signed, what exactly are you doing?
      - ashitlerferad 11y ago
        That is where signing other people's OpenPGP keys comes in, and the web of trust for those who haven't met their upstreams. github can also modify signed tags.
Ruud-v-A 11y ago
Exactly. It has the additional advantage of signing the history, not just one source dump.
FiloSottile 11y ago
That does not nearly have the same security level. Signing a tag in git is like signing a commit. A commit is nothing else than a reference to a tree SHA1 (and parent, message and author). That tree SHA1 contains SHA1 of the files and trees in the directory. Even assuming your client checks all the SHA1 and does not just trust the server (which is not the default), and that you are verifying the git tag signature correctly (which is harder than just checking a tarball), SHA1 collisions are considered practical.
- brillenfux 11y ago
  Git is essentially a Merkle DAG, thus a single reference to the root object suffices. Also collisions are only practical with large amounts of padding which would be visible. That's why nobody really cares too much about them for source control systems.
  - deleted 11y ago
    [deleted]

embik 11y ago

> 4. Go back to your "Releases" section and download the tarball mysoftware-0.4.tar.gz automatically generated by GitHub. If only. That's by far my biggest irk with GitHub releases because they only use the tag as name for the tarball (e.g. v0.4.tar.gz). Building RPM packages with those tarballs is a bit of a pain because RPM building tools expect all tarballs in one directory and you don't know which tarball belongs to which software. Maybe I'm just being ignorant here.

creshal 11y ago
It's not github's fault that RPM's tooling is stuck in the 90s, to be honest. makepkg and others support renaming tarballs during download to cover this use case.
- embik 11y ago
  I don't disagree, but it just feels off to tell a packaging tool to rename a tarball because the default name is not verbose enough to even identify the tarball.
  - justincormack 11y ago
    But even so it is designed to be likely to get accidental collisions.
jsizz 11y ago
Yes, you're being ignorant. Use '--content-disposition' with wget, or '--remote-name' in curl. The top level directory in the tarball corresponds to this name and everything works out as expected.
- embik 11y ago
  It's not only the name of the tarball after downloading it. RPM SPEC files require me to pass a full URL to the tarball's upstream source[1]: > Source0: The full URL for the compressed archive containing the (original) pristine source code, as upstream released it. "Source" is synonymous with "Source0". If you give a full URL (and you should), its basename will be used when looking in the SOURCES directory. But the only URL that is available is <Project URL>/archive/v0.1.1.tar.gz. The SPEC file is therefore defining a file named v0.1.1.tar.gz although I downloaded my tarball as foo-0.1.1.tar.gz. The packaging process looks for v0.1.1.tar.gz, fails to find it in my SOURCES directory and aborts. I'm not blaming GitHub here in any way, I just wish I had a bit more flexibility for the tarball URL. Would be neat. [1] https://fedoraproject.org/wiki/How_to_create_an_RPM_package#SPEC_file_overview https://fedoraproject.org/wiki/How_to_create_an_RPM_package#...

rickycook 11y ago

They missed the bit about verifying the validity of the content of the tarball... Given that you're essentially guarding against GitHub going rogue, that's half the point right?

creshal 11y ago
Verifying the content is one thing, what about making sure the archive itself isn't malicious and triggering tar/gzip/zip vulnerabilities? In my experience, github's archives are not generated in a reproducible way.
- creshal 11y ago
  Never mind that comment, as established in a different comment chain, it's an inherent limitation of zip archives. .tar.gz archives are fully reproducible.
- ashitlerferad 11y ago
  github archives are definitely generated in a reproducible way.
  - LukeShu 11y ago
    They are now. They weren't 2 or 3 years ago.

richardwhiuk 11y ago

What's the threat model you are defending against here? GitHub changing the binary being downloaded? They could serve a different signature. Another committer changing the tag? They could equally change the signature....

lmm 11y ago
> GitHub changing the binary being downloaded? They could serve a different signature. > Another committer changing the tag? They could equally change the signature.... No they couldn't, not unless they have your private key. (Debian does actually require all keys used to sign debian packages to have a trust path back to them - if you want to become Debian maintainer for a project you have to meet up with a Debian developer and have them sign your GPG key - so GitHub or another committer couldn't just use a different key to sign with).
edwintorok 11y ago
Github doesn't have your private key, so they won't be able to serve a valid signature. The only thing they could do is to substitute an older signed version when you request a newer version.
- ashitlerferad 11y ago
  They could create a new private key using your name. Most people will trust 'Verified' on a web page more than the OpenPGP web of trust.
  - sneak 11y ago
    Not sure why you're being downvoted, this "what is the threat model here" was my first thought, too, and this comment is also correct, as well.
    - ashitlerferad 11y ago
      Probably my "Most people" quip.

dmvaldman 11y ago

There appear to be a lot of shots taken behind the basket - even as far back at the 3 pt line. Is this a mistake in measurement?

sb8244 11y ago
https://news.ycombinator.com/item?id=11495374 https://news.ycombinator.com/item?id=11495374

cdnsteve 11y ago

Signed releases should be required for any package management system. I'm finding Ruby to be lacking in this area: `bundle install --trust-policy HighSecurity`