6 ms·

What kinds of things do you need to check for that aren’t automated? Maybe I’m spoiled by our CI pipeline. I’d love to take a look at your list

vortico 7y ago
Here's a somewhat redacted version. `m` is my alias for `make`. - Finalize source - Legal check `LICENSE-dist.txt` - Update `CHANGELOG.md` - Bump version in `Makefile` and `Core.json` - Commit "Bump version". - Build (three OS's can be done in parallel) - `m clean` - `git pull` - Make sure you have the latest `Fundamental.zip` package in source root. - `m dist` - Manually test installer and fragile features (audio drivers, patch loading) for ~10 minutes. - `m notarize` (on Mac) - `m upload` - `git tag vX.Y.Z` - `git push --tags` - Release - Update version title and URLs in `Rack.pug` - `m upload` - At this point, normal users have access to new version. - Update server version in `config.coffee` - `m restart` - At this point, normal users will swarm to download new version. Keep an eye on server bandwidth. - Publicize - Twitter https://twitter.com/home - Facebook https://www.facebook.com/vcvrack/ - Share on group - Forum https://community.vcvrack.com/c/announcements
- enriquto 7y ago
  this list is a very simple algorithm. Why don't you tun it into a script deploy.sh ?
  - vortico 7y ago
    No it's not.
    - enriquto 7y ago
      But it almost is, already! Half of the lines are literally shell commands. And except maybe for the legal check, the rest of the steps can also be written as shell one-liners. For example, to update the version on a README file you can "sed -i" with an appropriate replacement, and so on.
      - vortico 7y ago
        Can you explain how your ./deploy.sh script forks onto three laptops (Windows/Mac/Linux) and "plays around" with the installers/build a bit to make sure it works? I obviously can't release builds that don't even run because I didn't take a few minutes to test them.
        enriquto 7y ago
        Why don't you use virtual machines?
        vortico 7y ago
        Good question. 1) A MacOS virtual machine is probably illegal. 2) I need to test the builds on all three machines, and VMs don't handle OpenGL, USB drivers (e.g. for MIDI controllers), and audio drivers very well or at all. 3) I need to have a rough idea of performance of the build. VMs affect performance by non-constant/unpredictable factors. For example, CPU performance might be 1x the speed of a bare-metal OS, but graphics performance might be 0.25x. 4) I don't know how to write a shell script on a host computer that launches Windows 10 and runs commands with an MSYS2 Mingw64 shell.
        mr_mitm 7y ago
        > I don't know how to write a shell script on a host computer that launches Windows 10 and runs commands with an MSYS2 Mingw64 shell. Launching a VM via shell script is trivial. And you can install OpenSSH on Windows 10: https://github.com/PowerShell/Win32-OpenSSH https://github.com/PowerShell/Win32-OpenSSH
        enriquto 7y ago
        > A MacOS virtual machine is probably illegal. true. And that is a very sad state of affairs. I use the travis osx hosts for that, but it's not ideal. There's no interface, but at least you can check that the code compiles, runs, and passes automated tests. That's already huge! For linux and windows hosts, it seems to me that it is a solved problem, as pointed elsewhere.
  - tbrownaw 7y ago
    Some of the steps look like they'd be hard to do that for. Like for example the one about making sure the license is correct, and the one about testing parts that apparently don't have good automated tests / interact with the real world.
    - enriquto 7y ago
      > Some of the steps look like they'd be hard to do that for. Then those steps must be removed. I thought the mantra "deploy is ONE step" was a more or less universally acknowledged truth.
      - antoinevg 7y ago
        It looks like you're new here. Please repeat after me: In software engineering there are no more or less universally acknowledged truths.
        yjftsjthsd-h 7y ago
        Unfortunately, not everyone acknowledges this truth;)
        antoinevg 7y ago
        …or has a sense of humour about it! ;-)
      - creatornator 7y ago
        I mean unless a computer can check that all dependencies have compatible licenses, that is unrealistic. It's a necessary step for publicly released software to avoid legal issues
        cyphar 7y ago
        Linux distributions (though I can only speak to openSUSE's process) have automated scripts (which I believe are written by our lawyers) which check whether the license of a package matches the license of the files inside the package. It's how most package legal review gets done (and if the script can't figure it out, it gets escalated to our actual lawyers too review. You cannot submit a package to any one of the distributions we ship without the legal review being approved. So it is clearly possible to do -- and there are all sorts of tools which figure out what SPDX license entries apply for every dependency (or vendored dependency) of a given project.
  - happytoexplain 7y ago
    How have you come to that conclusion?
  - gdfasfklshg4 7y ago
    Because it involves inherently manual processes.
- jl-gitlab 7y ago
  Interesting list, thanks for sharing. I'm the PM for CI/CD at GitLab, and I don't mean to hijack the topic, but wanted to give a heads up that we are looking at building a feature for these kinds of procedures that aren't quite pipelines, based on Jupyter Runbooks. Would love to hear your feedback: https://gitlab.com/gitlab-org/gitlab/issues/9427 https://gitlab.com/gitlab-org/gitlab/issues/9427
  - vortico 7y ago
    Currently the most difficult and most time consuming step in the release checklist is finding space on my desk for three laptops so I can use them all simultaneously. Releasing only takes 1-3 hours, and I unfortunately wouldn't want to spend hours learning and setting up automated methods to perhaps shave 10 minutes off the process. Manual releasing allows me to iteratively improve the process and spot possible issues, whereas any automation would reduce build reliability.
    - judge2020 7y ago
      > whereas any automation would reduce build reliability. I guess this depends on what your release process is; for example, most npm libraries benefit from CI that runs the build script and runs `npm version $TAG && npm publish`. There's not much that could be improved here, especially if the build script is just an alias for running `webpack`.
- scarejunba 7y ago
  There's a method I read about on lobste.rs for this. You write down the list as you did and then turn pieces into "Do X by running Y" and over time more things get automated. That's a hefty deploy protocol, though, and automating cross-platform builds may be more trouble than it's worth if you're releasing rarely. It's pretty neat you follow a deploy checklist.
  - rgoulter 7y ago
    Related to the process of "write down what you did", I'm reminded of this article about "literate sysadmin/devops" which used Emacs' org-mode as a way of combining prose, commands and command output. http://howardism.org/Technical/Emacs/literate-devops.html http://howardism.org/Technical/Emacs/literate-devops.html
  - maccard 7y ago
    I read you should write it as an interactive script e.g.: input("Build X") input("Build Y") input("Zip X & Y") input("Upload release") THen run the script and press enter as you do the steps. It means if you're interrupted as your proceed, you can resume exacly where you were before. If you implement it as functions: def buildX(): input("Build X") def buildY(): input("Build Y") def ZipUp(): input("Zip X & Y") def Upload(): input("Upload release") if __name__ == '__main__': buildX() buildY() ZipUp() Upload() Then you can automate parts piecemeal.
- anon9001 7y ago
  I can do every step on your list with any CI platform. I do something extremely similar to build apps for ios (macos), android (linux), and web (linux). I think it's so important that it's one of the first things I do on any project. I wanted to give you an example, but it all seems so easy, I'm not sure where you're getting stuck.
  - gdfasfklshg4 7y ago
    Can you really do a legal review from a CI platform?
    - wwright 7y ago
      If it can be automated, then obviously, yeah. If not fully automated, many CI platforms allow you to pause the pipeline by requiring manual confirmation from an authorized user. I know Jenkins and CircleCI support this off the top of my head. You could have the CI perform any relevant searches or diffs, then display that to the user to get manual confirmation. It’s still not “perfect”, but it does allow you to reliably get someone to look at the data and say “yes” or “no.”
      - anon9001 7y ago
        Yeah, that's one approach. Another might be to commit some kind of `./legal/sign-off-v1.0.0` so you can see who did it, and so travis can know if it should deploy the build or not
  - deleted 7y ago
    [deleted]
gourlaysama 7y ago
The Scala release checklist [1] is a good example of a release checklist with both automated and manual steps. [1]: https://github.com/scala/scala-dev/issues/645 https://github.com/scala/scala-dev/issues/645

vortico 7y ago

Here's a somewhat redacted version. `m` is my alias for `make`. - Finalize source - Legal check `LICENSE-dist.txt` - Update `CHANGELOG.md` - Bump version in `Makefile` and `Core.json` - Commit "Bump version". - Build (three OS's can be done in parallel) - `m clean` - `git pull` - Make sure you have the latest `Fundamental.zip` package in source root. - `m dist` - Manually test installer and fragile features (audio drivers, patch loading) for ~10 minutes. - `m notarize` (on Mac) - `m upload` - `git tag vX.Y.Z` - `git push --tags` - Release - Update version title and URLs in `Rack.pug` - `m upload` - At this point, normal users have access to new version. - Update server version in `config.coffee` - `m restart` - At this point, normal users will swarm to download new version. Keep an eye on server bandwidth. - Publicize - Twitter https://twitter.com/home - Facebook https://www.facebook.com/vcvrack/ - Share on group - Forum https://community.vcvrack.com/c/announcements

enriquto 7y ago
this list is a very simple algorithm. Why don't you tun it into a script deploy.sh ?
- vortico 7y ago
  No it's not.
  - enriquto 7y ago
    But it almost is, already! Half of the lines are literally shell commands. And except maybe for the legal check, the rest of the steps can also be written as shell one-liners. For example, to update the version on a README file you can "sed -i" with an appropriate replacement, and so on.
    - vortico 7y ago
      Can you explain how your ./deploy.sh script forks onto three laptops (Windows/Mac/Linux) and "plays around" with the installers/build a bit to make sure it works? I obviously can't release builds that don't even run because I didn't take a few minutes to test them.
      - enriquto 7y ago
        Why don't you use virtual machines?
        vortico 7y ago
        Good question. 1) A MacOS virtual machine is probably illegal. 2) I need to test the builds on all three machines, and VMs don't handle OpenGL, USB drivers (e.g. for MIDI controllers), and audio drivers very well or at all. 3) I need to have a rough idea of performance of the build. VMs affect performance by non-constant/unpredictable factors. For example, CPU performance might be 1x the speed of a bare-metal OS, but graphics performance might be 0.25x. 4) I don't know how to write a shell script on a host computer that launches Windows 10 and runs commands with an MSYS2 Mingw64 shell.
        mr_mitm 7y ago
        > I don't know how to write a shell script on a host computer that launches Windows 10 and runs commands with an MSYS2 Mingw64 shell. Launching a VM via shell script is trivial. And you can install OpenSSH on Windows 10: https://github.com/PowerShell/Win32-OpenSSH https://github.com/PowerShell/Win32-OpenSSH
        enriquto 7y ago
        > A MacOS virtual machine is probably illegal. true. And that is a very sad state of affairs. I use the travis osx hosts for that, but it's not ideal. There's no interface, but at least you can check that the code compiles, runs, and passes automated tests. That's already huge! For linux and windows hosts, it seems to me that it is a solved problem, as pointed elsewhere.
- tbrownaw 7y ago
  Some of the steps look like they'd be hard to do that for. Like for example the one about making sure the license is correct, and the one about testing parts that apparently don't have good automated tests / interact with the real world.
  - enriquto 7y ago
    > Some of the steps look like they'd be hard to do that for. Then those steps must be removed. I thought the mantra "deploy is ONE step" was a more or less universally acknowledged truth.
    - antoinevg 7y ago
      It looks like you're new here. Please repeat after me: In software engineering there are no more or less universally acknowledged truths.
      - yjftsjthsd-h 7y ago
        Unfortunately, not everyone acknowledges this truth;)
        antoinevg 7y ago
        …or has a sense of humour about it! ;-)
    - creatornator 7y ago
      I mean unless a computer can check that all dependencies have compatible licenses, that is unrealistic. It's a necessary step for publicly released software to avoid legal issues
      - cyphar 7y ago
        Linux distributions (though I can only speak to openSUSE's process) have automated scripts (which I believe are written by our lawyers) which check whether the license of a package matches the license of the files inside the package. It's how most package legal review gets done (and if the script can't figure it out, it gets escalated to our actual lawyers too review. You cannot submit a package to any one of the distributions we ship without the legal review being approved. So it is clearly possible to do -- and there are all sorts of tools which figure out what SPDX license entries apply for every dependency (or vendored dependency) of a given project.
- happytoexplain 7y ago
  How have you come to that conclusion?
- gdfasfklshg4 7y ago
  Because it involves inherently manual processes.
jl-gitlab 7y ago
Interesting list, thanks for sharing. I'm the PM for CI/CD at GitLab, and I don't mean to hijack the topic, but wanted to give a heads up that we are looking at building a feature for these kinds of procedures that aren't quite pipelines, based on Jupyter Runbooks. Would love to hear your feedback: https://gitlab.com/gitlab-org/gitlab/issues/9427 https://gitlab.com/gitlab-org/gitlab/issues/9427
- vortico 7y ago
  Currently the most difficult and most time consuming step in the release checklist is finding space on my desk for three laptops so I can use them all simultaneously. Releasing only takes 1-3 hours, and I unfortunately wouldn't want to spend hours learning and setting up automated methods to perhaps shave 10 minutes off the process. Manual releasing allows me to iteratively improve the process and spot possible issues, whereas any automation would reduce build reliability.
  - judge2020 7y ago
    > whereas any automation would reduce build reliability. I guess this depends on what your release process is; for example, most npm libraries benefit from CI that runs the build script and runs `npm version $TAG && npm publish`. There's not much that could be improved here, especially if the build script is just an alias for running `webpack`.
scarejunba 7y ago
There's a method I read about on lobste.rs for this. You write down the list as you did and then turn pieces into "Do X by running Y" and over time more things get automated. That's a hefty deploy protocol, though, and automating cross-platform builds may be more trouble than it's worth if you're releasing rarely. It's pretty neat you follow a deploy checklist.
- rgoulter 7y ago
  Related to the process of "write down what you did", I'm reminded of this article about "literate sysadmin/devops" which used Emacs' org-mode as a way of combining prose, commands and command output. http://howardism.org/Technical/Emacs/literate-devops.html http://howardism.org/Technical/Emacs/literate-devops.html
- maccard 7y ago
  I read you should write it as an interactive script e.g.: input("Build X") input("Build Y") input("Zip X & Y") input("Upload release") THen run the script and press enter as you do the steps. It means if you're interrupted as your proceed, you can resume exacly where you were before. If you implement it as functions: def buildX(): input("Build X") def buildY(): input("Build Y") def ZipUp(): input("Zip X & Y") def Upload(): input("Upload release") if __name__ == '__main__': buildX() buildY() ZipUp() Upload() Then you can automate parts piecemeal.
anon9001 7y ago
I can do every step on your list with any CI platform. I do something extremely similar to build apps for ios (macos), android (linux), and web (linux). I think it's so important that it's one of the first things I do on any project. I wanted to give you an example, but it all seems so easy, I'm not sure where you're getting stuck.
- gdfasfklshg4 7y ago
  Can you really do a legal review from a CI platform?
  - wwright 7y ago
    If it can be automated, then obviously, yeah. If not fully automated, many CI platforms allow you to pause the pipeline by requiring manual confirmation from an authorized user. I know Jenkins and CircleCI support this off the top of my head. You could have the CI perform any relevant searches or diffs, then display that to the user to get manual confirmation. It’s still not “perfect”, but it does allow you to reliably get someone to look at the data and say “yes” or “no.”
    - anon9001 7y ago
      Yeah, that's one approach. Another might be to commit some kind of `./legal/sign-off-v1.0.0` so you can see who did it, and so travis can know if it should deploy the build or not
- deleted 7y ago
  [deleted]

gourlaysama 7y ago

The Scala release checklist [1] is a good example of a release checklist with both automated and manual steps. [1]: https://github.com/scala/scala-dev/issues/645 https://github.com/scala/scala-dev/issues/645