10 ms·

So You Want to Ship a Command-Line Tool for macOS (2022)

95 points by freetonik 3y ago

physicsguy 3y ago
It’s a weird time as a developer when it’s now easier to install WSL2 on an enterprise machine and use that for development than it is to use an actually Unix based OS
- notpushkin 3y ago
  Linux runs mostly fine nowadays – I'd say the experience is smoother than with Windows, even. Have you tried it recently?
  - reactordev 3y ago
    I think they were making a jab at Apple. Not saying they can’t run Unix.
    - asystole 3y ago
      <obligatory "macOS is Unix">
      - bmicraft 3y ago
        Obligatory "linux isn't unix"
  - lunfard000 3y ago
    Personally, it always run fine at the beginning until it isnt and then you have to spend the rest of the day trying to figure out what is wrong. For desktop that is it, servers can just run 24/7 without problems (mine just run containers). Plasma looks pretty good but is not nowhere as good as window's. I heard that they fixed power management but haven't tried myself. My last point is that hyper-v got soo good with GPU partitioning and can be fully managed using type-safe Powershell.
  - physicsguy 3y ago
    Last time I ran a Linux desktop, I had to reinstall my graphics drivers in a tty console every time I wanted to do a kernel upgrade, because the distro ones were ancient and that was the only way to get the latest from NVidia.
- AnonymousPlanet 3y ago
  Until you realise that WSL2 is nothing but a Linux VM with some shoddily added sugar that completely breaks down as soon as real work demands handling encodings properly, opening windows at the end of a pipe, and all those other things that I regularly want do with a Unix like system. All of this works seamlessly on Linux or macOS, but Windows is still a pathetic shitshow, WSL2 or not.
palad1n 3y ago
tldr: "don't"
modernerd 3y ago
Apple’s tools really could emit better messages when things go wrong. It’s sad that they hit a dead-end with this after everything they went through. Windows isn’t a cakewalk either (you now have to use a hardware security module to store your certs, which means USB stick based signing or using a cloud service that supports HSMs, plus certs themselves are a racket in terms of pricing).
- pansa2 3y ago
  But Windows is still happy to run downloaded executables that aren’t signed, isn’t it? OTOH it sounds like macOS requires everything to be signed.
  - viraptor 3y ago
    No, you have to do a bit of work to proceed with the installation. Windows with throw some scary warnings at you if you download something unsigned and try to run it. And if you're very unlucky and anger the SmartScreen, you may need to use power user level workarounds.
  - modernerd 3y ago
    No, Windows 11 presents two scary-looking popups for unsigned apps: - “Windows protected your PC” with some tiny blurb no-one seems to read and a single button labelled “Don’t run”. - If you do read the tiny blurb and spot the “More info” link, you can click it to get a summary about Windows Defender SmartScreen having saved the day by protecting you from this app by an “unknown publisher”, and then a second button becomes available that says, “Run anyway”. It’s pretty hostile and it effectively forces devs to throw 300-600usd/year at a cert provider plus spend time to configure signing via CI. There are also two levels of certificate, and the better one is only available if you’ve been in business for several years with an office address and public phone number (cert providers need to call you at your public number to validate your business ID). There are some ways around it (I think games distributed via Steam’s installer still don’t need to be signed) but those aren’t an option for many.
    - 3y ago
nox101 3y ago
it's not clear to me what you want to happen. It sounds like you have a command line tool that takes no arguments. you want a Mac user to be able to download a file, click that file, have it run once? and then delete itself? Not sure about this last part in any case I think there are plenty of apps that do this. Some are distributed as packages instead of apps. I'm not a Mac dev expert at all but I have used enough apps to know it's possible. Many of those apps are open source maybe look into them if you want to solve this issue. Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help? sorry I don't have concrete suggestions. I'm just confident there are solutions. I'm totally sympathetic to the fact they it's a pain in the ass and Apple, probably intentionally, doesn't document the process well or at all
- aslilac 3y ago
  I think the point is pretty clearly “Apple should do better and provide better tools”
- modernerd 3y ago
  > Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help? These only sign/notarise the dmg or app bundles (or the binaries inside them), so I don’t think that will help the OP unless they’re willing to wrap their CLI app in an Electron app, unfortunately.
- reactordev 3y ago
  This can all be achieved with a simple shell script to compile the binary on the devs machine. Xcode-select —install git clone… make
notpushkin 3y ago
Why not just ship it as a Homebrew package instead? It's jankier than any of the usual Linux packaging systems, but should get the work done. (Of course, it doesn't solve the “running from Finder” problem, but then again you don't usually run your command line tools by double clicking them in file manager, do you?)
- reactordev 3y ago
  If you read the article, you’d know.
  - kreetx 3y ago
    Do you understand the reasoning though? The explanation says the tool is responsible for "installing homebrew", but also that the developers this thing is for "refuse to install homebrew". Seems paradoxical. But if this blog post gets Apple to improve their code signing process, then it does have value :)
    - dagmx 3y ago
      The reason is that this is meant to be a script to bootstrap their startups systems to the way the poster wants. But I think they have a really odd set of requirements in that they don’t want their devs to run a CLI tool from the terminal, but want to run it from Finder?
      - reactordev 3y ago
        Yeah it’s pretty clear they have an IT problem. The requirements, the process, the solution, just wow…
    - hibbelig 3y ago
      My understanding is that the goal is as follows: 1. New team member gets new fresh Mac 2. New team member downloads the tool and runs it 3. New fresh Mac is now set up with a development environment suitable for the company, including a working Homebrew installation (or Nix or whatever)
viraptor 3y ago
Ok, so I gotta ask - why package it at all? I'd never recommend it for a public tool, but in an internal, trusted environment, why not post a "curl ... | sh" command? Or point them at the master/main.zip of the repo where that script lives in gitlab/hub/something? If this is aimed for developers, going through all of that to provide an installable and clickable thing feels a bit unnecessary... Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries. May be easier than doing it from scratch.
- madduci 3y ago
  > Ok, so I gotta ask - why package it at all? I'd never recommend it for a public tool, but in an internal, trusted environment, why not post a "curl ... | sh" command? Because I don't want to install some tools on my PC just to build _your tool_. A packaged application is less complicated for the final user to use it. In this, Windows and some Linux distribution have managed to create a good experience (MSI, AppImage, Flatpack)
  - fodkodrasz 3y ago
    You call MSI a good experience until you had to create one.
    - chii 3y ago
      The end user isn't privy to how an MSI is created - and that's all that matters. Developers suffer what they must, while the user suffers what they will.
      - fodkodrasz 3y ago
        The article is about developers suffering what they must, so I guess it matters only as much as the article we are commenting.
    - pjc50 3y ago
      MSI is one of the more insane file formats out there. MSIX/Appx is a little more sane but still quite bureaucratic to construct.
sigmoid10 3y ago
I've been through the same hell and there's basically just one way to publish software for macOS with no hassle for the user: Completely lock yourself in their ecosystem. That means working on a Mac, building in Xcode using Swift and publishing in the official app store. If you want to do anything that resembles the free software approaches of Linux, stay far away from Apple. If you really need to publish on Mac for whatever reason, it is so much better to dump the hassle on the user. Even if the core targets are not developers. And who knows, if you ruin the experience for enough users, Apple might actually bother to fix their system.
- isodev 3y ago
  And why not respect the platform you want to support? Just like you get various Linux/BSD distros and they all have a different take on packaging and distributions... you also have the way macOS does it. There is no inherent expectation that "what I do on Linux" will work on macOS... or every Linux even.
  - sigmoid10 3y ago
    That's exactly what's happening. But while most software devs still steer around Apple for these kinda of things, Windows and Linux have figured out a way to work nicely together. If you want to ship from one of them to the other, it is trivial - which is by no means expected or how it always was. And it did make everyone's lives easier.
    - isodev 3y ago
      To be exact though, Windows didn't figure out how to work together, they transplanted "a thing" (WSL) and I'm not sure that's a tool one can expect users who are not developers or IT folks to install or use. All "native" windows ways also require windowsy tooling (e.g. MSI, powershell etc...)
    - threeseed 3y ago
      I have no idea what magical thing Windows and Linux have figured out. Is it WSL2 ? Because you can distribute Mac CLI apps exactly the same way. The issue here is with the code-signing process which is optional but recommended just like on Windows.
iansinnott 3y ago
At a past job the team had a custom CLI tool to manage all sorts of dev tooling. Worked great, highly recommended. The distribution mechanism was to build from source[]. Once compiled, all other dev tasks were handled by the tool. If you compile it yourself you don't run into any of these issues. This approach is only viable for developer tools of course, but that's what the post is about. []: This was a Go project, so building from source was relatively straight forward.
pikseladam 3y ago
good article actually. apple don't want you to do it :) but why not use swift on xcode? i get that you don't want to but why is that?
saurik 3y ago
You actually can pass certificates in files on disk to Apple's codesign utility, it just involves understanding .keychain files. Also, while this only sometimes helps, it is worth looking for more error information in dmesg as that is where verification errors go.
flohofwoe 3y ago
Apparently it's about a command line tool that's not downloaded and started on the command line, but instead downloaded in a browser and started by double clicking in the Finder (apparently for non-technical users?). TBF, this also wouldn't work on Windows without an expensive EV code signing certificate (which isn't exactly trivial to use with GH Actions either: https://melatonin.dev/blog/how-to-code-sign-windows-installers-with-an-ev-cert-on-github-actions/ https://melatonin.dev/blog/how-to-code-sign-windows-installe...) ...from what I'm seeing on macOS, downloading and running a cmdline tool (built with the Xcode cmdline toolchain on GH Actions) inside the terminal still works without all the hassle described in the article - of course I wouldn't be surprised if that stopped working too one day :/ PS: I kinda wish that operating systems had builtin WASI runners for this kind of bootstrapping work.
fredsted 3y ago
My company ships a simple CLI, it's written in Node.js so it can be installed on any platform using npm. A a Docker image is also provided. Most of the target audience has either of these installed.
- Alifatisk 3y ago
  I think deno could compile js / ts scripts into a single cross-platform executable. No npm or docker needed.
isodev 3y ago
It seems strange to try and make a sandboxed tool to perform work which is not sandboxed at all... There is a class of macOS applications which can perform such functions and they're called Packages or Distributions (it's the wizard you see sometimes when an app needs to make changes to the system before you can use it). That's the way to deliver such applications from a downloadable source. Packages can be signed and notarised and none of the crazy workarounds listed in the post will be necessary.
Klonoar 3y ago
This is really specific, but for this point in the article: > There’s a long guide on Embedding a Command-Line Tool in a Sandboxed App, so I followed that, and then slowly, painfully, factored Xcode out of it, so that I wouldn’t have to figure out how to get a 10GB Xcode install onto the CI machine (remember, you need to be signed in to an Apple ID to download Xcode, and there’s no way to do it from the command-line). You could actually solve this with Rust and no Xcode whatsoever. cacao [1] and cargo-bundle [2] will produce an app bundle you could sign/notarize/distribute without needing to ever open Xcode. [1] https://github.com/ryanmcgrath/cacao https://github.com/ryanmcgrath/cacao [2] https://github.com/burtonageo/cargo-bundle https://github.com/burtonageo/cargo-bundle You wouldn't need to tangle with Swift, would only need enough Rust to hoist whatever script you're trying to run. In general though I find the article a bit odd - there are also quite a few examples of signing/notarizing/etc for GitHub Actions out there already, I went through this exercise a week ago and it wasn't that bad. Edit: Hell, you could also potentially just bundle your scripts up into a `.pkg` installer
shp0ngle 3y ago
Hm, this doesn't match my experience at all, I just use rcodesign (the third party rust tool), some plist and xar magic (that I honestly forgot) and call it a day it's sad that we need third party tool to be able to codesign and notarize effectively (and from other OSes and CIs), but it works now but maybe I miss something that the author needed and I didn't. The hardest part was cross-compiling for macOS but we did it too with these images from elasticsearch https://github.com/elastic/golang-crossbuild https://github.com/elastic/golang-crossbuild
dagmx 3y ago
Seems like part of the problem is that the author wants to both bootstrap homebrew, without users knowing the command line? In which case, they should do the same thing other platforms do: user scripts. Much like bat files on windows, macOS allows writing AppleScript which would sidestep their issues completely, or command files for shell scripts. Otherwise they don’t need to sign command line tools. macOS doesn’t block tools run in the terminal to the best of my knowledge. If they want to make an app, that also doubles as a command line tool, it’s pretty simple as well. You can check for command line options when running the main executable. I’ve done this several times. Apps aren’t magic. They can self sign that app, and their end users can bypass the unknown developer warning, or just launch the command line part. But again, just use AppleScript or command files.
- latexr 3y ago
  > Otherwise they don’t need to sign command line tools. macOS doesn’t block tools run in the terminal to the best of my knowledge. This is incorrect. You do need to sign command-line tools as otherwise they are blocked from running even in a terminal, at least on Apple Silicon.
  - dagmx 3y ago
    I believe you just need to remove the quarantine bit. I’ve definitely been distributing unsigned utility CLI tools to people on macOS 14. They’re built via CMake and so there’s no implicit signing. xattr -d com.apple.quarantine Foo The author of the post even mentions this BUT they’re operating in a very weird space where they want to make a command line tool but don’t want people to use a terminal to access it. To me it’s a nonsensical issue. I suspect the author got flustered with a set of ideals and didn’t take the necessary step back to reevaluate their situation
p0w3n3d 3y ago
Didn't we have recently a docker image here on HN that allows to build for macOS? Sadly I can't find it in a zounds of articles containing macOS and build
- actionfromafar 3y ago
  https://github.com/shepherdjerred/macos-cross-compiler https://github.com/shepherdjerred/macos-cross-compiler
ludovicianul 3y ago
As this targets developers, aren't developers supposed to be familiar with the terminal. So doing `chmod +x` is trivial. Especially since they probably use a lot of other tooling to build, package, deploy, etc.
- sssilverman 3y ago
  Seriously. You want to ask the CTO to make artifacts public because you don't want devs to set or clear an attribute bit? Talk about entirely the wrong priorities. Where did this coddled sense of entitlement come from? Are they creating their own SSH keys? Do they know the basics of the git cli? Or are these "magic commands" too? Do they know how to do their job? I sympathise that Apple's tooling could be better, but the conclusion and tone is ridiculous given what could charitably be described as 'unique' requirements. Our internal CLI tools have all largely been hassle free regardless of devs OS. We do use GoReleaser which abstracts away the multi arch builds, signing, and publishing to GitHub and Homebrew, which can be criticised sometimes as a bit of a crutch. But it's also been zero hassle for all of our Mac devs (the most junior of which are still expected to be able to download and execute a binary from a terminal.) and lets us get on with our day without prioritising 'requirements' that aren't even worth spending any time on. Why would you even ship a command-line tool if your apparent need is to avoid the command line?
gigatexal 3y ago
I’ve not used macOS in forever but I never had issues with cli binaries installed with Brew or Nix or just manually added to some path. Are they forcing all binaries to be signed now?
- threeseed 3y ago
  Nothing has changed. For some reason they want to distribute this command line script as a double-click installer which doesn't need to be signed either. Users will simply need to override the "this is untrusted and you shouldn't open this" dialog.
  - angulardragon03 3y ago
    Agreed. It doesn’t seem like there is anything you’d want to do in a binary that couldn’t be done by creating an installer .pkg and just including all the logic in the post install script. The user gets a double-click “installer”, and like you said, you don’t even need to sign it.
hibbelig 3y ago
I don't quite understand what the problem is with asking the developers to download the tool using curl. The developers could add their username and password to the command line...
deleted 3y ago
[deleted]
saagarjha 3y ago
> We also need to notarize the .app and the .dmg. Interestingly, you can only notarize .pkg, .dmg, and .app files (in .zips) — command-line tools can only be notarized if they’re embedded in one of the listed containers. I don't think this is true, actually. I've been notarizing command-line builds of unxip by shoving it in a zip and uploading it to notarytool for a while: https://github.com/saagarjha/unxip/blob/main/release.sh https://github.com/saagarjha/unxip/blob/main/release.sh. (You will note, amusingly, that I am suffering from the same -parse-as-library bug as the author.) After this is done I just chuck the zip file and upload the binary directly to GitHub. If you download the file from your browser and chmod +x it, you can double click on it and it runs. Obviously you can skip the chmod +x step by sending people the zip (which preserves permissions inside of itself). Also, FWIW, Apple's tooling sucks but if you check Console while it evaluates the code sometimes it will tell you why it doesn't like it. Most of the evaluation code is open source, too, so you can look up log messages and errors there: https://github.com/apple-oss-distributions/Security https://github.com/apple-oss-distributions/Security
- lapcat 3y ago
  Yes, you can notarize command-line tools. The only downside is that you can't staple the notarization ticket to the tool, so it has to be downloaded by syspolicyd on first run. In most cases that isn't an issue, though it won't work if your internet connection is down or if you're blocking syspolicyd to keep if from phoning home to Cupertino. ;-) Incidentally, the other day I discovered that the Arc web browser neglected to staple the notarization ticket to their app.
  - saagarjha 3y ago
    Yeah, this bit is quite unfortunate :(
happymellon 3y ago
My way around this in the past was to accept that curl was going to have to be called. Make a pubic repo that is just a facade, generates an ssh key, and calls GitHub (or whatever) to push the ssh key. curl -H "Authorization: token **********" --data '{"title":"test-key","key":"${key[...]}"}' https://api.github.com/user/keys https://api.github.com/user/keys On success you can then call the private repo and execute the cli installer. If the bootstrap is piped then it doesn't ever have to be installed, and you don't have it junking up your system. Extra points to pull down existing registed public keys and check if they are already there. https://coderwall.com/p/tpngvw/curl-your-github-public-keys https://coderwall.com/p/tpngvw/curl-your-github-public-keys
timvdalen 3y ago
> I’m seriously considering asking the CTO if we can make the repository public solely so we can use curl as a distribution mechanism. I think you should be able to use the new-style scoped GitHub access tokens to generate a token that'll allow access to the one resource.
- rjmunro 3y ago
  Just publish the script to a secret URL like: https://example.com/aeluaeualeraciuwefaljwefnalsduhgsdgasu.sh https://example.com/aeluaeualeraciuwefaljwefnalsduhgsdgasu.s... Rotate the URL from time to time.
iainmerrick 3y ago
There's a hidden assumption in the headline here: shipping a command-line tool to non-developers who don't use the command line is hard. If you want to make stuff double-clickable in the Finder you need to jump through a lot of hoops, yes. Shipping a command-line tool for developers is easy. Put it on Homebrew or NPM or something, or just give them a URL for curl. macOS is Unix, it's just that the Unix bits are carefully hidden away from most users.
- marxisttemp 3y ago
  > Homebrew Or MacPorts, which was created by Apple employee and inventor of the FreeBSD ports system Jordan Hubbard, has more packages than Homebrew, and has a much more correct design. Homebrew is a bit of a joke in comparison, not sure why it gets all the attention.
  - Alifatisk 3y ago
    Really? MacPorts has more packages than homebrew?
    - marxisttemp 3y ago
      Indeed, MacPorts has over 36,000. Homebrew doesn’t list an official number and I’m not at my computer to check myself, but the count was under 5,000 as of 2019: https://apple.stackexchange.com/questions/324084/how-can-i-determine-the-current-brew-package-count https://apple.stackexchange.com/questions/324084/how-can-i-d...
  - iainmerrick 3y ago
    I don't know either! I've tried MacPorts in the past but for some reason I always ended up on Homebrew again -- I can't recall why. I should probably give it another try. One possible minor advantage of Homebrew is that it works on both macOS and Linux. I recently found this very handy for setting up a project with developers on Macs and CI on Linux. Almost no OS-specific build steps were needed. A couple more points, from a quick scan of the respective docs: "Homebrew won’t install files outside its prefix and you can place a Homebrew installation wherever you like." The MacPorts docs are much less clear, I can't tell if it can do that. Homebrew gives "brew install --cask firefox" as an example of installing a GUI app. MacPorts doesn't seem to have Firefox. I'm curious what makes MacPorts "much more correct", if anyone wants to clarify that.
thrdbndndn 3y ago
> [workaround 2] Download the file with curl. Unix command-line tools like curl and tar don’t add the quarantine bit to the files they create on macOS, so it’s pretty easy to run software downloaded with them. Wait I'm confused by this part (never used MacOS). Which part of the chain mechanism mentioned above can be bypassed by just downloading the file via CURL? I didn't seem to see the author mentioned (not explicitly at least) which steps added these "quarantine bits".
- tener 3y ago
  Browsers tag downloaded files using filesystem features.
  - thrdbndndn 3y ago
    Do untagged files (like downloaded via curl) not require signing at all, or it's just lenient enough to not trigger the bugs/issues mentioned in the article (since their app is actually signed, after all)?
bionhoward 3y ago
One idea, if the customers tend to be NixOS enjoyers, would be to put the tool in Nix Packages and then use nix-shell for them to access it. Perhaps the nix packages configuration could reference a script with everything properly debugged and then the snake can eat its own tail. It seems like a provisioner to install and configure stuff, in which case one obvious method to install is copy pasting bash code. Have we really reached an era where folks expect to use CLI tools without using a CLI? Can nixpkgs install from brew? Maybe that’s the answer, use the annoying stuff and wrap it with something to make it less annoying. Or, you could wrap your CLI with Tauri and install with Cargo. Might be more robust but not in the comfort zone of a nix shop.
- beccaooo 3y ago
  You'll get a kick out of knowing the tool I was attempting to codesign is the tool that performs the initial install and configuration of Nix (and Homebrew) for our engineers :)
moondev 3y ago
"go run" is pretty awesome. go run github com/mikefarah/yq/v3@latest Assuming go is installed this works on all common platforms and architectures. I wonder if a stub program that packages go with something like cosmopolitan could make it a true universal cli
webworker 3y ago
> Do not attempt to “create software” for macOS. They don’t want you to. If you want to run your own programs, install Linux and suffer like you’re supposed to. Yeah, I'm starting to understand that more and more. I'm not a big fan of the PWA SPA-approach, but given what's going on I'm going to have to change my tune. Selecting the "Add to Dock..." option in Safari for YouTube Music has been kind of eye-opening. It works really well. I'm not sure what they're thinking here. Are native desktop apps done and over? It kind of seems like that's what they want based on behavior.

95 points by freetonik 3y ago

It’s a weird time as a developer when it’s now easier to install WSL2 on an enterprise machine and use that for development than it is to use an actually Unix based OS

notpushkin 3y ago
Linux runs mostly fine nowadays – I'd say the experience is smoother than with Windows, even. Have you tried it recently?
- reactordev 3y ago
  I think they were making a jab at Apple. Not saying they can’t run Unix.
  - asystole 3y ago
    <obligatory "macOS is Unix">
    - bmicraft 3y ago
      Obligatory "linux isn't unix"
- lunfard000 3y ago
  Personally, it always run fine at the beginning until it isnt and then you have to spend the rest of the day trying to figure out what is wrong. For desktop that is it, servers can just run 24/7 without problems (mine just run containers). Plasma looks pretty good but is not nowhere as good as window's. I heard that they fixed power management but haven't tried myself. My last point is that hyper-v got soo good with GPU partitioning and can be fully managed using type-safe Powershell.
- physicsguy 3y ago
  Last time I ran a Linux desktop, I had to reinstall my graphics drivers in a tty console every time I wanted to do a kernel upgrade, because the distro ones were ancient and that was the only way to get the latest from NVidia.
AnonymousPlanet 3y ago
Until you realise that WSL2 is nothing but a Linux VM with some shoddily added sugar that completely breaks down as soon as real work demands handling encodings properly, opening windows at the end of a pipe, and all those other things that I regularly want do with a Unix like system. All of this works seamlessly on Linux or macOS, but Windows is still a pathetic shitshow, WSL2 or not.

palad1n 3y ago

tldr: "don't"

modernerd 3y ago

Apple’s tools really could emit better messages when things go wrong. It’s sad that they hit a dead-end with this after everything they went through. Windows isn’t a cakewalk either (you now have to use a hardware security module to store your certs, which means USB stick based signing or using a cloud service that supports HSMs, plus certs themselves are a racket in terms of pricing).

pansa2 3y ago
But Windows is still happy to run downloaded executables that aren’t signed, isn’t it? OTOH it sounds like macOS requires everything to be signed.
- viraptor 3y ago
  No, you have to do a bit of work to proceed with the installation. Windows with throw some scary warnings at you if you download something unsigned and try to run it. And if you're very unlucky and anger the SmartScreen, you may need to use power user level workarounds.
- modernerd 3y ago
  No, Windows 11 presents two scary-looking popups for unsigned apps: - “Windows protected your PC” with some tiny blurb no-one seems to read and a single button labelled “Don’t run”. - If you do read the tiny blurb and spot the “More info” link, you can click it to get a summary about Windows Defender SmartScreen having saved the day by protecting you from this app by an “unknown publisher”, and then a second button becomes available that says, “Run anyway”. It’s pretty hostile and it effectively forces devs to throw 300-600usd/year at a cert provider plus spend time to configure signing via CI. There are also two levels of certificate, and the better one is only available if you’ve been in business for several years with an office address and public phone number (cert providers need to call you at your public number to validate your business ID). There are some ways around it (I think games distributed via Steam’s installer still don’t need to be signed) but those aren’t an option for many.
  - 3y ago

nox101 3y ago

it's not clear to me what you want to happen. It sounds like you have a command line tool that takes no arguments. you want a Mac user to be able to download a file, click that file, have it run once? and then delete itself? Not sure about this last part in any case I think there are plenty of apps that do this. Some are distributed as packages instead of apps. I'm not a Mac dev expert at all but I have used enough apps to know it's possible. Many of those apps are open source maybe look into them if you want to solve this issue. Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help? sorry I don't have concrete suggestions. I'm just confident there are solutions. I'm totally sympathetic to the fact they it's a pain in the ass and Apple, probably intentionally, doesn't document the process well or at all

aslilac 3y ago
I think the point is pretty clearly “Apple should do better and provide better tools”
modernerd 3y ago
> Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help? These only sign/notarise the dmg or app bundles (or the binaries inside them), so I don’t think that will help the OP unless they’re willing to wrap their CLI app in an Electron app, unfortunately.
reactordev 3y ago
This can all be achieved with a simple shell script to compile the binary on the devs machine. Xcode-select —install git clone… make

notpushkin 3y ago

Why not just ship it as a Homebrew package instead? It's jankier than any of the usual Linux packaging systems, but should get the work done. (Of course, it doesn't solve the “running from Finder” problem, but then again you don't usually run your command line tools by double clicking them in file manager, do you?)

reactordev 3y ago
If you read the article, you’d know.
- kreetx 3y ago
  Do you understand the reasoning though? The explanation says the tool is responsible for "installing homebrew", but also that the developers this thing is for "refuse to install homebrew". Seems paradoxical. But if this blog post gets Apple to improve their code signing process, then it does have value :)
  - dagmx 3y ago
    The reason is that this is meant to be a script to bootstrap their startups systems to the way the poster wants. But I think they have a really odd set of requirements in that they don’t want their devs to run a CLI tool from the terminal, but want to run it from Finder?
    - reactordev 3y ago
      Yeah it’s pretty clear they have an IT problem. The requirements, the process, the solution, just wow…
  - hibbelig 3y ago
    My understanding is that the goal is as follows: 1. New team member gets new fresh Mac 2. New team member downloads the tool and runs it 3. New fresh Mac is now set up with a development environment suitable for the company, including a working Homebrew installation (or Nix or whatever)

viraptor 3y ago

Ok, so I gotta ask - why package it at all? I'd never recommend it for a public tool, but in an internal, trusted environment, why not post a "curl ... | sh" command? Or point them at the master/main.zip of the repo where that script lives in gitlab/hub/something? If this is aimed for developers, going through all of that to provide an installable and clickable thing feels a bit unnecessary... Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries. May be easier than doing it from scratch.

madduci 3y ago
> Ok, so I gotta ask - why package it at all? I'd never recommend it for a public tool, but in an internal, trusted environment, why not post a "curl ... | sh" command? Because I don't want to install some tools on my PC just to build _your tool_. A packaged application is less complicated for the final user to use it. In this, Windows and some Linux distribution have managed to create a good experience (MSI, AppImage, Flatpack)
- fodkodrasz 3y ago
  You call MSI a good experience until you had to create one.
  - chii 3y ago
    The end user isn't privy to how an MSI is created - and that's all that matters. Developers suffer what they must, while the user suffers what they will.
    - fodkodrasz 3y ago
      The article is about developers suffering what they must, so I guess it matters only as much as the article we are commenting.
  - pjc50 3y ago
    MSI is one of the more insane file formats out there. MSIX/Appx is a little more sane but still quite bureaucratic to construct.

sigmoid10 3y ago

I've been through the same hell and there's basically just one way to publish software for macOS with no hassle for the user: Completely lock yourself in their ecosystem. That means working on a Mac, building in Xcode using Swift and publishing in the official app store. If you want to do anything that resembles the free software approaches of Linux, stay far away from Apple. If you really need to publish on Mac for whatever reason, it is so much better to dump the hassle on the user. Even if the core targets are not developers. And who knows, if you ruin the experience for enough users, Apple might actually bother to fix their system.

isodev 3y ago
And why not respect the platform you want to support? Just like you get various Linux/BSD distros and they all have a different take on packaging and distributions... you also have the way macOS does it. There is no inherent expectation that "what I do on Linux" will work on macOS... or every Linux even.
- sigmoid10 3y ago
  That's exactly what's happening. But while most software devs still steer around Apple for these kinda of things, Windows and Linux have figured out a way to work nicely together. If you want to ship from one of them to the other, it is trivial - which is by no means expected or how it always was. And it did make everyone's lives easier.
  - isodev 3y ago
    To be exact though, Windows didn't figure out how to work together, they transplanted "a thing" (WSL) and I'm not sure that's a tool one can expect users who are not developers or IT folks to install or use. All "native" windows ways also require windowsy tooling (e.g. MSI, powershell etc...)
  - threeseed 3y ago
    I have no idea what magical thing Windows and Linux have figured out. Is it WSL2 ? Because you can distribute Mac CLI apps exactly the same way. The issue here is with the code-signing process which is optional but recommended just like on Windows.

iansinnott 3y ago

At a past job the team had a custom CLI tool to manage all sorts of dev tooling. Worked great, highly recommended. The distribution mechanism was to build from source[]. Once compiled, all other dev tasks were handled by the tool. If you compile it yourself you don't run into any of these issues. This approach is only viable for developer tools of course, but that's what the post is about. []: This was a Go project, so building from source was relatively straight forward.

pikseladam 3y ago

good article actually. apple don't want you to do it :) but why not use swift on xcode? i get that you don't want to but why is that?

saurik 3y ago

You actually can pass certificates in files on disk to Apple's codesign utility, it just involves understanding .keychain files. Also, while this only sometimes helps, it is worth looking for more error information in dmesg as that is where verification errors go.

flohofwoe 3y ago

Apparently it's about a command line tool that's not downloaded and started on the command line, but instead downloaded in a browser and started by double clicking in the Finder (apparently for non-technical users?). TBF, this also wouldn't work on Windows without an expensive EV code signing certificate (which isn't exactly trivial to use with GH Actions either: https://melatonin.dev/blog/how-to-code-sign-windows-installers-with-an-ev-cert-on-github-actions/ https://melatonin.dev/blog/how-to-code-sign-windows-installe...) ...from what I'm seeing on macOS, downloading and running a cmdline tool (built with the Xcode cmdline toolchain on GH Actions) inside the terminal still works without all the hassle described in the article - of course I wouldn't be surprised if that stopped working too one day :/ PS: I kinda wish that operating systems had builtin WASI runners for this kind of bootstrapping work.

fredsted 3y ago

My company ships a simple CLI, it's written in Node.js so it can be installed on any platform using npm. A a Docker image is also provided. Most of the target audience has either of these installed.

Alifatisk 3y ago
I think deno could compile js / ts scripts into a single cross-platform executable. No npm or docker needed.

isodev 3y ago

It seems strange to try and make a sandboxed tool to perform work which is not sandboxed at all... There is a class of macOS applications which can perform such functions and they're called Packages or Distributions (it's the wizard you see sometimes when an app needs to make changes to the system before you can use it). That's the way to deliver such applications from a downloadable source. Packages can be signed and notarised and none of the crazy workarounds listed in the post will be necessary.

Klonoar 3y ago

This is really specific, but for this point in the article: > There’s a long guide on Embedding a Command-Line Tool in a Sandboxed App, so I followed that, and then slowly, painfully, factored Xcode out of it, so that I wouldn’t have to figure out how to get a 10GB Xcode install onto the CI machine (remember, you need to be signed in to an Apple ID to download Xcode, and there’s no way to do it from the command-line). You could actually solve this with Rust and no Xcode whatsoever. cacao [1] and cargo-bundle [2] will produce an app bundle you could sign/notarize/distribute without needing to ever open Xcode. [1] https://github.com/ryanmcgrath/cacao https://github.com/ryanmcgrath/cacao [2] https://github.com/burtonageo/cargo-bundle https://github.com/burtonageo/cargo-bundle You wouldn't need to tangle with Swift, would only need enough Rust to hoist whatever script you're trying to run. In general though I find the article a bit odd - there are also quite a few examples of signing/notarizing/etc for GitHub Actions out there already, I went through this exercise a week ago and it wasn't that bad. Edit: Hell, you could also potentially just bundle your scripts up into a `.pkg` installer

shp0ngle 3y ago

Hm, this doesn't match my experience at all, I just use rcodesign (the third party rust tool), some plist and xar magic (that I honestly forgot) and call it a day it's sad that we need third party tool to be able to codesign and notarize effectively (and from other OSes and CIs), but it works now but maybe I miss something that the author needed and I didn't. The hardest part was cross-compiling for macOS but we did it too with these images from elasticsearch https://github.com/elastic/golang-crossbuild https://github.com/elastic/golang-crossbuild

dagmx 3y ago

Seems like part of the problem is that the author wants to both bootstrap homebrew, without users knowing the command line? In which case, they should do the same thing other platforms do: user scripts. Much like bat files on windows, macOS allows writing AppleScript which would sidestep their issues completely, or command files for shell scripts. Otherwise they don’t need to sign command line tools. macOS doesn’t block tools run in the terminal to the best of my knowledge. If they want to make an app, that also doubles as a command line tool, it’s pretty simple as well. You can check for command line options when running the main executable. I’ve done this several times. Apps aren’t magic. They can self sign that app, and their end users can bypass the unknown developer warning, or just launch the command line part. But again, just use AppleScript or command files.

latexr 3y ago
> Otherwise they don’t need to sign command line tools. macOS doesn’t block tools run in the terminal to the best of my knowledge. This is incorrect. You do need to sign command-line tools as otherwise they are blocked from running even in a terminal, at least on Apple Silicon.
- dagmx 3y ago
  I believe you just need to remove the quarantine bit. I’ve definitely been distributing unsigned utility CLI tools to people on macOS 14. They’re built via CMake and so there’s no implicit signing. xattr -d com.apple.quarantine Foo The author of the post even mentions this BUT they’re operating in a very weird space where they want to make a command line tool but don’t want people to use a terminal to access it. To me it’s a nonsensical issue. I suspect the author got flustered with a set of ideals and didn’t take the necessary step back to reevaluate their situation

p0w3n3d 3y ago

Didn't we have recently a docker image here on HN that allows to build for macOS? Sadly I can't find it in a zounds of articles containing macOS and build

actionfromafar 3y ago
https://github.com/shepherdjerred/macos-cross-compiler https://github.com/shepherdjerred/macos-cross-compiler

ludovicianul 3y ago

As this targets developers, aren't developers supposed to be familiar with the terminal. So doing `chmod +x` is trivial. Especially since they probably use a lot of other tooling to build, package, deploy, etc.

sssilverman 3y ago
Seriously. You want to ask the CTO to make artifacts public because you don't want devs to set or clear an attribute bit? Talk about entirely the wrong priorities. Where did this coddled sense of entitlement come from? Are they creating their own SSH keys? Do they know the basics of the git cli? Or are these "magic commands" too? Do they know how to do their job? I sympathise that Apple's tooling could be better, but the conclusion and tone is ridiculous given what could charitably be described as 'unique' requirements. Our internal CLI tools have all largely been hassle free regardless of devs OS. We do use GoReleaser which abstracts away the multi arch builds, signing, and publishing to GitHub and Homebrew, which can be criticised sometimes as a bit of a crutch. But it's also been zero hassle for all of our Mac devs (the most junior of which are still expected to be able to download and execute a binary from a terminal.) and lets us get on with our day without prioritising 'requirements' that aren't even worth spending any time on. Why would you even ship a command-line tool if your apparent need is to avoid the command line?

gigatexal 3y ago

I’ve not used macOS in forever but I never had issues with cli binaries installed with Brew or Nix or just manually added to some path. Are they forcing all binaries to be signed now?

threeseed 3y ago
Nothing has changed. For some reason they want to distribute this command line script as a double-click installer which doesn't need to be signed either. Users will simply need to override the "this is untrusted and you shouldn't open this" dialog.
- angulardragon03 3y ago
  Agreed. It doesn’t seem like there is anything you’d want to do in a binary that couldn’t be done by creating an installer .pkg and just including all the logic in the post install script. The user gets a double-click “installer”, and like you said, you don’t even need to sign it.

hibbelig 3y ago

I don't quite understand what the problem is with asking the developers to download the tool using curl. The developers could add their username and password to the command line...

deleted 3y ago

[deleted]

saagarjha 3y ago

> We also need to notarize the .app and the .dmg. Interestingly, you can only notarize .pkg, .dmg, and .app files (in .zips) — command-line tools can only be notarized if they’re embedded in one of the listed containers. I don't think this is true, actually. I've been notarizing command-line builds of unxip by shoving it in a zip and uploading it to notarytool for a while: https://github.com/saagarjha/unxip/blob/main/release.sh https://github.com/saagarjha/unxip/blob/main/release.sh. (You will note, amusingly, that I am suffering from the same -parse-as-library bug as the author.) After this is done I just chuck the zip file and upload the binary directly to GitHub. If you download the file from your browser and chmod +x it, you can double click on it and it runs. Obviously you can skip the chmod +x step by sending people the zip (which preserves permissions inside of itself). Also, FWIW, Apple's tooling sucks but if you check Console while it evaluates the code sometimes it will tell you why it doesn't like it. Most of the evaluation code is open source, too, so you can look up log messages and errors there: https://github.com/apple-oss-distributions/Security https://github.com/apple-oss-distributions/Security

lapcat 3y ago
Yes, you can notarize command-line tools. The only downside is that you can't staple the notarization ticket to the tool, so it has to be downloaded by syspolicyd on first run. In most cases that isn't an issue, though it won't work if your internet connection is down or if you're blocking syspolicyd to keep if from phoning home to Cupertino. ;-) Incidentally, the other day I discovered that the Arc web browser neglected to staple the notarization ticket to their app.
- saagarjha 3y ago
  Yeah, this bit is quite unfortunate :(

happymellon 3y ago

My way around this in the past was to accept that curl was going to have to be called. Make a pubic repo that is just a facade, generates an ssh key, and calls GitHub (or whatever) to push the ssh key. curl -H "Authorization: token **********" --data '{"title":"test-key","key":"${key[...]}"}' https://api.github.com/user/keys https://api.github.com/user/keys On success you can then call the private repo and execute the cli installer. If the bootstrap is piped then it doesn't ever have to be installed, and you don't have it junking up your system. Extra points to pull down existing registed public keys and check if they are already there. https://coderwall.com/p/tpngvw/curl-your-github-public-keys https://coderwall.com/p/tpngvw/curl-your-github-public-keys

timvdalen 3y ago

> I’m seriously considering asking the CTO if we can make the repository public solely so we can use curl as a distribution mechanism. I think you should be able to use the new-style scoped GitHub access tokens to generate a token that'll allow access to the one resource.

rjmunro 3y ago
Just publish the script to a secret URL like: https://example.com/aeluaeualeraciuwefaljwefnalsduhgsdgasu.sh https://example.com/aeluaeualeraciuwefaljwefnalsduhgsdgasu.s... Rotate the URL from time to time.

iainmerrick 3y ago

There's a hidden assumption in the headline here: shipping a command-line tool to non-developers who don't use the command line is hard. If you want to make stuff double-clickable in the Finder you need to jump through a lot of hoops, yes. Shipping a command-line tool for developers is easy. Put it on Homebrew or NPM or something, or just give them a URL for curl. macOS is Unix, it's just that the Unix bits are carefully hidden away from most users.

marxisttemp 3y ago
> Homebrew Or MacPorts, which was created by Apple employee and inventor of the FreeBSD ports system Jordan Hubbard, has more packages than Homebrew, and has a much more correct design. Homebrew is a bit of a joke in comparison, not sure why it gets all the attention.
- Alifatisk 3y ago
  Really? MacPorts has more packages than homebrew?
  - marxisttemp 3y ago
    Indeed, MacPorts has over 36,000. Homebrew doesn’t list an official number and I’m not at my computer to check myself, but the count was under 5,000 as of 2019: https://apple.stackexchange.com/questions/324084/how-can-i-determine-the-current-brew-package-count https://apple.stackexchange.com/questions/324084/how-can-i-d...
- iainmerrick 3y ago
  I don't know either! I've tried MacPorts in the past but for some reason I always ended up on Homebrew again -- I can't recall why. I should probably give it another try. One possible minor advantage of Homebrew is that it works on both macOS and Linux. I recently found this very handy for setting up a project with developers on Macs and CI on Linux. Almost no OS-specific build steps were needed. A couple more points, from a quick scan of the respective docs: "Homebrew won’t install files outside its prefix and you can place a Homebrew installation wherever you like." The MacPorts docs are much less clear, I can't tell if it can do that. Homebrew gives "brew install --cask firefox" as an example of installing a GUI app. MacPorts doesn't seem to have Firefox. I'm curious what makes MacPorts "much more correct", if anyone wants to clarify that.

thrdbndndn 3y ago

> [workaround 2] Download the file with curl. Unix command-line tools like curl and tar don’t add the quarantine bit to the files they create on macOS, so it’s pretty easy to run software downloaded with them. Wait I'm confused by this part (never used MacOS). Which part of the chain mechanism mentioned above can be bypassed by just downloading the file via CURL? I didn't seem to see the author mentioned (not explicitly at least) which steps added these "quarantine bits".

tener 3y ago
Browsers tag downloaded files using filesystem features.
- thrdbndndn 3y ago
  Do untagged files (like downloaded via curl) not require signing at all, or it's just lenient enough to not trigger the bugs/issues mentioned in the article (since their app is actually signed, after all)?

bionhoward 3y ago

One idea, if the customers tend to be NixOS enjoyers, would be to put the tool in Nix Packages and then use nix-shell for them to access it. Perhaps the nix packages configuration could reference a script with everything properly debugged and then the snake can eat its own tail. It seems like a provisioner to install and configure stuff, in which case one obvious method to install is copy pasting bash code. Have we really reached an era where folks expect to use CLI tools without using a CLI? Can nixpkgs install from brew? Maybe that’s the answer, use the annoying stuff and wrap it with something to make it less annoying. Or, you could wrap your CLI with Tauri and install with Cargo. Might be more robust but not in the comfort zone of a nix shop.

beccaooo 3y ago
You'll get a kick out of knowing the tool I was attempting to codesign is the tool that performs the initial install and configuration of Nix (and Homebrew) for our engineers :)

moondev 3y ago

"go run" is pretty awesome. go run github com/mikefarah/yq/v3@latest Assuming go is installed this works on all common platforms and architectures. I wonder if a stub program that packages go with something like cosmopolitan could make it a true universal cli

webworker 3y ago

> Do not attempt to “create software” for macOS. They don’t want you to. If you want to run your own programs, install Linux and suffer like you’re supposed to. Yeah, I'm starting to understand that more and more. I'm not a big fan of the PWA SPA-approach, but given what's going on I'm going to have to change my tune. Selecting the "Add to Dock..." option in Safari for YouTube Music has been kind of eye-opening. It works really well. I'm not sure what they're thinking here. Are native desktop apps done and over? It kind of seems like that's what they want based on behavior.