4 ms·

Updating Desktop Rust

65 points by piker 1y ago

saagarjha 1y ago
> On POSIX systems, in theory, we can just update the files in place. Except for dynamically loaded assets, this should work just fine. The OS treats file locking as advisory and open file descriptors are irrelevant, so even if the file is running you overwrite it in place and the user will get the freshest version on the next launch. If you do this on macOS, and your code is codesigned, you are setting yourself for pain. Instead you should replace the file entirely so the file gets a new vnode.
- piker 1y ago
  That's super interesting. Tritium is signed and notarized (as is required) and uses the clean second-binary approach on all platforms, so I never ran into this. One thing that did come up was the requirement to use `ditto` rather than a standard unzipping utility to preserve some macOS-sensitive metadata. [EDIT: the post has been updated for this point. Thanks so much. https://tritium.legal/blog/update#3 https://tritium.legal/blog/update#3.]
  - saagarjha 1y ago
    FWIW https://developer.apple.com/documentation/security/updating-mac-software https://developer.apple.com/documentation/security/updating-... is the relevant documentation for this if you want more details.
- silon42 1y ago
  Not just MacOS, on Linux this is the correct strategy too. It covers more cases: either a new instance is starting, or it hasn't fully loaded the file yet.
- formerly_proven 1y ago
  Binaries are memory-mapped, there is no guarantee for code to be resident. Modifying a running binary hence can cause a mixture of code to be paged into a running application. Which is why unix traditionally has a special "non-advisory" file locking for mapped binaries, which tries to give you an ETXTBSY error. This is nowhere near fool-proof though, e.g. Linux stopped doing this for mapped .so files, it probably doesn't work on NFS etc.
- fulafel 1y ago
  Traditionally if you update the contents of a running binary in-place in Unix you're liable to get a SIGBUS. The way to do it is unlink the existing binary and create a new one with the same name. Which is eg what install(1) does.
  - amluto 1y ago
    SIGBUS or SIGSEGV if you’re lucky. It’s really an outrageous memory safety violation and could do anything, even allow RCE (in a narrow window).
  - kelnos 1y ago
    That isn't robust to failures, though. What you sould be doing is first writing the new binary using a temporary file name, and then you can rename the new file to the correct binary name, which is an atomic operation.
IshKebab 1y ago
If you want to do this really well I'd do it like A/B updates on Android work. 1. Have two directories for the app: A and B. One contains the "current" version and one contains (possibly) the "next" version. 2. Have the actual app you start be a separate launcher program that just picks which version to launch. 3. While the app is running, periodically check for updates. If an update is available download and unzip it to the A or B directory that isn't currently in use. 4. Next time the launcher program starts it can say "an updated version is already installed, would you like to use it" (or you can just do that automatically if you - or your user - decide that's the best option). If so it marks the appropriate A/B directory as "current" and launches that one. Zero delay for users, works on all OSes, works with "always use the latest version" as well as "ask the user to update". I've never tried this tbf - just an idea.
- whytevuhuni 1y ago
  One caveat (which most apps already do anyway) is to ensure apps have at most one open instance (i.e. switch focus to the existing one), otherwise this might block both versions and result in some really confusing errors for the user, or corrupt a running app’s state if it doesn’t expect its resources to change.
lifthrasiir 1y ago
For Windows, there is an alternative approach: simply rename the running application to a different name (e.g. app.exe to app-old-timestamp.exe). This frees up the file name which can be overwritten to any other executable. The only problem I'm aware of is that this process may leave app.exe non-existent for a moment [1], but in return a need for keeping two executables vanishes. [1] Technically speaking I believe this can be solved with transactional NTFS (deprecated but still available as of Windows 11, AFAIK).
- CaptainOfCoit 1y ago
  Can you rename applications that are currently running? Last I recall, trying to rename a binary you're currently running would lead to some weird "cannot access" errors or similar. But I mostly use Linux, maybe I misunderstood it. Last time I came across it, I worked around it by having a "launcher" that wrote to a temporary path, closed old process, moved and started again. Slight indirection, but seemed to always work and be robust enough, compared to what we initially tried.
  - lifthrasiir 1y ago
    Surprisingly, yes. Go does this for example.
  - diath 1y ago
    Yes, I was recently working for an updater for a game and that's essentially how I solved it. The updater renames the current .dll and .exe files to .dll.old and .exe.old, then downloads the new .dll and .exe files, then on next launch, the application looks for .dll.old and .exe.old files in the current working directory and deletes them.
    - r_hanz 1y ago
      This is exactly how I've achieved this on Windows in the past. However, since writing that solution, I have come to understand that utilizing AppDomains (for .Net Framework) and AssemblyLoadContexts (for .Net Core) to load/unload the binaries seems to be the intended workflow.
mrasong 1y ago
Love that they prioritized both privacy (no sketchy background daemons) and security (launch-time checks) for legal apps—finally a team that gets how critical "no surprises" is for tools handling sensitive docs.
goodpoint 1y ago
> An integrated drafting environment needs to be trusted with reading, editing and redlining confidential and trade secret documents. And yet it's running on windows?!
- CaptainOfCoit 1y ago
  Like it or not, confidential and trade secret documents pass through and are edited on Windows machines, among others.
  - goodpoint 1y ago
    Not my point. TFA is writing about the importance of security and privacy, not me.
    - CaptainOfCoit 1y ago
      You seem to be missing the point. Your complaint is that it runs on Windows, which fails to take into account the target audience of this (lawyers) who most likely are running Windows. So TFA is about the "importance of security and privacy" even on Windows, since the application is only available for Windows.
ginko 1y ago
At least on linux programs shouldn't update themselves. This is the package manager's job.
- vbezhenar 1y ago
  Not everyone prefers to use package manager. I think that package manager is weird concept and should not exist.
  - CaptainOfCoit 1y ago
    Do you feel the same way about app stores too, like AppStore or Windows Store? What is your preferred way of downloading/installation applications and keeping them up to date? Personally I love being able to run one command and update everything in one swoop when I have time for it, instead of individually updating things all the time.
    - vbezhenar 1y ago
      > Do you feel the same way about app stores too, like AppStore or Windows Store? AppStores are good enough, as long as they're optional file hosting and censorship service and do not change actual application. So basically I want no middle-man between developer build and my computer. If developer uploads their application to AppStore and the app gets downloaded to my computer unchanged, that's fine. If some unrelated person pulls sources from github, alters them, builds them and ships them to my computer, that's what I don't like. > What is your preferred way of downloading/installation applications and keeping them up to date? Best way is statically linked binary available for direct download (many Golang and Rust apps are distributed this way directly from github). I check for updates when I feel like it, which might not be the best way, so some automatic update reminders are welcome, but by no means are necessary. Second best way is targzipped directory which I unpack into something like ~/apps/idea and then it keeps itself up to date automatically. Another thing that I don't like about package managers is that they spill application files all over filesystem. That's also wrong. Files which belong to application must be kept in one place. Ideally separated by chroot-like mechanisms.
  - 1y ago
tyilo 1y ago
I use the `self_replace` crate for this: https://docs.rs/self-replace/latest/self_replace/ https://docs.rs/self-replace/latest/self_replace/
eviks 1y ago
If only desktop OS had decades to properly sort the updates out so that apps wouldn't have to reinvent the wheel...
- piker 1y ago
  The problem is they opted for lock-in via walled-off app stores given the opportunity.
amluto 1y ago
I’m surprised there’s no attempt to use an A/B or multi version approach. It’s only a partial solution, but the main binary could be a very minimal launcher that enumerates installed versions and decides which version to run. This results in no window at all in which the launcher doesn’t work, and it makes recovery from a bad update much, much easier. Updating the launcher would still be tricky, but the launcher is tiny, so deleting old versions in a timely manner is not so important.
- cozzyd 1y ago
  You just use a similar approach to how you update the bootloader on a microcontroller... the "application" does it.
- ginko 1y ago
  It's bad enough that shit happens on the web. I don't want to be treated like a test subject by my own desktop software.
cozzyd 1y ago
Why not just have a "A new version is available" thing pop up at the top of the window (that can be closed out)? A user is not necessarily connected to fast or cheap internet at all times. Automatically downloaded updates are a bane on infrastructure at remote field stations, for example. Also, it any updating mechanism should be trivially overridable by configuration so that the system package manager can be used, if the application is installed by the system network manager. I hate it when applications manage their own updates... give me a yum repo instead.
veyh 1y ago
> Windows generally treats running binaries as locked. Any attempt to overwrite a running binary should throw an exception. Again, the Zed source provides a reasonable solution. While it's true that you can't overwrite a running binary on Windows, you can still rename it. For example, the updater I wrote for AutoPTT downloads the new update to a temporary directory, renames all old files to ${file}.bak, then moves over the files from the temporary folder, and finally runs the new binary while exiting the old one.

65 points by piker 1y ago

> On POSIX systems, in theory, we can just update the files in place. Except for dynamically loaded assets, this should work just fine. The OS treats file locking as advisory and open file descriptors are irrelevant, so even if the file is running you overwrite it in place and the user will get the freshest version on the next launch. If you do this on macOS, and your code is codesigned, you are setting yourself for pain. Instead you should replace the file entirely so the file gets a new vnode.

piker 1y ago
That's super interesting. Tritium is signed and notarized (as is required) and uses the clean second-binary approach on all platforms, so I never ran into this. One thing that did come up was the requirement to use `ditto` rather than a standard unzipping utility to preserve some macOS-sensitive metadata. [EDIT: the post has been updated for this point. Thanks so much. https://tritium.legal/blog/update#3 https://tritium.legal/blog/update#3.]
- saagarjha 1y ago
  FWIW https://developer.apple.com/documentation/security/updating-mac-software https://developer.apple.com/documentation/security/updating-... is the relevant documentation for this if you want more details.
silon42 1y ago
Not just MacOS, on Linux this is the correct strategy too. It covers more cases: either a new instance is starting, or it hasn't fully loaded the file yet.
formerly_proven 1y ago
Binaries are memory-mapped, there is no guarantee for code to be resident. Modifying a running binary hence can cause a mixture of code to be paged into a running application. Which is why unix traditionally has a special "non-advisory" file locking for mapped binaries, which tries to give you an ETXTBSY error. This is nowhere near fool-proof though, e.g. Linux stopped doing this for mapped .so files, it probably doesn't work on NFS etc.
fulafel 1y ago
Traditionally if you update the contents of a running binary in-place in Unix you're liable to get a SIGBUS. The way to do it is unlink the existing binary and create a new one with the same name. Which is eg what install(1) does.
- amluto 1y ago
  SIGBUS or SIGSEGV if you’re lucky. It’s really an outrageous memory safety violation and could do anything, even allow RCE (in a narrow window).
- kelnos 1y ago
  That isn't robust to failures, though. What you sould be doing is first writing the new binary using a temporary file name, and then you can rename the new file to the correct binary name, which is an atomic operation.

IshKebab 1y ago

If you want to do this really well I'd do it like A/B updates on Android work. 1. Have two directories for the app: A and B. One contains the "current" version and one contains (possibly) the "next" version. 2. Have the actual app you start be a separate launcher program that just picks which version to launch. 3. While the app is running, periodically check for updates. If an update is available download and unzip it to the A or B directory that isn't currently in use. 4. Next time the launcher program starts it can say "an updated version is already installed, would you like to use it" (or you can just do that automatically if you - or your user - decide that's the best option). If so it marks the appropriate A/B directory as "current" and launches that one. Zero delay for users, works on all OSes, works with "always use the latest version" as well as "ask the user to update". I've never tried this tbf - just an idea.

whytevuhuni 1y ago
One caveat (which most apps already do anyway) is to ensure apps have at most one open instance (i.e. switch focus to the existing one), otherwise this might block both versions and result in some really confusing errors for the user, or corrupt a running app’s state if it doesn’t expect its resources to change.

lifthrasiir 1y ago

For Windows, there is an alternative approach: simply rename the running application to a different name (e.g. app.exe to app-old-timestamp.exe). This frees up the file name which can be overwritten to any other executable. The only problem I'm aware of is that this process may leave app.exe non-existent for a moment [1], but in return a need for keeping two executables vanishes. [1] Technically speaking I believe this can be solved with transactional NTFS (deprecated but still available as of Windows 11, AFAIK).

CaptainOfCoit 1y ago
Can you rename applications that are currently running? Last I recall, trying to rename a binary you're currently running would lead to some weird "cannot access" errors or similar. But I mostly use Linux, maybe I misunderstood it. Last time I came across it, I worked around it by having a "launcher" that wrote to a temporary path, closed old process, moved and started again. Slight indirection, but seemed to always work and be robust enough, compared to what we initially tried.
- lifthrasiir 1y ago
  Surprisingly, yes. Go does this for example.
- diath 1y ago
  Yes, I was recently working for an updater for a game and that's essentially how I solved it. The updater renames the current .dll and .exe files to .dll.old and .exe.old, then downloads the new .dll and .exe files, then on next launch, the application looks for .dll.old and .exe.old files in the current working directory and deletes them.
  - r_hanz 1y ago
    This is exactly how I've achieved this on Windows in the past. However, since writing that solution, I have come to understand that utilizing AppDomains (for .Net Framework) and AssemblyLoadContexts (for .Net Core) to load/unload the binaries seems to be the intended workflow.

mrasong 1y ago

Love that they prioritized both privacy (no sketchy background daemons) and security (launch-time checks) for legal apps—finally a team that gets how critical "no surprises" is for tools handling sensitive docs.

goodpoint 1y ago

> An integrated drafting environment needs to be trusted with reading, editing and redlining confidential and trade secret documents. And yet it's running on windows?!

CaptainOfCoit 1y ago
Like it or not, confidential and trade secret documents pass through and are edited on Windows machines, among others.
- goodpoint 1y ago
  Not my point. TFA is writing about the importance of security and privacy, not me.
  - CaptainOfCoit 1y ago
    You seem to be missing the point. Your complaint is that it runs on Windows, which fails to take into account the target audience of this (lawyers) who most likely are running Windows. So TFA is about the "importance of security and privacy" even on Windows, since the application is only available for Windows.

ginko 1y ago

At least on linux programs shouldn't update themselves. This is the package manager's job.

vbezhenar 1y ago
Not everyone prefers to use package manager. I think that package manager is weird concept and should not exist.
- CaptainOfCoit 1y ago
  Do you feel the same way about app stores too, like AppStore or Windows Store? What is your preferred way of downloading/installation applications and keeping them up to date? Personally I love being able to run one command and update everything in one swoop when I have time for it, instead of individually updating things all the time.
  - vbezhenar 1y ago
    > Do you feel the same way about app stores too, like AppStore or Windows Store? AppStores are good enough, as long as they're optional file hosting and censorship service and do not change actual application. So basically I want no middle-man between developer build and my computer. If developer uploads their application to AppStore and the app gets downloaded to my computer unchanged, that's fine. If some unrelated person pulls sources from github, alters them, builds them and ships them to my computer, that's what I don't like. > What is your preferred way of downloading/installation applications and keeping them up to date? Best way is statically linked binary available for direct download (many Golang and Rust apps are distributed this way directly from github). I check for updates when I feel like it, which might not be the best way, so some automatic update reminders are welcome, but by no means are necessary. Second best way is targzipped directory which I unpack into something like ~/apps/idea and then it keeps itself up to date automatically. Another thing that I don't like about package managers is that they spill application files all over filesystem. That's also wrong. Files which belong to application must be kept in one place. Ideally separated by chroot-like mechanisms.
- 1y ago

tyilo 1y ago

I use the `self_replace` crate for this: https://docs.rs/self-replace/latest/self_replace/ https://docs.rs/self-replace/latest/self_replace/

eviks 1y ago

If only desktop OS had decades to properly sort the updates out so that apps wouldn't have to reinvent the wheel...

piker 1y ago
The problem is they opted for lock-in via walled-off app stores given the opportunity.

amluto 1y ago

I’m surprised there’s no attempt to use an A/B or multi version approach. It’s only a partial solution, but the main binary could be a very minimal launcher that enumerates installed versions and decides which version to run. This results in no window at all in which the launcher doesn’t work, and it makes recovery from a bad update much, much easier. Updating the launcher would still be tricky, but the launcher is tiny, so deleting old versions in a timely manner is not so important.

cozzyd 1y ago
You just use a similar approach to how you update the bootloader on a microcontroller... the "application" does it.
ginko 1y ago
It's bad enough that shit happens on the web. I don't want to be treated like a test subject by my own desktop software.

cozzyd 1y ago

Why not just have a "A new version is available" thing pop up at the top of the window (that can be closed out)? A user is not necessarily connected to fast or cheap internet at all times. Automatically downloaded updates are a bane on infrastructure at remote field stations, for example. Also, it any updating mechanism should be trivially overridable by configuration so that the system package manager can be used, if the application is installed by the system network manager. I hate it when applications manage their own updates... give me a yum repo instead.

veyh 1y ago

> Windows generally treats running binaries as locked. Any attempt to overwrite a running binary should throw an exception. Again, the Zed source provides a reasonable solution. While it's true that you can't overwrite a running binary on Windows, you can still rename it. For example, the updater I wrote for AutoPTT downloads the new update to a temporary directory, renames all old files to ${file}.bak, then moves over the files from the temporary folder, and finally runs the new binary while exiting the old one.