5 ms·

> Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, Be aware that uv will create a full copy of that environ

> Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies, Be aware that uv will create a full copy of that environment for each script by default. Depending on your number of scripts, this could become wasteful really fast. There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default, or which disadvantages this has, but so far it's working fine for me, and saved me several gigabytes of storage.

sc077y 1y ago
By default it will create hard links for python packages, so it won't consume any more memory (besides the small overhead of hard links).
- Hackbraten 1y ago
  Unless it can't because you happen to have mounted your user cache directory from a different volume in an attempt to debloat your hourly backups.
  - zahlman 1y ago
    ... your hourly backups aren't also using the same hard-linking strategy?
    - Hackbraten 1y ago
      They’re ZFS snapshots, so no. That’s why I’m forced to keep my cache directory in a different dataset. Ironically, if my backup scheme were using hard links, then I could simply exclude the cache directory from backup, so I’d have no reason to do that mountpoint spiel, and uv’s hard links would work normally.
      - zahlman 1y ago
        ... ZFS snapshots don't produce copies of the underlying file data, do they? (Yeah, I'm still a dinosaur on ext4...)
        Hackbraten 1y ago
        Nothing wrong with ext4! Having choices and preferences is a good thing. You’re correct, ZFS snapshots don’t produce copies, at least not at the time they’re being created. They work a little like copy-on-write.
  - sc077y 1y ago
    In that case, you use copy OR what you can can also do, if you really care about disk usage, is use symbolic links between the drives. have a .venv sym link on drive A (raid 1) point to the uv_cache_dir's venv on drive B (raid 0). I have not tested though what happens when you unmount and sync.
- zahlman 1y ago
  As far as I can tell, this only applies to the wheel contents, not to the .pyc bytecode cache created by the interpreter. If you use the defaults, Python will just create per-environment copies on demand; if you precompile with `--compile-bytecode`, uv will put the results directly in the installed copy rather than caching them and hard-linking from there. I plan on offering this kind of cached precompiled bytecode in PAPER, but I know that compiled bytecode includes absolute paths (used for displaying stack traces) that will presumably then refer to the cached copies. I'll want to test as best I can that this doesn't break anything more subtle.
zahlman 1y ago
>There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default The hard-link strategy still saves disk space — although not as much (see other replies and discussion). Possible reasons not to symlink that I can think of: * stack traces in uncaught exceptions might be wonky (I haven't tried it) * it presumably adds a tiny hit to imports from Python, but I find it hard to imagine anyone caring * with the hard-linking strategy, you can purge the cache (perhaps accidentally?) without affecting existing environments

sc077y 1y ago

By default it will create hard links for python packages, so it won't consume any more memory (besides the small overhead of hard links).

Hackbraten 1y ago
Unless it can't because you happen to have mounted your user cache directory from a different volume in an attempt to debloat your hourly backups.
- zahlman 1y ago
  ... your hourly backups aren't also using the same hard-linking strategy?
  - Hackbraten 1y ago
    They’re ZFS snapshots, so no. That’s why I’m forced to keep my cache directory in a different dataset. Ironically, if my backup scheme were using hard links, then I could simply exclude the cache directory from backup, so I’d have no reason to do that mountpoint spiel, and uv’s hard links would work normally.
    - zahlman 1y ago
      ... ZFS snapshots don't produce copies of the underlying file data, do they? (Yeah, I'm still a dinosaur on ext4...)
      - Hackbraten 1y ago
        Nothing wrong with ext4! Having choices and preferences is a good thing. You’re correct, ZFS snapshots don’t produce copies, at least not at the time they’re being created. They work a little like copy-on-write.
- sc077y 1y ago
  In that case, you use copy OR what you can can also do, if you really care about disk usage, is use symbolic links between the drives. have a .venv sym link on drive A (raid 1) point to the uv_cache_dir's venv on drive B (raid 0). I have not tested though what happens when you unmount and sync.
zahlman 1y ago
As far as I can tell, this only applies to the wheel contents, not to the .pyc bytecode cache created by the interpreter. If you use the defaults, Python will just create per-environment copies on demand; if you precompile with `--compile-bytecode`, uv will put the results directly in the installed copy rather than caching them and hard-linking from there. I plan on offering this kind of cached precompiled bytecode in PAPER, but I know that compiled bytecode includes absolute paths (used for displaying stack traces) that will presumably then refer to the cached copies. I'll want to test as best I can that this doesn't break anything more subtle.

zahlman 1y ago

>There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default The hard-link strategy still saves disk space — although not as much (see other replies and discussion). Possible reasons not to symlink that I can think of: * stack traces in uncaught exceptions might be wonky (I haven't tried it) * it presumably adds a tiny hit to imports from Python, but I find it hard to imagine anyone caring * with the hard-linking strategy, you can purge the cache (perhaps accidentally?) without affecting existing environments