7 ms·

I enjoy using uv a lot but am getting afraid that it is getting bloated for no reason. For ex., the number of niche flags that a lot of subcommands support is v

by alkh 1y ago

I enjoy using uv a lot but am getting afraid that it is getting bloated for no reason. For ex., the number of niche flags that a lot of subcommands support is very high + some of them seemingly achieve the same result(uv run --no-project and uv run --active). I'd rather them working on improving existing tools and documentation than adding new (redundant) functionality

treyd 1y ago
Are they baked into the main executable or are they separate binaries (a la apt, cargo, etc)?
- charliermarsh 1y ago
  It's a separate binary -- we install Ruff if you invoke `uv format`. So if you don't invoke `uv format`, there's no impact on the binary size, etc.
  - HelloNurse 1y ago
    But, for example, my project already uses Ruff, and I have to worry about having a "managed" extra copy of Ruff that subtly alters the normal functioning both of "uv tool run" and of ruff itself.
    - zanie 1y ago
      It's intentionally distinct from the `uv tool` interface — it won't change `ruff` or `uv tool run` behaviors.
benreesman 1y ago
It's really difficult to do Python projects in a sound, reproducible, reasonably portable way. uv sync is in general able to build you only a package set that it can promise to build again. But it can't in general build torch-tensorrt or flash-attn because it has no way of knowing if Mercury was in retrograde when you ran pip. They are trying to thread a delicate an economically pivotal needle: the Python community prizes privatizing the profits and socializing the costs of "works on my box". The cost of making the software deployable, secure, repeatable, reliable didn't go away! It just became someone else's problem at a later time in a different place with far fewer degrees of freedom. Doing this in a way that satisfies serious operations people without alienating the "works on my box...sometimes" crowd is The Lord's Work.
- kouteiheika 1y ago
  > But it can't in general build torch-tensorrt or flash-attn because it has no way of knowing if Mercury was in retrograde when you ran pip. This is a self-inflicted wound, since flash attention insist on building a native C++ extension which is completely unnecessary in this case. What you can do is the following: 1) Compile your CUDA kernels offline. 2) Include those compiled kernels in a package you push to pypi. 3) Call into the kernels with pure Python, without going through a C++ extension. I do this for the CUDA kernels I maintain and it works great. Flash attention currently publishes 48 (!) different packages[1], for different combinations of pytorch and C++ ABI. With this approach it would have to only publish one, and it would work for every combination of Python and pytorch. [1] - https://github.com/Dao-AILab/flash-attention/releases/tag/v2.8.3 https://github.com/Dao-AILab/flash-attention/releases/tag/v2...
  - twothreeone 1y ago
    While shipping binary kernels may be a workaround for some users, it goes against what many people would consider "good etiquette" for various valid reasons, such as hackability, security, or providing free (as in liberty) software.
    - rkangel 1y ago
      Shipping binary artifacts isn't inherently a bad thing - that's what (most) Linux Distros do after all! The important distinction is how that binary package was arrived at. If it's a mystery and the source isn't available then that's bad. If it's all in the open source repo and part of the Python package build and is completely reproducible then that's great.
      - benreesman 1y ago
        GP's solution is the one that I (and I think most people) ultimately wind up using. But it's not a nice time in the usual "oh we'll just compile it" sense of a typical package. flash-attn in particular has its build so badly misconfigured and is so heavy that it will lock up a modern Zen 5 machine with 128GB of DDR5 if you don't re-nice ninja (assuming of course that you remembered it just won't work without a pip-visible ninja). It can't build a wheel (at least not obviously) that will work correctly on Ampere and Hopper both, it incorrectly declares it's dependencies so it will demand torch even if torch is in your pyproject.toml and you end up breaking build isolation. So now you've got your gigabytes of fragile wheel that won't run on half your cards, let's make a wheel registry. Oh, and machine learning everything needs it: half of diffusers crashes at runtime without it. Runtime. The dirty little secret of these 50MM offers at AI companies is that way more people understand the math (which is actually pretty light compared to say graduate physics) than can build and run NVIDIA wheels at scale. The wizards who Zuckerberg will fellate are people who know some math and can run Torch on a mixed Hopper/Blackwell fleet. And this (I think) is Astral's endgame. I think pyx is going to fix this at scale and they're going to abruptly become more troublesome to NVIDIA than George Hotz or GamersNexus.
darkamaul 1y ago
I don’t understand why adding a subcommand to uv is being considered bloat. uv is already a complex tool, but it has excellent documentation. Adding a straightforward, self-explanatory subcommand like this doesn’t seem out of place.
- zelphirkalt 1y ago
  Developing a good and configurable formatter is a project almost as complex as uv's main focus, managing packages. As such it will divert time and attention from its main purpose and the outcome is questionable, considering how many bad formatters are out there. For example the "opinionated" black is terrible and has no awareness of semantics.
  - alfons_foobar 1y ago
    The formatter already exists, my understanding is that this is merely an alias for "uvx ruff"...
  - charliermarsh 1y ago
    We already develop a formatter: Ruff (https://github.com/astral-sh/ruff https://github.com/astral-sh/ruff). Ruff and uv are built by the same team. `uv format` is just an optional front-end to `ruff format`.
- doctorpangloss 1y ago
  > I don’t understand why adding a subcommand to uv is being considered bloat. It's simple: - Everything I use and I see: good - Everything I don't use and I see: bloat - Everything I use and I don't see: good - Everything I think I don't use and I don't see: Java
fluidcruft 1y ago
When talking about uv, this comes across similar to saying that "the 'make' command has too many targets".

by alkh 1y ago

treyd 1y ago

Are they baked into the main executable or are they separate binaries (a la apt, cargo, etc)?

charliermarsh 1y ago
It's a separate binary -- we install Ruff if you invoke `uv format`. So if you don't invoke `uv format`, there's no impact on the binary size, etc.
- HelloNurse 1y ago
  But, for example, my project already uses Ruff, and I have to worry about having a "managed" extra copy of Ruff that subtly alters the normal functioning both of "uv tool run" and of ruff itself.
  - zanie 1y ago
    It's intentionally distinct from the `uv tool` interface — it won't change `ruff` or `uv tool run` behaviors.

benreesman 1y ago

It's really difficult to do Python projects in a sound, reproducible, reasonably portable way. uv sync is in general able to build you only a package set that it can promise to build again. But it can't in general build torch-tensorrt or flash-attn because it has no way of knowing if Mercury was in retrograde when you ran pip. They are trying to thread a delicate an economically pivotal needle: the Python community prizes privatizing the profits and socializing the costs of "works on my box". The cost of making the software deployable, secure, repeatable, reliable didn't go away! It just became someone else's problem at a later time in a different place with far fewer degrees of freedom. Doing this in a way that satisfies serious operations people without alienating the "works on my box...sometimes" crowd is The Lord's Work.

kouteiheika 1y ago
> But it can't in general build torch-tensorrt or flash-attn because it has no way of knowing if Mercury was in retrograde when you ran pip. This is a self-inflicted wound, since flash attention insist on building a native C++ extension which is completely unnecessary in this case. What you can do is the following: 1) Compile your CUDA kernels offline. 2) Include those compiled kernels in a package you push to pypi. 3) Call into the kernels with pure Python, without going through a C++ extension. I do this for the CUDA kernels I maintain and it works great. Flash attention currently publishes 48 (!) different packages[1], for different combinations of pytorch and C++ ABI. With this approach it would have to only publish one, and it would work for every combination of Python and pytorch. [1] - https://github.com/Dao-AILab/flash-attention/releases/tag/v2.8.3 https://github.com/Dao-AILab/flash-attention/releases/tag/v2...
- twothreeone 1y ago
  While shipping binary kernels may be a workaround for some users, it goes against what many people would consider "good etiquette" for various valid reasons, such as hackability, security, or providing free (as in liberty) software.
  - rkangel 1y ago
    Shipping binary artifacts isn't inherently a bad thing - that's what (most) Linux Distros do after all! The important distinction is how that binary package was arrived at. If it's a mystery and the source isn't available then that's bad. If it's all in the open source repo and part of the Python package build and is completely reproducible then that's great.
    - benreesman 1y ago
      GP's solution is the one that I (and I think most people) ultimately wind up using. But it's not a nice time in the usual "oh we'll just compile it" sense of a typical package. flash-attn in particular has its build so badly misconfigured and is so heavy that it will lock up a modern Zen 5 machine with 128GB of DDR5 if you don't re-nice ninja (assuming of course that you remembered it just won't work without a pip-visible ninja). It can't build a wheel (at least not obviously) that will work correctly on Ampere and Hopper both, it incorrectly declares it's dependencies so it will demand torch even if torch is in your pyproject.toml and you end up breaking build isolation. So now you've got your gigabytes of fragile wheel that won't run on half your cards, let's make a wheel registry. Oh, and machine learning everything needs it: half of diffusers crashes at runtime without it. Runtime. The dirty little secret of these 50MM offers at AI companies is that way more people understand the math (which is actually pretty light compared to say graduate physics) than can build and run NVIDIA wheels at scale. The wizards who Zuckerberg will fellate are people who know some math and can run Torch on a mixed Hopper/Blackwell fleet. And this (I think) is Astral's endgame. I think pyx is going to fix this at scale and they're going to abruptly become more troublesome to NVIDIA than George Hotz or GamersNexus.

darkamaul 1y ago

I don’t understand why adding a subcommand to uv is being considered bloat. uv is already a complex tool, but it has excellent documentation. Adding a straightforward, self-explanatory subcommand like this doesn’t seem out of place.

zelphirkalt 1y ago
Developing a good and configurable formatter is a project almost as complex as uv's main focus, managing packages. As such it will divert time and attention from its main purpose and the outcome is questionable, considering how many bad formatters are out there. For example the "opinionated" black is terrible and has no awareness of semantics.
- alfons_foobar 1y ago
  The formatter already exists, my understanding is that this is merely an alias for "uvx ruff"...
- charliermarsh 1y ago
  We already develop a formatter: Ruff (https://github.com/astral-sh/ruff https://github.com/astral-sh/ruff). Ruff and uv are built by the same team. `uv format` is just an optional front-end to `ruff format`.
doctorpangloss 1y ago
> I don’t understand why adding a subcommand to uv is being considered bloat. It's simple: - Everything I use and I see: good - Everything I don't use and I see: bloat - Everything I use and I don't see: good - Everything I think I don't use and I don't see: Java

fluidcruft 1y ago

When talking about uv, this comes across similar to saying that "the 'make' command has too many targets".