15 ms·

PYX: The next step in Python packaging

• megamix 1y ago
  No Thanks. For majority of my use case pip is just fine. I’m not here to chase time just to live life
  • fastball 1y ago
    Strange to reject something when you don't even understand what it is. PYX is a package registry, and therefore an alternative to PyPI (like how JSR is an alternative to NPM). The alternative to `pip` that Astral has built is called `uv`. Feel free to not use that either, but personally I'd consider it if I were you. It has full pip compatibility (with significantly more speed), and many other nice features besides.
    • megamix 1y ago
      Yes, it still follow the same PEP standard no? So why would I invest in something like that and I’m never impressed by “speed improvements”. uv high speed infrastructure is not really what is my bottleneck in projects.
      • fastball 1y ago
        Two other major benefits besides speed were listed in the opening paragraph about why you might want to use PYX. Likewise with uv, speed is only one of the (many) improvements over pip.
  • joshdavham 1y ago
    This is a low quality comment. From the guidelines [1] > Please don't post shallow dismissals, especially of other people's work. [1] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    • megamix 1y ago
      Thanks for pointing that out. You could also say it’s a condensed high entropy comment. Anyway I’ll outline a more thorough dismissal. Edit: and turns out a smart guy is top rated comment atm "All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community."
• notatallshaw 1y ago
  Probably the more useful blog post: https://astral.sh/blog/introducing-pyx https://astral.sh/blog/introducing-pyx
  • 6thbit 1y ago
    Thanks that’s bit less cryptic than the linked page. Still don’t get how they are solving what they claim to solve.
    • chao- 1y ago
      I suspect that, in order to succeed, they will need to build something that is isomorphic to Nix.
      • anon7000 1y ago
        They already built uv, which works extremely well for that
      • conradev 1y ago
        Yeah, and uv2nix is already pretty good! I wonder if pyx will be competing with uv2nix. It's easy to compete with Nix tooling, but pretty hard to compete with the breadth of nixpkgs.
    • normerdfe 1y ago
      I spend little time with Python, but I didn’t have any problems using uv. Given how great uv is, I’d like to use pyx, but first it would be good if they could provide a solid argument for using it.
    • twelvechairs 1y ago
      I'm guesssing from the UV page [0] its mainly if the speed of pip is a problem for you? [0] https://docs.astral.sh/uv/ https://docs.astral.sh/uv/
      • benreesman 1y ago
• wiseowise 1y ago
  [flagged]
• PaulHoule 1y ago
  Been waiting for something like this to make it easier to manage multi-package projects.
• ddavis 1y ago
  Been waiting to see what Astral would do first (with regards to product). Seems like a mix of artifactory and conda? artifactory providing a package server and conda trying to fix the difficulty that comes from Python packages with compiled components or dependencies, mostly solved by wheels, but of course PyTorch wheels requiring specific CUDA can still be a mess that conda fixes
  • notatallshaw 1y ago
    Given Astral's heavy involvement in the wheelnext project I suspect this index is an early adopter of Wheel Variants which are an attempt to solve the problems of CUDA (and that entire class of problems not just CUDA specifically) in a more automated way than even conda: https://wheelnext.dev/proposals/pepxxx_wheel_variant_support/ https://wheelnext.dev/proposals/pepxxx_wheel_variant_support...
    • zanie 1y ago
      It's actually not powered by Wheel Variants right now, though we are generally early adopters of the initiative :)
      • notatallshaw 1y ago
        Well it was just a guess, "GPU-aware" is a bit mysterious to those of us on the outside ;).
• TheChaplain 1y ago
  Pyx is just a registry, just like Pypi, or did I misunderstood it?
  • bitpush 1y ago
    Sounds like it. Also .. > pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever.
    • woodruffw 1y ago
      FWIW, I think the full paragraph around that snippet is important context: > Beyond the product itself, pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever. Nothing changes there. Instead, we'll offer paid, hosted services that represent the "natural next thing you need" when you're already using our tools: the Astral platform. pyx itself is not a tool, it's a service.
  • woodruffw 1y ago
    Not exactly -- part of pyx is a registry (and that part speaks the same standards as PyPI), but the bigger picture is that pyx part of a larger effort to make Python packaging faster and more cohesive for developers. To be precise: pyx isn't intended to be a public registry or a free service; it's something Astral will be selling. It'll support private packages and corporate use cases that are (reasonably IMO) beyond PyPI's scope. (FD: I work on pyx.)
    • 6thbit 1y ago
      > but the bigger picture is that pyx part of a larger effort to make Python packaging faster and more cohesive for developers Can you elaborate on what will make it faster and cohesive?
    • aragilar 1y ago
      I'm confused how pyx would achieve this. In my experience unless uv/pyx is planning on pursuing being a new conda (and even then it's pretty incomplete), a single cohesive isn't possible because uv/pyx can't be the universal single source of truth. The value of Python has always been its ability to be integrated into a larger whole, and so the Python tooling must always defer to the tooling for the larger whole (and that space is composed of numerous competing tools and ecosystems). PyPI-provided non-pure wheels don't integrate well outside the ecosystem that's been built there, so I don't see how doubling down on it solves the underlying problem?
• ctoth 1y ago
  As I said a couple weeks ago, they're gonna have to cash out at some point. The move won't be around Uv -- it'll be a protected private PyPi or something. https://news.ycombinator.com/item?id=44712558 https://news.ycombinator.com/item?id=44712558 Now what do we have here?
  • kinow 1y ago
    I haven't adopted uv yet watching to see what will be their move. We recently had to review our use of Anaconda tools due to their changes, then review Qt changes in license. Not looking forward to another license ordeal.
    • zanie 1y ago
      We're hoping that building a commercial service makes it clear that we have a sustainable business model and that our tools (like uv) will remain free and permissively licensed. (I work at Astral)
      • simonw 1y ago
        I think having a credible, proven business model is a feature of an open source project - without one there are unanswered questions about ongoing maintenance. I'm glad to see Astral taking steps towards that.
      • jsmeaton 1y ago
        I've been wondering where the commercial service would come in and this sounds like just the right product that aligns with what you're already doing and serves a real need. Setting up scalable private registries for python is awful.
    • __mharrison__ 1y ago
      You know what they say: The best time to adopt uv was last year... I'm all seriousness, I'm all in on uv. Better than any competition by a mile. Also makes my training and clients much happier.
    • 1y ago
• Myrmornis 1y ago
  I wonder whether it will have a flat namespace that everyone competes over or whether the top-level keys will be user/project identifiers of some sort. I hope the latter.
  • tmvphil 1y ago
    Fundamentally we still have the flat namespace of top level python imports, which is the same as the package name for ~95% of projects, so I'm not sure how they could really change that.
    • notatallshaw 1y ago
      Package names and module names are not coupled to each other. You could have package name like "company-foo" and import it as "foo" or "bar" or anything else. But you can if you want have a non-flat namespace for imports using PEP 420 – Implicit Namespace Packages, so all your different packages "company-foo", "company-bar", etc. can be installed into the "company" namespace and all just work. Nothing stops an index from validating that wheels use the same name or namespace as their package names. Sdists with arbitrary backends would not be possible, but you could enforce what backends were allowed for certain users.
  • cr125rider 1y ago
    Once we learn to namespace things it’s gonna be so nice. Seems we keep re-learning that lesson…
    • Y_Y 1y ago
      import io from contextlib import redirect_stdout as r b=io.StringIO() with r(b):import this print(b.getvalue().splitlines()[-1]) > Namespaces are one honking great idea -- let's do more of those!
  • actinium226 1y ago
    I feel like github solved this problem pretty well.
• monster_truck 1y ago
  I've been burned too many times by embracing open source products like this. We've been fed promises like these before. They will inevitably get acquired. Years of documentation, issues, and pull requests will be deleted with little-to-no notice. An exclusively commercial replacement will materialize from the new company that is inexplicably missing the features you relied on in the first place.
  • deleted 1y ago
    [deleted]
  • woodruffw 1y ago
    For what it's worth, I understand this concern. However, I want to emphasize that pyx is intentionally distinct from Astral's tools. From the announcement post: > Beyond the product itself, pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever. Nothing changes there. Instead, we'll offer paid, hosted services that represent the "natural next thing you need" when you're already using our tools: the Astral platform. Basically, we're hoping to address this concern by building a separate sustainable commercial product rather than monetizing our open source tools.
    • pbowyer 1y ago
      This is the way it's worked for PHP's composer packages, and it's worked well.
    • o11c 1y ago
      The entire reason people choose "permissive licenses" is so that it won't last forever. At best, the community can fork the old version without any future features. Only viral licenses are forever.
      • woodruffw 1y ago
        I don't think this is true -- a license's virality doesn't mean that its copyright holders can't switch a future version to a proprietary license; past grants don't imply grants to future work under any open source license.
• simonw 1y ago
  This is effectively what Charlie said they were going to build last September when quizzed about their intended business model on Mastodon: https://hachyderm.io/@charliermarsh/113103564055291456 https://hachyderm.io/@charliermarsh/113103564055291456
  • pietroppeter 1y ago
    And this fact effectively builds trust in the vision and in execution.
• metalliqaz 1y ago
  > Waitlist > Private registry ouch.
  • yoavm 1y ago
    I actually think this is great. If Astral can figure out a way to make money using a private registry (something that is used mainly by companies), then they'll have to resources to keep building their amazing open-source projects — Ruff and uv. That's a huge win for Python.
    • erikgahner 1y ago
      100% agree. I am more than happy to see Astral taking steps in this direction. People can continue to use uv, ruff, and ty without having to pay anything, but companies that benefit tremendously from open source initiatives can pay for a private package registry and directly support the continued development of said tools.
    • orra 1y ago
      In particular I think it's nice for uv and ruff to remain open source, not open core. And as you say, companies always need paid private registries, for their internal software. A true win-win.
• shining7526 1y ago
  If uv can install a Python version in 32ms (a time that I saw and confirmed just this morning), then sign me up.
  • enmyj 1y ago
    amen
• _verandaguy 1y ago
  Soon: there are 14 competing Python packaging standards. This is a joke, obviously. We've had more than 14 for years.
  • woodruffw 1y ago
    Python packaging has a lot of standards, but I would say most of them (especially in the last decade) don't really compete with each other. They lean more towards the "slow accretion of generally considered useful features" style. This itself is IMO a product of Python having a relatively healthy consensus-driven standardization process for packaging, rather than an authoritative one. If Python had more of an authoritative approach, I don't think the language would have done as well as it has. (Source: I've written at least 5 PEPs.)
    • callc 1y ago
      Do you really think Python’s consensus-driven language development is better than authoritarian? I am honestly tired of the Python packing situation. I breathe a sigh of relief in language like Go and Rust with an “authoritative” built-in solution. I wouldn’t mind the 30 different packaging solutions as long as there was authoritative “correct” solution. All the others would then be opt-in enhancements as needed. I guess a good thought experiment would be if we were to design a packaging system (or decide not to) for a new PL like python, what would it look like?
      • woodruffw 1y ago
        > I breathe a sigh of relief in language like Go and Rust with an “authoritative” built-in solution. I don't know about Go, but Rust's packaging isn't authoritative in the sense that I meant. There's no packaging BDFL; improvements to Rust packaging happen through a standards process that closely mirrors that of Python's PEPs. I think the actual difference between Rust and Python is that Rust made the (IMO correct) decision early on to build a single tool for package management, whereas Python has historically had a single installer and left every other part of package management up to the user. That's a product of the fact that Python is more of a patchwork ecosystem and community than Rust is, plus the fact that it's a lot older and a lot bigger (in terms of breadth of user installation base). Basically, hindsight is 20/20. Rust rightly benefited from Python's hard lesson about not having one tool, but they also rightly benefited from Python's good experience with consensus-driven standardization.
• cpeterso 1y ago
  How do you pronounce "pyx"? Pikes, picks, pie-ex?
  • woodruffw 1y ago
    We've been pronouncing it pea-why-ecks, like uv (you-vee) and ty (tee-why). But I wouldn't say that's permanent yet.
    • aaronvg 1y ago
      definitely will just say picks...
    • xavdid 1y ago
      that's fascinating - I've definitely been saying "you've" and "tie". I assumed this was "picks"
    • r_lee 1y ago
      py-x like pie-ex?
  • Y_Y 1y ago
    I demand it be pronounced like the existing English word "pyx" (aka "pyxis"), meaning a weird religious box: https://en.wikipedia.org/wiki/Pyx https://en.wikipedia.org/wiki/Pyx
• m4r71n 1y ago
  What does GPU-aware mean in terms of a registry? Will `uv` inspect my local GPU spec and decide what the best set of packages would be to pull from Pyx? Since this is a private, paid-for registry aimed at corporate clients, will there be an option to expose those registries externally as a public instance, but paid for by the company? That is, can I as a vendor pay for a Pyx registry for my own set of packages, and then provide that registry as an entrypoint for my customers?
  • charliermarsh 1y ago
    > Will `uv` inspect my local GPU spec and decide what the best set of packages would be to pull from Pyx? We actually support this basic idea today, even without pyx. You can run (e.g.) `uv pip install --torch-backend=auto torch` to automatically install a version of PyTorch based on your machine's GPU from the PyTorch index. pyx takes that idea and pushes it further. Instead of "just" supporting PyTorch, the registry has a curated index for each supported hardware accelerator, and we populate that index with pre-built artifacts across a wide range of packages, versions, Python versions, PyTorch versions, etc., all with consistent and coherent metadata. So there are two parts to it: (1) when you point to pyx, it becomes much easier to get the right, pre-built, mutually compatible versions of these things (and faster to install them); and (2) the uv client can point you to the "right" pyx index automatically (that part works regardless of whether you're using pyx, it's just more limited). > Since this is a private, paid-for registry aimed at corporate clients, will there be an option to expose those registries externally as a public instance, but paid for by the company? That is, can I as a vendor pay for a Pyx registry for my own set of packages, and then provide that registry as an entrypoint for my customers? We don't support this yet but it's come up a few times with users. If you're interested in it concretely feel free to email me (charlie@).
    • scarlehoff 1y ago
      Hi Charlie what happens in a situation in which I might have access to a login node, from which I can install packages, but then the computing nodes don't have internet access. Can I define in some hardware.toml the target system and install there even if my local system is different? To be more specific, I'd like to do `uv --dump-system hardware.toml` in the computing node and then in the login node (or my laptop for that matter) just do `uv install my-package --target-system hardware.toml` and get an environment I can just copy over.
• dakiol 1y ago
  I'm brushing up with Python for a new job, and boy what a ride. Not because of the language itself but the tooling around packages. I'm coming from Go and TS/JS and while these two ecosystems have their own pros and cons, at least they are more or less straightforward to get onboarded (there are 1 or 2 tools you need to know about). In Python there are dozens of tools/concepts related to packaging: pip, easy_install, setuptools, setup.py, pypy, poetry, uv, venv, virtualenv, pipenv, wheels, ... There's even an entire website dedicated to this topic: https://packaging.python.org https://packaging.python.org Don't understand how a private company like Astral is leading here. Why is that hard for the Python community to come up with a single tool to rule them all? (I know https://xkcd.com/927/ https://xkcd.com/927/). Like, you could even copy what Go or Node are doing, and make it Python-aware; no shame on that. Instead we have these who-knows-how-long-they-will-last tools every now and then. They should remove the "There should be one-- and preferably only one --obvious way to do it." from the Python Zen.
  • formerly_proven 1y ago
    > Don't understand how a private company like Astral is leading here. Why is that hard for the Python community to come up with a single tool to rule them all? (I know https://xkcd.com/927/ https://xkcd.com/927/). Like, you could even copy what Go or Node are doing, and make it Python-aware; no shame on that. Instead we have these who-knows-how-long-they-will-last tools every now and then. Python packaging is (largely) solving problems that Go and Node packaging are not even trying to address.
    • Spivak 1y ago
      Specifically simultaneous distribution of precompiled binaries for many different OS and hardware configurations and built-on-demand source distribution of non-Python software to be used as dependencies with as little (ideally none) host setup by the user all installable under a single name/version everywhere. imagine a world without: failed to build native gem extension
    • asa400 1y ago
      As someone outside those communities, could you elaborate?
• m_kos 1y ago
  > Why is it so hard to install PyTorch, or CUDA, or libraries like FlashAttention or DeepSpeed that build against PyTorch and CUDA? This is so true! On Windows (and WSL) it is also exacerbated by some packages requiring the use of compilers bundled with outdated Visual Studio versions, some of which are only available by manually crafting download paths. I can't wait for a better dev experience.
  • miohtama 1y ago
    In the past, part of the definition of an operating system was that it ships with a compiler.
    • int_19h 1y ago
      When was that ever a part of the definition? It was part of the early Unix culture, sure, but even many contemporary OSes didn't ship with compilers, which were a separate (and often very expensive!) piece of software. OTOH today most Linux distros don't install any dev tools by default on a clean install. And, ironically, a clean install of Windows has .NET, which includes a C# compiler.
  • giancarlostoro 1y ago
    Stuff like that led me fully away from Ruby (due to Rails), which is a shame, I see videos of people chugging along with Ruby and loving it, and it looks like a fun language, but when the only way I can get a dev environment setup for Rails is using DigitalOcean droplets, I've lost all interest. It would always fail at compiling something for Rails. I would have loved to partake in the Rails hype back in 2012, but over the years the install / setup process was always a nightmare. I went with Python because I never had this issue. Now with any AI / CUDA stuff its a bit of a nightmare to the point where you use someone's setup shell script instead of trying to use pip at all.
    • nurettin 1y ago
      Have you tried JRuby? It might be a bit too large for your droplet, but it has the java versions of most gems and you can produce cross-platform jars using warbler.
• rob 1y ago
  Is there a big enough commercial market for private Python package registries to support an entire company and its staff? Looks like they're hiring for $250k engineers, starting a $26k/year OSS fund, etc. Expenses seem a bit high if this is their first project unless they plan on being acquired?
  • eldenring 1y ago
    Its interesting because the value is definitely there. Every single python developer you meet (many of who are highly paid) has a story about wasting a bunch of time on these things. The question is how much of this value can Astral capture. I think based on the quality of their work, there's also an important component which is trust. I'd trust and pay for a product from them much more readily than an open source solution with flaky maintainers.
    • est31 1y ago
      Yeah they certainly generate a lot of value by providing excellent productivity tooling. The question is how they capture some of that value, which is notoriously hard with an OSS license. A non-OSS license creates the adobe trap on the other hand, where companies deploy more and more aggressive moetization strategies, making life worse and worse for users of the software.
  • dimatura 1y ago
    Just one data point, but if it's as nice to use as their open source tools and not outrageously expensive, I'd be a customer. Current offerings for private python package registries are kind of meh. Always wondered why github doesn't offer this.
  • int_19h 1y ago
    Continuum has been doing something very similar with Anaconda, and they've been around for over a decade now.
    • hobofan 1y ago
      From what I can tell they (had to?) ramp up their aggressiveness regarding getting paid though. They had contacted a company I had worked for asking them to purchase a license, because apparently somewhere in the company some dev workflow had contacted the conda servers regularly. We never ended up tracing it down as it stopped some weeks before them contacting us, according to our network logs. In general, you don't resort to such sales tactics unless there is good cause (significant usage, where you as unpaid vendor also have leverage) or you just end up burning potential future customers (as was the case here, as they chose a bad contact point inside the company).
• fph 1y ago
  I lost track of how many different ways to install a Python library there are at the moment.
  • nodesocket 1y ago
    Much better than the Node a handful of years back. Everybody used NPM, everybody switched to Yarn, everybody switched back to NPM.
    • no_wizard 1y ago
      Thats because npm finally decided to adopt enough features as time went on that it could be used in place of yarn and eventually, if they adopt enough of the features of pnpm, it will replace that too. Though speaking as a long time developer in the ecosystem, switching between npm, yarn, and pnpm is fairly trivial in my experience. Especially after node_gyp went away
    • kavaruka 1y ago
      I think node has a better tooling and ecosystem right now. Astral is doing a great job to reduce the gap.
      • int_19h 1y ago
        Node ecosystem still has the problem where if you try to build a project two years later, chances are good it won't work, because breaking changes are so common, and this is then multiplied across all the tiny packages that are dependencies of your dependencies.
        • adithyassekhar 1y ago
          Don't lockfiles help?
          • thrown-0825 1y ago
            deps are delisted from npm and versions can be deleted
• JackYoustra 1y ago
  The real thing that I hope someone is able to solve is downloading such huge amounts of unnecessary code. As I understand, the bulk of the torch binary is just a huge nvfatbin compiled for every SM under the sun when you usually just want it to run on whatever accelerators you have on hand. Even just making narrow builds of like `pytorch-sm120a` (with stuff like cuBLAS thin binaries paired with it too) as part of a handy uv extra or something like that would make it much quicker and easier.
  • mikepurvis 1y ago
    Another piece is that PyPI has no index— it's just a giant list of URLs [1] where any required metadata (eg, the OS, python version, etc) is encoded in the filename. That makes it trivial to throw behind a CDN since it's all super static, but it has some important limitations: - there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info) - there's no way to get hashes of the archives - there's no way to do things like reverse-search (show me everything that depends on x) I'm assuming that a big part of pyx is introducing a dynamically served (or maybe even queryable) endpoint that can return package metadata and let uv plan ahead better, identify problems and conflicts before they happen, install packages in parallel, etc. Astral has an excellent track record on the engineering and design side, so I expect that whatever they do in this space will basically make sense, it will eventually be codified in a PEP, and PyPI will implement the same endpoint so that other tools like pip and poetry can adopt it. [1]: Top-level: https://pypi.org/simple/ https://pypi.org/simple/ Individual package: https://pypi.org/simple/pyyaml/ https://pypi.org/simple/pyyaml/
    • zahlman 1y ago
      Your information is out of date. > there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info) Not true for wheels; PyPI implements https://peps.python.org/pep-0658/ https://peps.python.org/pep-0658/ here. You can pre-download just the dependency info instead. For sdists, this is impossible until we can drop support for a bunch of older packages that don't follow modern standards (which is to say, including the actual "built" metadata as a PKG-INFO file, and having that file include static data for at least name, version and dependencies). I'm told there are real-world projects out there for which this is currently impossible, because the dependencies... depend on things that can't be known without inspecting the end user's environment. At any rate, this isn't a PyPI problem. > there's no way to get hashes of the archives This is provided as a URL fragment on the URLs, as described in https://peps.python.org/pep-0503/ https://peps.python.org/pep-0503/. Per PEP 658, the hash for the corresponding metadata files is provided in the data-dist-info-metadata (and data-core-metadata) attributes of the links. But yes, there is no reverse-search support.
• forrestthewoods 1y ago
  Neat. uv is spectacular. But I don’t get it. How does it work? Why is it able to solve the Python runtime dependency problem? I thought uv had kinda already solved that? Why is a new thingy majig needed?
  • simonw 1y ago
    This is a server hosted SaaS product that acts as a private registry for your company's Python packages. uv becomes a client that can install those packages from your private registry. I imagine pip will be able to install them from a PYX registry too.
  • zahlman 1y ago
    > Why is it able to solve the Python runtime dependency problem? I thought uv had kinda already solved that? The dependencies in question are compiled C code that Python interfaces with. Handling dependencies for a graph of packages that are all implemented in pure Python, is trivial. C never really solved all the ABI issues and especially for GPU stuff you end up having to link against very specific details of the local architecture. Not all of these can be adequately expressed in the current Python package metadata system. Aside from that, a lot of people would like to have packages that use pre-installed dependencies that came with the system, but the package metadata isn't designed to express those dependencies, and you're also on your own for actually figuring out where they are at runtime, even if you take it on blind faith that the user separately installed them. This is not really my wheelhouse, though; you'll find a much better write-up at https://pypackaging-native.github.io/ https://pypackaging-native.github.io/ .
    • forrestthewoods 1y ago
      > especially for GPU stuff you end up having to link against very specific details of the local architecture. Hrm. This doesn’t sound right to me. Any project should target a particular version of Cuda and then the runtime machine simply needs to have that version available. Right? > a lot of people would like to have packages that use pre-installed dependencies that came with the system Those people are wrong. Everything these days requires Docker because it’s the only way to deploy software that can reliable not crash on startup. (This is almost entirely a Linux self induced problem)
• Animats 1y ago
  [flagged]
  • TOGoS 1y ago
    I'm a fan of Deno's deno run http://uri.of.the/dang.program.ts
    • wraptile 1y ago
      smells like piping to bash
      • TOGoS 1y ago
        Same "easy to run" benefit, but Deno has the advantage that it caches everything and won't mess screw your computer unless you `--allow-write` or whatever.
• ossusermivami 1y ago
  i wonder if nix has been considered
• lardbgard 1y ago
  [flagged]
  • tomhow 1y ago
    Please stop posting like this. HN is not for self-promotion; it's for sharing and discussing interesting topics and projects. We ban people who post like this continually.
• nilslindemann 1y ago
  I do not trust Astral. Much ad language. They do not explain what an installation of their software does to my system. They use the word "platform".
  • wtallis 1y ago
    It's hosted SaaS; you don't install it on your system.
    • nilslindemann 1y ago
      Now I trust them even less. [1] [1] https://rwsdk.com/blog/saas-is-just-vendor-lock-in-with-better-branding https://rwsdk.com/blog/saas-is-just-vendor-lock-in-with-bett...
• FergusArgyll 1y ago
  Astral is the coolest startup
• tomwphillips 1y ago
  >Modern I'll pass. I'd rather have the battle-tested old thing, thanks.
  • LtWorf 1y ago
    I've been limiting myself to whatever is available on debian and it's been fine for me since several years. I don't understand why people who don't do weird AI stuff would use any of that instead of sticking to distribution packages and having the occasional 1 or 2 external modules that aren't packaged.
    • tomwphillips 1y ago
      Indeed, to expand on my remark: I wrote Python in academia for ~6 years and then professionally for nearly a decade in data science, data engineering and backend web apps. Virtualenv was fine. Pipenv had a nicer CLI and easier to use dependency pinning. But fundamentally all this stuff worked fine.
    • zahlman 1y ago
      Because making external modules cooperate with the system environment is awkward at best (and explicitly safeguarded against since 3.11, since it can cause serious problems otherwise even with "user" installs), and installing the distro's packages in a separate environment is not supported as far as I can tell. And because the system environment is often deliberately crippled; it may not even include the entire standard library.
      • LtWorf 1y ago
        > the system environment is often deliberately crippled; it may not even include the entire standard library. well getting a good system is step 1
  • benatkin 1y ago
    uv is battle tested It's the default installation path for aider, for instance.
• runningmike 1y ago
  All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.
  • tempest_ 1y ago
    I share your concern but I have saved so much time with uv already that I figure ill ride it till the VC enshitification kills the host. Hopefully at the point the community is centralized enough to move in one direction.
    • deleted 1y ago
      [deleted]
    • NegativeLatency 1y ago
      I've been heartened by the progress that opentofu has made, so I think if it gets enough momentum it could survive the inevitable money grab
    • anitil 1y ago
      I agree, now I just use uv and forget about it. It does use up a fair bit of disk, but disk is cheap and the bootstrapping time reduction makes working with python a pleasure again
      • nemosaltat 1y ago
        Couldn’t agree more and the `uv run executable.sh` that contains a shebang, imports and then python is just magical.
        • frickinLasers 1y ago
          Is that much different than the python inline script format? https://peps.python.org/pep-0723/ https://peps.python.org/pep-0723/
• rtpg 1y ago
  I really want to pay someone money to run package repo mirrors for me, but my problems have been more with npm than with Pypi. Astral, if you're listening.... maybe tackle JS packaging too?
• kristoff_it 1y ago
  Is this going to solve the combinatorial explosion of pre-building native dependencies for every possible target? Python should get rid of its training wheels :^) https://kristoff.it/blog/python-training-wheels/ https://kristoff.it/blog/python-training-wheels/
  • zahlman 1y ago
    > Emphasis mine. It would indeed be hard to survive without that kind of support from a corporation. A user on HN estimated the yearly cost of this traffic at around 12 million USD/year (according to AWS Cloudfront rates), more than four times the full operating budget of the Python Software Foundation as of 2024. (As the user in question: three times.) > Leverage progress in the systems programming ecosystem to create repeatable builds. Turn prebuilt binaries from “sources” into cacheable artifacts that can be deleted and reconstructed at will. Institute a way of creating secondary caches that can start shouldering some of the workload. This doesn't avoid the need for the wheels to exist and be publicly available. People running CI systems should figure out local caching that actually works, sure. But if you delete that cacheable artifact on the public PyPI website for something like, I don't know, numpy-2.3.2-cp312-cp312-win_arm64.whl, you're going to be re-creating it (and having it downloaded again) constantly. Windows users are just not going to be able to build that locally. And you know, space usage isn't the problem here — we're talking about a few hard drives' worth of space. The number of downloads is the problem. Outside of CI, I guess that's mostly driven by end users defaulting to the latest version of everything, every time they make a new virtual environment, rather than using whatever's in their package installer's cache. I do know that uv makes the latter a lot easier.
    • physicsguy 1y ago
      > Windows users are just not going to be able to build that locally. I think that's being too kind to Linux + Mac users, they are often not generally able to do it either in my experience, especially with Python stuff which tends to need more bleeding edge deps than the local package manager can supply.
• pshirshov 1y ago
  Again! ezsetup, setuptools, conda, poetry, uv, now this.
  • thrown-0825 1y ago
    Yeah but this one has VC backing and a business model!
• jacques_chester 1y ago
  Interesting watching this part of the landscape heating up. For repos you've got stalwarts like Artifactory and Nexus, with upstart Cloudsmith. For libraries you've got the OG ActiveState, Chainguard Libraries and, until someone is distracted by a shiny next week, Google Assured Open Source. Sounds like Pyx is trying to do a bit of both. Disclosure: I have interacted a bunch with folks from all of these things. Never worked for or been paid by, though.
• hommes-r 1y ago
  Only thing that is unclear to me is to which extend this setup depends on the package publisher. PyPi might be terrible at least it just works when you want to publish that it leads to more complexity for the ones that are looking to use this piece of free software is not for the maintainer. Maybe they are only targeting dev tooling companies as a way to simplify how they distribute. Especially in the accelerated compute era.
• lvl155 1y ago
  I don’t know how I feel about one company dominating this space. I love what they do but what happens 5 years down the road?
  • xyst 1y ago
    Sold to Broadcom, probably
• no_wizard 1y ago
  Good on you guys! I wanted to start a business exactly like this years ago, when I actually worked in Python. I ended up not doing so, because at the time (circa 2014-2015) I was told it would never take off, no way to get funding. I'm glad you're able to do what ultimately I was not!
• Aeolun 1y ago
  Yay, _another_, probably incompattible, python package manager has arrived.
• NeutralForest 1y ago
  Cool idea! I think I could benefit from this at my job if they're able to eat Anaconda's lunch and provide secure, self-hosted artifacts.
• nixpulvis 1y ago
  Can I ask a dumb question. Why does Ruby (for example) not have this problem, but python still can't ship a standard solution which isn't constantly changing and rolled up in some corporate offering?
  • semiquaver 1y ago
    Before you answer that you have to answer what problem this is solving that PyPI doesn’t already address. uv works great against “legacy” package indexes so I’m not really clear why it’s needed other than to introduce lock-in to a for-profit facility.
  • nhumrich 1y ago
    GPU/c-bindings. Python packaging for for Python only modules has never been a problem. When people say they hate python packaging they are usually talking about being able to successfully install dependencies without much thinking. But, the biggest reason that doesn't work is because of the dependencies that have to be compiled. Which brings it's own problems. Have you ever had a c dependency on node or ruby on a system that wasn't the same system they built it with? Turns out it sucks in all the languages. It's just that the amount of c-level packages in python is quite larger than say ruby. The likelihood of a problem is significantly larger. Especially in the GPU accelerated space.
    • procaryote 1y ago
      Python packaging for pure python has definitely been a problem with competing methods and limitations. Venv hasn't always been around for example, and before it was, dependency conflicts between two things running on the same machine were hairy
  • bgwalter 1y ago
    Because CPython and PyPA are dysfunctional organizations in the hands of people who are in the right (often corporate) cliques. Don't expect anything from there.
  • int_19h 1y ago
• jsmeaton 1y ago
  Astral folks that are around - there seems to be a bit of confusion in the product page that the blog post makes a little more clear. > The next step in Python packaging The headline is the confusing bit I think - "oh no, another tool already?" IMO you should lean into stating this is going to be a paid product (answering how you plan to make money and become sustainable), and highlight that this will help solve private packaging problems. I'm excited by this announcement by the way. Setting up scalable private python registries is a huge pain. Looking forward to it!
  • zanie 1y ago
    Thanks for the feedback!
    • IshKebab 1y ago
      I would also put this list of issues that this fixes higher. It makes it more obvious what the point is. (And also a setuptools update literally broke our company CI last week so I was like "omg yes" at that point.)
    • divbzero 1y ago
      The combination of – “client (uv) and server (pyx)” and – “You can use it to host your own internal packages, or as an accelerated, configurable frontend to public sources like PyPI and the PyTorch index.” is what really helped me understand what pyx aims to be.
• sigmonsays 1y ago
  this is so comical, entirely https://xkcd.com/927/ https://xkcd.com/927/ python has burned me with it's packaging so many times.
• russellbeattie 1y ago
  What are the reasons that Python can't implement the same sort of module/packaging system as NodeJS? That seems to work well enough. Executing a Python script in the same directory as some sort of project.json file that contains all the complicated dependency details would be a pretty good solution to me. But I'm probably missing a whole bunch of details. (Feel free to educate me). In general I really dislike the current system of having to use new environment variables in a new session in order to isolate Py scripts. It has always seemed like a hack with lots of footguns. Especially if you forget which console is open.
  • kouteiheika 1y ago
    It can. That's what uv is. Put an '#!/usr/bin/env -S uv run python' shebang in your script, add a `pyproject.toml` with all of your deps, and you're done.
    • dwattttt 1y ago
      > add a `pyproject.toml` with all of your deps Or put your dependencies inline in the script (https://packaging.python.org/en/latest/specifications/inline-script-metadata/ https://packaging.python.org/en/latest/specifications/inline...), and it's just a single file script again. But with dependency support EDIT to clarify it was just the pyproject.toml that inline metadata could replace
    • russellbeattie 1y ago
      Again, you're creating a new session with special environmental variables. It's nice that it uses pyproject.toml (well, except for the toml part), but it still needs uv. I was suggesting that the Python executable have this sort of functionality built in, with no need for a new session.
• zeckalpha 1y ago
  > There should be one-- and preferably only one --obvious way to do it.
  • pmarreck 1y ago
    This is not only the main reason Python turned me off, they can't even stick to it with their umpteen package managers, which is hypocrisy
  • geophph 1y ago
    It’s been pointed out to me before that: the two ways of spacing the em dash on that quote is a joke about how it's not actually possible to do that? (And there's a third way in another line of the zen) Do I buy it? Not sure. But apparently there’s more to this line than what it suggests.
• safetytrick 1y ago
  Python packaging is the least zen of python thing about python.
• deleted 1y ago
  [deleted]
• kccqzy 1y ago
  I hate that they are using the pyx name; it's the extension for Cython files. It's going to cause at least a moment of confusion for people. They could have easily checked for name collision in the Python ecosystem but they chose not to do that; that's like a middle finger gesture to the community.
• sdotdev 1y ago
  When this releases it will be crazy, Ive always wondered why something like this didn't already exist. Really useful concept especially for school.
• CivBase 1y ago
  I feel like I must be the crazy one for never having a problem with just vanilla pip, PyPi, and venv (or virtualenv for old Python 2 stuff). Maybe it's just my use case?
• twarge 1y ago
  The real pyx is an absolutely wonderful graphing package. It's like Tex in that everything looks wonderful and publication-quality. https://pyx-project.org/gallery/graph/index.html https://pyx-project.org/gallery/graph/index.html
  • almostgotcaught 1y ago
    there's something about these comments ("name-collision") that drives me up the wall. do y'all realize multiple things can have the same name? for example, did you know there are many people with exactly the same names: https://www.buzzfeed.com/kristenharris1/famous-people-same-name https://www.buzzfeed.com/kristenharris1/famous-people-same-n... and yet no one bemoans this (hospitals don't consult name registries before filling out birth certificates). that's because it's almost always extremely clear from context. > The real pyx what makes that pyx any more "real" than this pyx? it's the extension of the language py plus a single letter. there are probably a thousand projects that could rightfully use that combination of letters as a name.
    • twarge 1y ago
      it's the pyx you get with `pip install pyx`?
    • iepathos 1y ago
      Human naming has nothing to do with software naming which seems obvious but apparently not. Python package creators should check the pypi registry for names and generally avoid name collisions where reasonable. Common sense applies for reduced confusion for users globally and also for potential legal issues if any party trademarks their software name. What makes one pyx more real than the other is one was first and took the spot on pypi. Simple as that. https://pypi.org/project/PyX/ https://pypi.org/project/PyX/
      • almostgotcaught 1y ago
        > https://pypi.org/project/PyX/ https://pypi.org/project/PyX/ the last release is Oct 16, 2022. are we doing this like jerseys - the name is now retired because pyx won all the championships?
• brettpro 1y ago
  Welp, I guess it's time to start pulling all the uv deps out of our builds and enjoy the extra 5 minutes of calm per deploy. I'm not gonna do another VC-fueled supply chain poisoning switcheroo under duress of someone else's time crunch to start churning profit.
• __loam 1y ago
  Python devs need to find God
  • jjgreen 1y ago
    [300 comment angry discussion of which God]
• gnubee 1y ago
  Isn't all of this solved with containers (or virtual machines)? Just lxc/lxd/docker a new container, and no more worries about virtual environments or conda or whatever other bandaids. A container per project. A container per version of a project. Heck, a container per package in a project!
• rcleveng 1y ago
  Just try using ux with Google Artifact Registry and AWS CodeArtifact. You need the username in the URL with GAR and you need it in the environment variable with AWS. I'm not sure who's causing the pain with using uv with private repositories (but I'd bet it's uv), but they are selling a solution to that pain. Full disclosure, uv rocks and is way better than poetry, setuptools and whatever complicated and non-zen way packaging python has had in the past.
  • ohmahjong 1y ago
    In my experience (albeit from about 4 years ago) having some kind of private package with pipenv (or even plain pip) was really annoying. We went the route of a private repo on Github rather than a private package repository, which maybe set us up for exactly that annoyance.
    • aragilar 1y ago
      I find this strange, I always have private repositories using devpi (which also acts as a caching mirror) and it's easy to configure pip.
  • anentropic 1y ago
    any kind of AWS authentication is always a pain in the neck
• enmyj 1y ago
  using mypy regularly on a large project is pretty slow and painful, so I personally welcome any improvements no matter the origin!!
• kergonath 1y ago
  Sweet. Yet another Python packaging silver bullet. I can’t wait to ignore this one while I am waiting for next month’s.
• sussmannbaka 1y ago
  It’s the year 21XX. Another HN story about python packaging being solved for good this time hits the front page. I try to use a python app, am impressed by its capability to generate 10 pages of red text and go back to cryosleep.
• deleted 1y ago
  [deleted]
• renewiltord 1y ago
  `uv` is incredible software and I love it. I am willing to try anything from this team.
• chickenzzzzu 1y ago
  This industry is fucking doomed.
• dirkc 1y ago
  My 2c when it comes to python packaging: 1. do dev in an isolated environment like docker 2. install deps from config - ie. build a new image when adding a dep
• pjmlp 1y ago
  How many Python packaging solutions do we need? Thankfully I only use it for OS and application scripting and keep myself to the standard library and application exposed APIs as much as possible, even if there are marvelous alternatives outside the standard library.
• nprateem 1y ago
  An amazing number of words that basically say nothing
• rmonvfer 1y ago
  To be honest, this was just a matter of time. As a long time Python developer, I just can’t wrap my head around the lack of something like this. GitHub was going to get hosted packages for Python but never did because it “didn’t align with their strategy objectives and a reallocation of resources” [1] (or some other similar corpospeak) Astral is a great company and I think we can’t question what they’ve achieved and provided to the Python community. uv is a game changer and solves one of the core issues with Python by providing a unified tool that’s also fast, reliable and easy to use. In fact, after using uv for the first time (coming from a combination of pyenv + poetry) I never wanted to go back and this is something all of my peers have experienced too. I’m glad it’s Astral who is doing this, and of course they will have to make money one way or another (which is perfectly fine and I don’t think anyone on this forum can be against that, as long as they are actually providing real value) but I was honestly tired of the paralysis on this matter. I did try to build a registry (pyhub.net) but being just one person with almost no resources and having another full time business made it impossible. Anyway, congrats to the team for the effort! [1] https://github.com/orgs/community/discussions/8542 https://github.com/orgs/community/discussions/8542
  • hexo 1y ago
    to be honest. ill never use uv. python ecosystem tools should be in python.
    • cma256 1y ago
      Wait until you find out what language Python is written in.
      • hexo 1y ago
        Last time I looked it was in C.
    • LucasOe 1y ago
      Why?
    • jpambrun 1y ago
• alexshagov 1y ago
  I personally use Nixpkgs to fully isolate python installation on per project basis. And inside this isolated env, once python is built, usually using pip is enough and works well This way, !each of my repositories has its own nix file defining the list of dependencies and it can easily be built per system as well
  • codethief 1y ago
    I love Nix as much as the next guy, but I also need my coworkers to be able to get things to work on their non-NixOS machines. Have you found an easy way to have Nix install packages through uv (from a pyproject.toml / uv.lock files)? Right now I just do programs.nix-ld.enable = true; and use uv completely separately from Nix.
    • sporeray 1y ago
      uv2nix is great from my experience https://github.com/pyproject-nix/uv2nix https://github.com/pyproject-nix/uv2nix
      • codethief 1y ago
        Thanks! Is there a simple example of how to use it? I have found their docs with those gigantic code examples impenetrable.
• jones89176 1y ago
  Obligatory xkcd: https://xkcd.com/1987/ https://xkcd.com/1987/ (I've seen https://xkcd.com/927/ https://xkcd.com/927/ in the comments a few times, but apparently I'm the first one who had this one in mind)
• sbinnee 1y ago
  I think it’s a good monetization strategy for astral team. I just hope that they do not lock uv only on pyx and that uv seamlessly works with pypi.
• CloseChoice 1y ago
  Does anyone have insights how this compares to anaconda's approach? To me both seem very similar, ux <-> conda, pyx <-> conda-forge. Sure, astral's products are remarkable and widely loved, but I would like to understand if there's a USP beyond that?
  • rockmeamedee 1y ago
    Yeah, uv is cool and a step above conda, but that business model doesn’t look very profitable… They did say they want their thing to have understanding of the code, so maybe they’ll sell semgrep-like features and SBOM/compliance on top. Semgrep is ok popular, but if it maybe bundled into something else (like the package registry itself) that might get enough people over the line to buy it. Private registries and “supply chain security” tools individually aren’t the hottest market, but maybe together the bundle could provide enough value. Let’s see how it goes.
• michaelcampbell 1y ago
  Who's going to write YAPP, yet another python packager? (Although I see the name is already taken for a different type of project.)
• insane_dreamer 1y ago
  I have no connection to astral, and while we dabble in uv (which has been great) we mostly use conda because of CUDA linking etc, but an alternative is at the least welcome and at best an improvement, so I don’t get the degree of cynicism or even hostility on this thread. If you don’t want to use pyx stick with pypi or write your own damn python packaging server.
• danluvscock 1y ago
  [dead]
• bandrami 1y ago
  Oh my God how many times is Python going to do this?