5 ms·

PyOCI – Publish and install private Python packages using OCI/Docker registries

29 points by robinhoodexe 1y ago

globular-toast 1y ago
Or you could use GitLab which has support for Python packages without a proxy.
tuananh 1y ago
can we agree to use OCI for everything :D
- deknos 1y ago
  as long as we can convert the OCI to an bootable VM image, i am fine with that. But i also think, there's an size limit
  - yjftsjthsd-h 1y ago
    There are still growing pains, but https://github.com/osbuild/bootc-image-builder https://github.com/osbuild/bootc-image-builder exists and is likely to become exactly that in the general case (as it already is for the redhat family).
  - tetha 1y ago
    Oh those size limits are pushed plenty by AI images, no worries. I recently had a good laugh when I found a docker image that was 2 - 3 times as big as the OS partition of a lot of our smaller servers. And our OS image build order would reuse layers better than those.
    - bravetraveler 1y ago
      No doubt, I've regularly encountered ~2TB container images with enough layers to make one weep. SISO, slop in/slop out (sorry).
      - grepfru_it 1y ago
        Time to find out if one can make a dockerbomb image >:-)
        bravetraveler 1y ago
        I believe in you, 'fallocate' can be put into entrypoint :P This way the size is a surprise, not constant
  - tuananh 1y ago
    i think it's already been done with bootable containers. redhat has recently GA bootable container as well.
  - deleted 1y ago
    [deleted]
yjftsjthsd-h 1y ago
Would be nice to see at least a high-level overview of how it works under the hood. Is it doing anything interesting with reusable layers? Actually, thinking further that might be a moot point; I feel like running as a remote proxy loses some fun optimization chances. I could imagine a world where you install a package from the host's pip/uv, and then add it to a container image, and both of those are actually the same thing on disk. (Granted, that's likely harder to implement)
- allexv 1y ago
  Hi, I included a few details here: https://github.com/AllexVeldman/pyoci/blob/main/docs/design.md https://github.com/AllexVeldman/pyoci/blob/main/docs/design.... which I recognise does not contain much. It's basically a single layer image with a multi-platform image index per tag(version) on top. No smarts added, performance wise it's probably worse than running your own dedicated pypi index or any purpose-built system, my main goal is a private index for if you don't have access to one already.
  - yjftsjthsd-h 1y ago
    (Thanks for replying!) That's fair. I'm used to thinking of OCI images as being a particular optimization (there's a reason we don't ship containers around ass rootfs tarballs), but for your usecase it may well be a premature/pointless optimization.
allexv 1y ago
Thank you for posting my project here! Only noticed it just now due to the uptick in starts on github :)

29 points by robinhoodexe 1y ago

globular-toast 1y ago

Or you could use GitLab which has support for Python packages without a proxy.

tuananh 1y ago

can we agree to use OCI for everything :D

deknos 1y ago
as long as we can convert the OCI to an bootable VM image, i am fine with that. But i also think, there's an size limit
- yjftsjthsd-h 1y ago
  There are still growing pains, but https://github.com/osbuild/bootc-image-builder https://github.com/osbuild/bootc-image-builder exists and is likely to become exactly that in the general case (as it already is for the redhat family).
- tetha 1y ago
  Oh those size limits are pushed plenty by AI images, no worries. I recently had a good laugh when I found a docker image that was 2 - 3 times as big as the OS partition of a lot of our smaller servers. And our OS image build order would reuse layers better than those.
  - bravetraveler 1y ago
    No doubt, I've regularly encountered ~2TB container images with enough layers to make one weep. SISO, slop in/slop out (sorry).
    - grepfru_it 1y ago
      Time to find out if one can make a dockerbomb image >:-)
      - bravetraveler 1y ago
        I believe in you, 'fallocate' can be put into entrypoint :P This way the size is a surprise, not constant
- tuananh 1y ago
  i think it's already been done with bootable containers. redhat has recently GA bootable container as well.
- deleted 1y ago
  [deleted]

yjftsjthsd-h 1y ago

Would be nice to see at least a high-level overview of how it works under the hood. Is it doing anything interesting with reusable layers? Actually, thinking further that might be a moot point; I feel like running as a remote proxy loses some fun optimization chances. I could imagine a world where you install a package from the host's pip/uv, and then add it to a container image, and both of those are actually the same thing on disk. (Granted, that's likely harder to implement)

allexv 1y ago
Hi, I included a few details here: https://github.com/AllexVeldman/pyoci/blob/main/docs/design.md https://github.com/AllexVeldman/pyoci/blob/main/docs/design.... which I recognise does not contain much. It's basically a single layer image with a multi-platform image index per tag(version) on top. No smarts added, performance wise it's probably worse than running your own dedicated pypi index or any purpose-built system, my main goal is a private index for if you don't have access to one already.
- yjftsjthsd-h 1y ago
  (Thanks for replying!) That's fair. I'm used to thinking of OCI images as being a particular optimization (there's a reason we don't ship containers around ass rootfs tarballs), but for your usecase it may well be a premature/pointless optimization.

allexv 1y ago

Thank you for posting my project here! Only noticed it just now due to the uptick in starts on github :)