4 ms·

Is it possible to publish Python code for others to use simply by putting a repo on a webserver? In other words, can someone put a line like this into their re

by TekMol 3y ago

Is it possible to publish Python code for others to use simply by putting a repo on a webserver? In other words, can someone put a line like this into their requirements.txt file? https://somewhere.com/some/git/repo.git https://somewhere.com/some/git/repo.git

Znafon 3y ago
Yes, you can do this: https://pip.pypa.io/en/stable/topics/vcs-support/ https://pip.pypa.io/en/stable/topics/vcs-support/
- TekMol 3y ago
  That is pretty awesome! Why is all the "wheels", "eggs", "PyPi" ... stuff needed then?
  - waynesonfire 3y ago
    New college grad needs something todo?
    - rvnx 3y ago
      Probably Ruby owns the award for the worst package names. Can you guess what Sorbet is doing in Ruby ? Or Oniguruma ? Or Ahoy ? Or Figaro ?
  - woodrowbarlow 3y ago
    a lot of Python packages contain parts written in native code (often C or Rust) for performance reasons. pip _can_ compile the code at install-time if your PC has the right build tools -- or it can download pre-built binaries for your host in the form of a "wheel". pypi has a little more metadata than a git repo, and a place to host version-controlled binaries.
  - aniforprez 3y ago
    Wheels are essentially zip files with all the code required to run it contained in one handy little file for all platforms that the package maintainer chooses to build for. Instead of having to download the source of the packages and building on your machines, you have a cache-able neat little file that contains potentially everything you need to use the package. It makes package installations way faster because it skips the build steps and will ship all the binaries required to run the package (ideally). Eggs were this but are not in use anymore in favor of using wheels. Pypi is the package repository for python packages just like npm
  - kobalsky 3y ago
    pypi is needed because those personal webservers will inevitably go down. why not github? well, it didn’t use to exist before pypi and it microsoft may still pull a good one eggs and wheels solve binary distribution between other things, so you can install something that requires compilation without having a compiler or the development libraries installed and to avoid bundling files unnecessary during runtime.
  - matsemann 3y ago
    Because so much Python code binds to C extensions that you will need to compile for the correct platform. For instance the new version of Pydantic is now backed by Rust. To avoid everyone having to compile the rust code on their platform, you can upload it pre-compiled for linux, windows etc.
    - fmajid 3y ago
      Some packages require build-time dependencies not everyone has, like cryptography now requiring Rust. Some packages like SciPy take a very long time to build, and require a Fortran compiler.
deleted 3y ago
[deleted]
Hasnep 3y ago
Yes, you just need to add `git+` before the URL, see the pip documentation: https://pip.pypa.io/en/stable/topics/vcs-support/ https://pip.pypa.io/en/stable/topics/vcs-support/
mathisfun123 3y ago
Yes https://github.com/iree-org/iree-samples/blob/main/requirements.txt#L3 https://github.com/iree-org/iree-samples/blob/main/requireme... You can do even better using GitHub releases: the GET for a releases page with assets (ie use inspector) functions as an index url. So you can have a repo that publishes wheels to a release called eg `latest` that can function as the index for that package/wheel. You can even have multiple repos that publish to a single such wheel repo release repo (using the ncipollo/release-action GHA).
- TekMol 3y ago
  I don't know what "GitHub releases" are. And I don't want to know :) I prefer to use as few technologies as possible. And completely stay away from propriatary technologies.
  - mathisfun123 3y ago
    > I prefer to use as few technologies as possible. And completely stay away from propriatary technologies. This is the most boring and tedious archetype out there.
whalesalad 3y ago
you can install a package straight from Github: pip install https://github.com/whalesalad/verisign-namestudio/archive/refs/heads/master.zip or from the git repo: pip install git+https://github.com/whalesalad/verisign-namestudio.git

by TekMol 3y ago

Znafon 3y ago

Yes, you can do this: https://pip.pypa.io/en/stable/topics/vcs-support/ https://pip.pypa.io/en/stable/topics/vcs-support/

TekMol 3y ago
That is pretty awesome! Why is all the "wheels", "eggs", "PyPi" ... stuff needed then?
- waynesonfire 3y ago
  New college grad needs something todo?
  - rvnx 3y ago
    Probably Ruby owns the award for the worst package names. Can you guess what Sorbet is doing in Ruby ? Or Oniguruma ? Or Ahoy ? Or Figaro ?
- woodrowbarlow 3y ago
  a lot of Python packages contain parts written in native code (often C or Rust) for performance reasons. pip _can_ compile the code at install-time if your PC has the right build tools -- or it can download pre-built binaries for your host in the form of a "wheel". pypi has a little more metadata than a git repo, and a place to host version-controlled binaries.
- aniforprez 3y ago
  Wheels are essentially zip files with all the code required to run it contained in one handy little file for all platforms that the package maintainer chooses to build for. Instead of having to download the source of the packages and building on your machines, you have a cache-able neat little file that contains potentially everything you need to use the package. It makes package installations way faster because it skips the build steps and will ship all the binaries required to run the package (ideally). Eggs were this but are not in use anymore in favor of using wheels. Pypi is the package repository for python packages just like npm
- kobalsky 3y ago
  pypi is needed because those personal webservers will inevitably go down. why not github? well, it didn’t use to exist before pypi and it microsoft may still pull a good one eggs and wheels solve binary distribution between other things, so you can install something that requires compilation without having a compiler or the development libraries installed and to avoid bundling files unnecessary during runtime.
- matsemann 3y ago
  Because so much Python code binds to C extensions that you will need to compile for the correct platform. For instance the new version of Pydantic is now backed by Rust. To avoid everyone having to compile the rust code on their platform, you can upload it pre-compiled for linux, windows etc.
  - fmajid 3y ago
    Some packages require build-time dependencies not everyone has, like cryptography now requiring Rust. Some packages like SciPy take a very long time to build, and require a Fortran compiler.

deleted 3y ago

[deleted]

Hasnep 3y ago

Yes, you just need to add `git+` before the URL, see the pip documentation: https://pip.pypa.io/en/stable/topics/vcs-support/ https://pip.pypa.io/en/stable/topics/vcs-support/

mathisfun123 3y ago

Yes https://github.com/iree-org/iree-samples/blob/main/requirements.txt#L3 https://github.com/iree-org/iree-samples/blob/main/requireme... You can do even better using GitHub releases: the GET for a releases page with assets (ie use inspector) functions as an index url. So you can have a repo that publishes wheels to a release called eg `latest` that can function as the index for that package/wheel. You can even have multiple repos that publish to a single such wheel repo release repo (using the ncipollo/release-action GHA).

TekMol 3y ago
I don't know what "GitHub releases" are. And I don't want to know :) I prefer to use as few technologies as possible. And completely stay away from propriatary technologies.
- mathisfun123 3y ago
  > I prefer to use as few technologies as possible. And completely stay away from propriatary technologies. This is the most boring and tedious archetype out there.

whalesalad 3y ago

you can install a package straight from Github: pip install https://github.com/whalesalad/verisign-namestudio/archive/refs/heads/master.zip or from the git repo: pip install git+https://github.com/whalesalad/verisign-namestudio.git