4 ms·

This might be off-topic, but time to chime-in with my experience of packaging an open source software written in python. Once, I spent a good deal of my time m

This might be off-topic, but time to chime-in with my experience of packaging an open source software written in python. Once, I spent a good deal of my time making a grammar-based phonetic transliterator engine for an Indic language. I was mostly a python developer with some prior experience in C and plain Make files. After a couple of rewrites, I was mostly happy with the results. Then was the time for packaging and making it a usable software. The application needed to install some files under root so that linux's input subsystem (IBus) could recognize them. Also, it dependent on some system packages(both python based and non-python based) like dictionaries for generating suggestions. As I approached the problem, I learned that I had to learn autotools and a bunch of very debian specific scripts which were pretty much focused on packaging applications written in C. It was like an arcane magic. I would probably had to produce & package at least 10 software to make the return of investment on that knowledge. I also tried to use python's setuptools, which felt equally dirty without the benefit of generating a proper .deb package. The impedance mismatch of the whole packaging architecture and dynamic language ecosystem was soul crushing. In the end, I wrote a python script which dumps the project in /opt and installs/uninstalls the system files and called it a day. The end result was that user adoption of my software was very low. One reason for that might be the unusual way of installation(download a tar file, unzip and run a installer from command line).

dredmorbius 8y ago
Did you consult the NMG or ask for assistance? https://www.debian.org/doc/manuals/maint-guide/ https://www.debian.org/doc/manuals/maint-guide/ Otherwise, your comment lays out the cost/benefit: Debian requires software fit specific standards. The reward is an unparallelled distribution and maintenance system. Note too: there's plenty of python-based software in Debian. https://www.debian.org/doc/packaging-manuals/python-policy/python.html https://www.debian.org/doc/packaging-manuals/python-policy/p...
- titanix88 8y ago
  Yes, I did try to comprehend the NMG, but didn't ask for anyones help. But AFAIR, back then (mid 2015), the python specific parts of the packaging manuals were scattered between some presentations and loose documentation here and there. Thanks for the link though. I might get back to it. Still, I believe that a robust system like debian could be achieved by more simpler means. It's packaging system is an invention of a different era and needs more streamlining to accommodate dynamic language developers. Python was invented in 90's and it's took 20+ years for debian to setup some serious guidelines for python developers. What if I want to package a software written in another dynamic language which was invented a month ago? These questions need to be addressed. Disclaimer: I am not claiming that I could device a solution. Just that, despite my best intentions and best of efforts, I couldn't handle the difficulty of packaging.
  - mmt 8y ago
    > Python was invented in 90's and it's took 20+ years for debian to setup some serious guidelines for python developers. I've only ever packaged in-house (within a company I'm working for) Python apps, very simple opern-source apps and libraries and modified existing packaging for new versions of such open source code that otherwise already had available packaging [1], but my experience has been different to the point that I disagree [2]. Since I'm a sysadmin and not a developer, my strategy was not just to use the documentation, but also to look at how other packages handled certain situations, especially if I knew of an app that was relatively complicated in terms of build process. For Python, that was mostly a challenge of unravelling each developer's build/install instructions (easy_install/setup.py/pip) into something that matched an existing debianized Python package. > Still, I believe that a robust system like debian could be achieved by more simpler means. It's packaging system is an invention of a different era and needs more streamlining to accommodate dynamic language developers. As a sysadmin, I object to the characterization of the debian package system as being "of a different era", mostly because I don't believe there's anything about this era, nor about dynamic languages (which have been around for a very long time anyway) that has eliminated the needs that it addresses. That robustness is achieved because of the requirement of up-front effort to conform to a standard that has remained stable across eras. > What if I want to package a software written in another dynamic language which was invented a month ago? Same as for every other language since the beginning of time: put in the effort (including asking for help from the community) and conform. If you need a language-specific standard (such as the OS-default for egg-equivalents), then make one up (consistent with other languages) and get your new language's community behind it, if that's possible. Of course, the other option is just to exist exclusively within that language's ecosystem (e.g. pip, gem, cpan) and wait for other interested parties to do the OS-level packaging for you. That happens, too, but it can slow adoption if your "market" is production environments and not just other devs in the ecosystem. [1] Although sometimes with inadequate/inaccurate "debianization" (e.g. missing expicitly listing a depency) [2] I do agree that the documentation isn't necessarily organized in the best fashion, nor that the latest info is in one place, but it's out there. There might also not be One True Way, but it's arguable if that's a bad thing.
stcredzero 8y ago
One reason for that might be the unusual way of installation(download a tar file, unzip and run a installer from command line). That's not too far off from how I install new versions of golang. Just leave out the last step and add a 0th step where you delete the old version first.

dredmorbius 8y ago

Did you consult the NMG or ask for assistance? https://www.debian.org/doc/manuals/maint-guide/ https://www.debian.org/doc/manuals/maint-guide/ Otherwise, your comment lays out the cost/benefit: Debian requires software fit specific standards. The reward is an unparallelled distribution and maintenance system. Note too: there's plenty of python-based software in Debian. https://www.debian.org/doc/packaging-manuals/python-policy/python.html https://www.debian.org/doc/packaging-manuals/python-policy/p...

titanix88 8y ago
Yes, I did try to comprehend the NMG, but didn't ask for anyones help. But AFAIR, back then (mid 2015), the python specific parts of the packaging manuals were scattered between some presentations and loose documentation here and there. Thanks for the link though. I might get back to it. Still, I believe that a robust system like debian could be achieved by more simpler means. It's packaging system is an invention of a different era and needs more streamlining to accommodate dynamic language developers. Python was invented in 90's and it's took 20+ years for debian to setup some serious guidelines for python developers. What if I want to package a software written in another dynamic language which was invented a month ago? These questions need to be addressed. Disclaimer: I am not claiming that I could device a solution. Just that, despite my best intentions and best of efforts, I couldn't handle the difficulty of packaging.
- mmt 8y ago
  > Python was invented in 90's and it's took 20+ years for debian to setup some serious guidelines for python developers. I've only ever packaged in-house (within a company I'm working for) Python apps, very simple opern-source apps and libraries and modified existing packaging for new versions of such open source code that otherwise already had available packaging [1], but my experience has been different to the point that I disagree [2]. Since I'm a sysadmin and not a developer, my strategy was not just to use the documentation, but also to look at how other packages handled certain situations, especially if I knew of an app that was relatively complicated in terms of build process. For Python, that was mostly a challenge of unravelling each developer's build/install instructions (easy_install/setup.py/pip) into something that matched an existing debianized Python package. > Still, I believe that a robust system like debian could be achieved by more simpler means. It's packaging system is an invention of a different era and needs more streamlining to accommodate dynamic language developers. As a sysadmin, I object to the characterization of the debian package system as being "of a different era", mostly because I don't believe there's anything about this era, nor about dynamic languages (which have been around for a very long time anyway) that has eliminated the needs that it addresses. That robustness is achieved because of the requirement of up-front effort to conform to a standard that has remained stable across eras. > What if I want to package a software written in another dynamic language which was invented a month ago? Same as for every other language since the beginning of time: put in the effort (including asking for help from the community) and conform. If you need a language-specific standard (such as the OS-default for egg-equivalents), then make one up (consistent with other languages) and get your new language's community behind it, if that's possible. Of course, the other option is just to exist exclusively within that language's ecosystem (e.g. pip, gem, cpan) and wait for other interested parties to do the OS-level packaging for you. That happens, too, but it can slow adoption if your "market" is production environments and not just other devs in the ecosystem. [1] Although sometimes with inadequate/inaccurate "debianization" (e.g. missing expicitly listing a depency) [2] I do agree that the documentation isn't necessarily organized in the best fashion, nor that the latest info is in one place, but it's out there. There might also not be One True Way, but it's arguable if that's a bad thing.

stcredzero 8y ago

One reason for that might be the unusual way of installation(download a tar file, unzip and run a installer from command line). That's not too far off from how I install new versions of golang. Just leave out the last step and add a 0th step where you delete the old version first.