8 ms·
> Don't write internal cli tools in python 100%. I stopped writing anything that had to be deployed (basically everything except Jupyter notebooks for data stu
by travisd 5y ago
> Don't write internal cli tools in python
100%. I stopped writing anything that had to be deployed (basically everything except Jupyter notebooks for data stuff) in Python because it’s truly a nightmare. Go and goreleaser is great for writing a CLI (and if it’s public, it can auto generate binaries and upload to GitHub, create a Homebrew/Scoop bucket, etc)
- apple4ever 5y agoI've written tons of internal tools with Python, and I have no problem getting it deployed. Just keep them up to date with changes and always use system provided packages, and there is no issue.
- throwaway984393 5y agoFor the past 3 years I've written almost all of my users' system tools in Bash v3. Nobody has yet reported a bug to me. Runs fine on Linux, Mac, and WSL, on every architecture. Single file, easy for anyone to read and edit, gets the job done. I don't even have to care that it's unpopular because there are no installation steps. Just 'bash foo.sh'. Sometimes that means they need to download some Go app that my script uses, but that's much better than me having to write and support all that other code. If I have to do weird things with data structures or complex logic, I reach for Python, but I do it one of two ways: 1) so stupid that "python foo.py" works (no external deps), or 2) publish it to pypi so the user can just 'pip install foobar'.
- exdsq 5y agoI love bash. It's probably my favourite language for productivity. I spent some time a while back trying to write a Lisp in bash, and have written a ~1kloc performance testing tool to get results while an entire team worked on a Haskell implementation.
- raffraffraff 5y agoI'm also a huge bash fan, and agree that distributing Python tools is a mess. But "Runs fine on Linux, Mac and WSL" is a stretch. Bash on Mac OS is a ball of shit unless you're installing a newer GNU bash via homebrew. Mac OS bash is just too old. I can't even remember the list of stuff that breaks because I stopped trying. If the target audience is mostly Mac users I'll just write zsh to begin with.
- throwaway984393 5y agoI ran into those problems too initially, but I've learned to avoid them. The version of Bash on Macs is version 3.2.57 iirc. This page has documentation of which features were introduced in v3, v4 (https://tldp.org/LDP/abs/html/bash2.html https://tldp.org/LDP/abs/html/bash2.html). What I personally do is write POSIX shell, then add in things like arrays, $BASH_SOURCE, printf %q, and a few other handy features that have been around for a while. To test it, there is an official Docker image for Bash v3 (https://hub.docker.com/_/bash https://hub.docker.com/_/bash).
- andreineculau 5y agoI've also been having bash as a strict req for tool language. As you say there are 1% or less cases where you can benefit from using smth else but that's 1%! My team had js as the main language and that's what we had that 1% in. Mostly no-deps scripts but where deps where needed we could hack around npx to install the deps on-demands. I would use deno today for that actually. But! Almost nobody gets it. If you're into operations: "here we write our tool chain in Python". If you're into dev: "here we write our tool chain in js/go/php/etc/etc Bash would just be a +1" +1??? You still need to write bash, or shell in general, for orchestration of build/ci/cd pipelines. If someone thinks that in 2021 they can escape learning shell, they are out of their minds. So no, it's not a +1. Bash is the language that is not only available quite everywhere but it's also the one that allows you to write oneliner solutions https://twitter.com/andreineculau/status/1466107915450916867?s=21 https://twitter.com/andreineculau/status/1466107915450916867... and move on. As devs we should spend 10x more on solid foundations that work as many of us, for as many years to come, than we do on hip-new-thingie-that-promises-to-fix-all-my-problems. It's as if we took the regex joke and our brains could only comprehend: I'm never using a regex. Keep it simple? Never.
- pizza234 5y agoI have a similar strategy but Bash... "easy for anyone to read and edit"? Heck no. Bash is a terrible language by modern standards, with obscure syntax, many weird quirks, no type checking, no return values... How could developers program in Bash before Shellcheck!? I think that internally, I draw the line at around 100 lines. Even refactoring a program shorter than that it's not a pleasant experience (not difficult, but requires a great deal of attention, when compared to any modern language).
- throwaway984393 5y agoI wouldn't call Bash a "language" per se... Bash is a user interface, first and foremost. Bash "scripting" is literally just the running of Bash shell commands in bulk. But every language does have people making the same complaints. Every language has quirks. Every language has obscure syntax (if you haven't found it yet, you just haven't used it enough...). Every language has people say either "There's no type checking!", or "It forces type checking!". And there are absolutely return values, they're just either integers or raw data on standard output. But the point is, it's not supposed to be a "good programming language". It just has to be more useful than manually typing in shell commands one at a time. You're right about them needing to be short. Ideally every program should be short. But scripts are especially supposed to be short, because they're never supposed to get complex (or be fast/efficient); that's when you need a "real" program. Even the bottom of the Bash man page says about itself: "BUGS: It's too big and too slow." (fwiw, back in the day we didn't use linters, we just had to write it all correctly the first time. punch card typos are a memorable lesson in checking your work)
- pizza234 5y agoIt's technically a glue language, and can be considered a language due to the constructs it provides (e.g. arrays and hash maps). The fact that every language has a variable amount of a certain property doesn't mean that the amount is the same for all the languages. The amount of domain knowledge required for simple programs is considerably higher in Bash, than any related language; even iterating the lines of a string can be unintuitive to casual users. Bash is weakly typed, and weak typing is considerably unsafe (bug-prone) than any other related language (e.g. Python, which is has taken has share of shell scripting in modern operating systems). Returning values via stdout is a workaround. If other parts of the function send messages to stdout, one needs to use a combination of redirects, which the vast majority of the programmers don't know. Calling the ability to return a value from 0 to 255 support for return value is a big stretch. I think that Bash is a necessity, and I frequently use it (and it's fun-ish for small scripts), but it definitely doesn't fit the definition of "easy", being considerably more error-prone and unintuitive to any comparable language, possibly even when compare to Perl.
- throwawayboise 5y agoNot all platforms have bash by default.
- MichaelMoser123 5y agoi understand that there are subtle differences between minor versions of python, and its batteries. But what about putting it all into a docker image?
- bryanrasmussen 5y agoso instead of choosing a different language without particular problems, add another tool in your chain to manage to handle the problems? Based on the idea choose the right tool for the job it seems python is not right for this job.
- MichaelMoser123 5y agoDon't know what the 'right' tool is. i think it is easier to code up stuff in Python, as compared to golang. I am much more productive in Python, the language is more expressive, imho. An alternative would be to write in Perl5, perl is pretty much fixed, these days (if you insist on not using containers) Try to compile your go tools in year, or two. If think that you will be hard pressed to find the dependent library versions. Packaging is not the strongest part of go. A version in your package translates to a tag in the referenced git repository, now sometimes people do funny things with their repositories, and older tags disappear. Python and Perl5 are better, at packaging. At least you have all of the older library versions on https://pypi.org/ https://pypi.org/ I think, that the decentralized approach to packaging (as used in golang) is far from optimal. Maybe this is all due to the fact, that google is using a mono repository, internally for their own code. So that they are not eating their own 'dog food'. Sorry for ranting, judgements like 'right tools' really get me going..
- travisd 5y ago“Right tool” is not a value judgement. Python is better suited to certain applications, and go is better suited to others (with overlap obviously). This is partially a function of the language itself (eg, statically compiled or not) and partially a function of ecosystem (eg, writing container tools, go is by far the most pragmatic choice and Python for many ML applications). I think GP was just saying that most times, fewer moving parts is better. Avoiding Docker is great if you don’t need it (especially for a CLI tool). Obviously there are trade offs that must be evaluated in the context of a given problem and domain.
- Too 5y agohttps://docs.python.org/3/library/zipapp.html https://docs.python.org/3/library/zipapp.html Using the zipapp module, it is possible to create self-contained Python programs, which can be distributed to end users who only need to have a suitable version of Python installed on their system. The key to doing this is to bundle all of the application’s dependencies into the archive, along with the application code Venv+pip or pipx are also fairly good if you have a pip registry.
- chana_masala 5y ago> who only need to have a suitable version of Python installed on their system. And there's the problem
- moltar 5y agoI like oclif - it’s what powers Heroku CLI. Great TypeScript support.
- thunkshift1 5y agoWhy is python bad for cli applications
- tyingq 5y agoRight from the very first bytes of your cli app you run into issues. The interpreter line. You have to choose #!/usr/bin/python or #!/usr/bin/python2 or #!/usr/bin/python3 (or similar variations with /usr/bin/env). All of these happily ignored by Windows. The latter two may not exist on some platforms. The first one could be invoking either a python2 or python3 interpreter. There's some python magic for determining the version, but if you use a feature not supported by the version it raises an uncatchable exception...because python's parser has control before your code has control. This isn't the only deployment issue, but a good example of what you're in for.
- throwaway19937 5y agoPackaging and deploying Python apps is an adventure.
- privacyonsec 5y agoyou should try PyInstaller
- apple4ever 5y agoIf you are good, you don't have to worry about it. I never have to, because my scripts use packages easily available from pip and I keep them up to date as things change.
- harel 5y agoIf you don't know how to do something, everything is an adventure.
- willcipriano 5y agoI agree. I've used Python for a decade. Go and Node are a pain in the ass compared to Python, simply because I am unfamiliar with them. Use tox to create a virtualenv and then you can run irrespective of system packages and build the virtualenv in a single 'tox' command.
- doctor_eval 5y agoIf your users are all developers I’ve found `go install` an absolute winner for CLIs.
- tgbugs 5y agoI have the same issue. I'm pretty sure you can dig through my post history and find a blanket statement of me swearing off python for all new projects. Here are some ... reflections. In short a package manager + git is the easiest way to distribute and maintain software ... once you know how to use them. Configuring in a new environment? There is nothing that works reliably, other than maybe a full blown human level intelligent agent that is a senior developer with direct access to poke the system until it works. Further, developers with little time or experience packaging software for regular users may balk at what they rightly perceive as a route that is more difficult and creates more work in the long run because as a dev you can't just drop a "fix pushed" message your messenger of choice and go back to work. Static linking seems like it is a solution until you have to distribute to multiple operating systems. Only things like cosmopolitan libc have been able to overcome this, but even that requires a binary blob. In long .... I maintain a dozen or so python packages. Small stuff, mostly used by myself and my research group. At some point I needed to make some of that functionality available for other devs and some non-technical users. It was and is a massive pain. Why was it such an issue? I think it is because many of the quick internal scripts were originally written assuming that only technical users as advanced or nearly as advanced as the author would be using them, and that any such user would be able to use git, configure their environment, and generally keep their system up to date using a package manager. Aside. Add to that the fact that my dev environment is Gentoo, which has one of the sanest python environment management solutions available, and every time I step outside it or get reports from the poor lost souls trying to run my code "elsewhere" I enter a world of sanity destroying chaos. The workarounds that I created to try and mitigate some of the issues were poorly designed because I tried to YAGNI without realizing that you inevitably gonna need it (YIGNI?) and that the technical debt surrounding issues with interfacing with the environment is 10x more time consuming than anything else, especially if it is in the form of reports from souls lost in the warp. Even if a dev can use git and a package manager they can still get stuck configuring the environment. Seems easy until there are secrets to distribute or someone needs to run something in a sightly different environment and nothing works. For non-technical users it can be a stretch to get docker set up correctly, especially if they need X11. If I can get ssh access to the machine then there is a chance, but that doesn't scale beyond maybe 2 or 3 users. I have seen enough of this to have started to seriously consider just learning how to create installers for macos and windows and build them via CI. The long term solution is to switch off python completely, but the legacy code will continue to haunt us for a long time. I've learned docker to try to create controlled environments. I've looked at statically linking SBCL to musl libc. I've looked into cosmopolitan libc. I've explored trying to use Emacs, elisp, and org-mode as another possible route to bootstrap stable and predictable environments. I've looked into Gentoo prefix. I've wondered whether I could get users to install Pharo, but realized that it is already a tossup whether they can get docker running. I'm so sick of having to repeatedly do searches in high dimensional configuration space (fight with the environment) just to get some random code to run for a user who can't deal with the configuration space themselves (and it's not their fault!). At this point it would be easier to wire all my users up to a mitogen botnet ... except that some of them use windows! Telling a user "install python" or "install x" and then run this code leaves out the "and resolve the 10 completely novel errors you will encounter along the way." This is because there are at least 5 ways to "install python" (oh, don't forget the even bigger nightmare that is "install pip") on any given operating system (forget distros [0]), and each one results in a setup where things like data or configuration files are installed in different places with no obvious way to figure out where those places are, or you are forced to resort to a runtime lookup that causes CLI program startup time to explode to hundreds of milliseconds, making the entire point of having a quick CLI script moot. Hello world may be the easiest program to write but it is by far the hardest program to run. Dependencies? Data? Network? Good luck. 0. https://repology.org/project/python/packages https://repology.org/project/python/packages
- deleted 5y ago[deleted]