5 ms·

It would be super nice if there was a package offering "native" shell experience in python. Say from pyshell import sh, ShellError def func(a,b)

It would be super nice if there was a package offering "native" shell experience in python. Say from pyshell import sh, ShellError def func(a,b) output, retcode = sh("df -k | awk '{print $5}'") if retcode == 0: return output else: raise ShellError(output,retcode)

mkesper 8y ago
https://shell.readthedocs.io/en/latest/ https://shell.readthedocs.io/en/latest/ looks promising, will try next time before writing "just a small script" in bash (small scripts always grow and you pile on dirt on them when starting in bash).
- flagada 8y ago
  This looks great, thanks! It looks very easy compared to subprocess.run and friends. I hope it'll help convincing my colleagues to default to python instead of bash (or, heavens forbid, csh)
amelius 8y ago
Isn't this what os.system(...) does? By the way, why not put the raise inside sh? EDIT: Ok, the difference is that os.system does not return stdout. But you can use the subprocess package to capture stdout (and stderr as well). If you want pipe syntax, you can invoke e.g. bash -c "...", where the dots are your command. But the problem with this is that it's prone to injection attacks, unless you are very careful. I guess the same warning applies in your original "sh" example.
pushpop 8y ago
I disagree. If I’m porting a shell script to another language then I would want to use as much native code as possible rather than shelling out. Sure, it takes a little more effort, but the output is more idiomatic code that runs faster (fork() is slow).
lars512 8y ago
The sh module is really excellent, and it lets you take the best parts of bash (the external commands, pipes) and ditch the bad parts (string handling, general programming). I think it’s what you’re looking for. https://amoffat.github.io/sh/ https://amoffat.github.io/sh/
- tarruda 8y ago
  I wrote https://github.com/tarruda/python-ush https://github.com/tarruda/python-ush for my own scripts. It was inspired by "sh", but my version supports native "lazy" pipelines (using python operator overloading) and other nice features not provided by sh. Also it works seamless between Unix/windows I was also planning to extend it to support python3 asyncio, but never got around it
groestl 8y ago
This is a simple example, but usually you have command line arguments, the output can be large, and you might have input to process. As soon as you have that, you arrive at a more complex API, but one where you don't shoot yourself in the foot so easily (e.g. with special characters in file names and friends).
tarruda 8y ago
Check out https://github.com/tarruda/python-ush https://github.com/tarruda/python-ush It provides an idiomatic shell scripting experience for python. For example, it uses the "pipe" operator overload to allow complex pipelines to be created (pipes are created using platform native pipes, so there's no performance difference from a shell pipeline). The same operator can also be used for redirections
Twirrim 8y ago
Fabric'll do that for you (or rather Invoke, I guess I should say as part of the Fabric 2 changes) >>> from invoke import run >>> date = run("date") Sat Apr 13 11:09:18 PDT 2019 >>> dir(date) ['__bool__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'command', 'encoding', 'env', 'exited', 'failed', 'hide', 'ok', 'pty', 'return_code', 'shell', 'stderr', 'stdout'] >>> date.return_code 0 >>> date.stdout 'Sat Apr 13 11:09:18 PDT 2019\n' >>> do_they_exist = run("grep banana /etc/passwd") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/__init__.py", line 46, in run return Context().run(command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/context.py", line 94, in run return self._run(runner, command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/context.py", line 101, in _run return runner.run(command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/runners.py", line 271, in run return self._run_body(command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/runners.py", line 404, in _run_body raise UnexpectedExit(result) invoke.exceptions.UnexpectedExit: Encountered a bad command exit code! Command: 'grep banana /etc/passwd' Exit code: 1 Stdout: already printed Stderr: already printed If it gets a non-zero code back (like grep does when it fails to get a match), it raises an UnexpectedExit exception.

mkesper 8y ago

https://shell.readthedocs.io/en/latest/ https://shell.readthedocs.io/en/latest/ looks promising, will try next time before writing "just a small script" in bash (small scripts always grow and you pile on dirt on them when starting in bash).

flagada 8y ago
This looks great, thanks! It looks very easy compared to subprocess.run and friends. I hope it'll help convincing my colleagues to default to python instead of bash (or, heavens forbid, csh)

amelius 8y ago

Isn't this what os.system(...) does? By the way, why not put the raise inside sh? EDIT: Ok, the difference is that os.system does not return stdout. But you can use the subprocess package to capture stdout (and stderr as well). If you want pipe syntax, you can invoke e.g. bash -c "...", where the dots are your command. But the problem with this is that it's prone to injection attacks, unless you are very careful. I guess the same warning applies in your original "sh" example.

pushpop 8y ago

I disagree. If I’m porting a shell script to another language then I would want to use as much native code as possible rather than shelling out. Sure, it takes a little more effort, but the output is more idiomatic code that runs faster (fork() is slow).

lars512 8y ago

The sh module is really excellent, and it lets you take the best parts of bash (the external commands, pipes) and ditch the bad parts (string handling, general programming). I think it’s what you’re looking for. https://amoffat.github.io/sh/ https://amoffat.github.io/sh/

tarruda 8y ago
I wrote https://github.com/tarruda/python-ush https://github.com/tarruda/python-ush for my own scripts. It was inspired by "sh", but my version supports native "lazy" pipelines (using python operator overloading) and other nice features not provided by sh. Also it works seamless between Unix/windows I was also planning to extend it to support python3 asyncio, but never got around it

groestl 8y ago

This is a simple example, but usually you have command line arguments, the output can be large, and you might have input to process. As soon as you have that, you arrive at a more complex API, but one where you don't shoot yourself in the foot so easily (e.g. with special characters in file names and friends).

tarruda 8y ago

Check out https://github.com/tarruda/python-ush https://github.com/tarruda/python-ush It provides an idiomatic shell scripting experience for python. For example, it uses the "pipe" operator overload to allow complex pipelines to be created (pipes are created using platform native pipes, so there's no performance difference from a shell pipeline). The same operator can also be used for redirections

Twirrim 8y ago

Fabric'll do that for you (or rather Invoke, I guess I should say as part of the Fabric 2 changes) >>> from invoke import run >>> date = run("date") Sat Apr 13 11:09:18 PDT 2019 >>> dir(date) ['__bool__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'command', 'encoding', 'env', 'exited', 'failed', 'hide', 'ok', 'pty', 'return_code', 'shell', 'stderr', 'stdout'] >>> date.return_code 0 >>> date.stdout 'Sat Apr 13 11:09:18 PDT 2019\n' >>> do_they_exist = run("grep banana /etc/passwd") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/__init__.py", line 46, in run return Context().run(command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/context.py", line 94, in run return self._run(runner, command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/context.py", line 101, in _run return runner.run(command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/runners.py", line 271, in run return self._run_body(command, **kwargs) File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/runners.py", line 404, in _run_body raise UnexpectedExit(result) invoke.exceptions.UnexpectedExit: Encountered a bad command exit code! Command: 'grep banana /etc/passwd' Exit code: 1 Stdout: already printed Stderr: already printed If it gets a non-zero code back (like grep does when it fails to get a match), it raises an UnexpectedExit exception.