5 ms·

Python command line oneliners

74 points by xenator 14y ago

cjgk 14y ago
Kind of related: cat messy.json | python -mjson.tool
- siddboots 14y ago
  This is one of the examples in the article.
  - cjgk 14y ago
    So, if I had spent the time used to create a HN account on reading the article another time, I wouldn't have to feel this stupid...
- glenjamin 14y ago
  It's worth noting that this doesn't retain ordering
pixelbeat 14y ago
Related: http://www.pixelbeat.org/programming/evanescent_python.html http://www.pixelbeat.org/programming/evanescent_python.html
- xenator 14y ago
  In article I focused on usefulness of python one liners regardless of what technology they use. This one more for python devs.
gvalkov 14y ago
This article isn't as much about one-liners as it is about the usefulness of the if __name__ == '__main__' idiom. It's great when a module can be consumed as a library and as a command-line script at the same time. Unfortunately, Python isn't really that good for writing one-liners, especially those that process input. At one point I was sufficiently unhappy about this, that I tried doing something about it: https://github.com/gvalkov/python-oneliner
- pgorla 14y ago
  It also looks like this was just for self-reference; it's not an article as it is a collection of useful tips. I write a lot of these to remind myself of tool usage.
pgorla 14y ago
Also, the last line to check for 'one-liners' won't get them all: $ grep -r "if __name__ == '__main__'" /path/to/lib Should be: $ grep -r "if __name__ == ['\"]__main__['\"]" /path/to/lib
- cabalamat 14y ago
  You could also do this: grep -n __main__ *.py|grep __name__ which turns up few false positives.
- e12e 14y ago
  If you're not storing that somewhere, but rather typing it, I find you're often better off by being a little more forgiving, eg: $ grep -r "if __name__ == .__main__." /path/to/lib or even: $ grep -Er 'if\W__name__\W==\W.__main__.'
keenerd 14y ago
Most of these should be tools of last resort, the technological equivalent knowing how to start a fire without matches. The performance of SimpleHTTPServer is abysmal, particularly if you want to serve a few files over 100MB or thousands of little files. Even toy webservers blow the Python versions away. If I need a quick one-off server to share files over the LAN, webfs [1] is first on the draw. 90% of my uses don't even need fiddling with the command options. Though webfs is fairly limited, SSL is the most complicated thing it can do. For slightly more advanced one-off webservers, thttpd [2] is wonderful. It's got many of the features of heavy duty webservers (plus great throttling controls) but every option is tune-able from the command line. And as an aside, my own jshon [3] is usually nicer and faster than mjson.tool, at least until you hit a fairly high level of complexity. [1] http://linux.bytesex.org/misc/webfs.html http://linux.bytesex.org/misc/webfs.html [2] http://www.acme.com/software/thttpd/ http://www.acme.com/software/thttpd/ [3] http://kmkeen.com/jshon/ http://kmkeen.com/jshon/ edit: Thanks dschep.
- lucian1900 14y ago
  It's useful for quickly getting an HTTP server. No one in their right mind would use it for anything more. There are plenty of other very good python web servers.
  - keenerd 14y ago
    I am arguing it is not even useful for that. If you simply want to share files across a LAN (the use case for these oneliners), SimpleHTTPServer falls flat on its face. It'll transfer large files at DSL-like rates and time out completely on directories with large numbers of files in them. What are you supposed to do with an http server after getting one quickly? Stare at the open port? The only thing quick about SimpleHTTPServer is the startup time. Don't actually try using the server. The specific instance that put me off of the python servers was when a friend downloaded a CableGate tarball and someone else wanted to look at it. No one had a flash drive on them, but the fellow did have python installed. One SimpleHTTPServer later and the tarball was being transferred at 100KB/sec, slower than the original download. The cables were all html anyway so let's try uncompressing the tarball and serving those. Now the server would time out instead. What do people actually use SimpleHTTPServer for?
dgulino 14y ago
There are a few scripts that facilitate the use of python on the command line: https://gist.github.com/dgulino/4750088 https://gist.github.com/dgulino/4750088 (mine) http://code.activestate.com/recipes/437932-pyline-a-grep-like-sed-like-command-line-tool/ http://code.activestate.com/recipes/437932-pyline-a-grep-lik...
alecthomas 14y ago
I recently wrote PAWK (https://github.com/alecthomas/pawk https://github.com/alecthomas/pawk) in order to give me Python at the command line. It's exactly what you think it is: Python AWK. It's served me well so far.

74 points by xenator 14y ago

cjgk 14y ago

Kind of related: cat messy.json | python -mjson.tool

siddboots 14y ago
This is one of the examples in the article.
- cjgk 14y ago
  So, if I had spent the time used to create a HN account on reading the article another time, I wouldn't have to feel this stupid...
glenjamin 14y ago
It's worth noting that this doesn't retain ordering

pixelbeat 14y ago

Related: http://www.pixelbeat.org/programming/evanescent_python.html http://www.pixelbeat.org/programming/evanescent_python.html

xenator 14y ago
In article I focused on usefulness of python one liners regardless of what technology they use. This one more for python devs.

gvalkov 14y ago

This article isn't as much about one-liners as it is about the usefulness of the if __name__ == '__main__' idiom. It's great when a module can be consumed as a library and as a command-line script at the same time. Unfortunately, Python isn't really that good for writing one-liners, especially those that process input. At one point I was sufficiently unhappy about this, that I tried doing something about it: https://github.com/gvalkov/python-oneliner

pgorla 14y ago
It also looks like this was just for self-reference; it's not an article as it is a collection of useful tips. I write a lot of these to remind myself of tool usage.

pgorla 14y ago

Also, the last line to check for 'one-liners' won't get them all: $ grep -r "if __name__ == '__main__'" /path/to/lib Should be: $ grep -r "if __name__ == ['\"]__main__['\"]" /path/to/lib

cabalamat 14y ago
You could also do this: grep -n __main__ *.py|grep __name__ which turns up few false positives.
e12e 14y ago
If you're not storing that somewhere, but rather typing it, I find you're often better off by being a little more forgiving, eg: $ grep -r "if __name__ == .__main__." /path/to/lib or even: $ grep -Er 'if\W__name__\W==\W.__main__.'

keenerd 14y ago

Most of these should be tools of last resort, the technological equivalent knowing how to start a fire without matches. The performance of SimpleHTTPServer is abysmal, particularly if you want to serve a few files over 100MB or thousands of little files. Even toy webservers blow the Python versions away. If I need a quick one-off server to share files over the LAN, webfs [1] is first on the draw. 90% of my uses don't even need fiddling with the command options. Though webfs is fairly limited, SSL is the most complicated thing it can do. For slightly more advanced one-off webservers, thttpd [2] is wonderful. It's got many of the features of heavy duty webservers (plus great throttling controls) but every option is tune-able from the command line. And as an aside, my own jshon [3] is usually nicer and faster than mjson.tool, at least until you hit a fairly high level of complexity. [1] http://linux.bytesex.org/misc/webfs.html http://linux.bytesex.org/misc/webfs.html [2] http://www.acme.com/software/thttpd/ http://www.acme.com/software/thttpd/ [3] http://kmkeen.com/jshon/ http://kmkeen.com/jshon/ edit: Thanks dschep.

lucian1900 14y ago
It's useful for quickly getting an HTTP server. No one in their right mind would use it for anything more. There are plenty of other very good python web servers.
- keenerd 14y ago
  I am arguing it is not even useful for that. If you simply want to share files across a LAN (the use case for these oneliners), SimpleHTTPServer falls flat on its face. It'll transfer large files at DSL-like rates and time out completely on directories with large numbers of files in them. What are you supposed to do with an http server after getting one quickly? Stare at the open port? The only thing quick about SimpleHTTPServer is the startup time. Don't actually try using the server. The specific instance that put me off of the python servers was when a friend downloaded a CableGate tarball and someone else wanted to look at it. No one had a flash drive on them, but the fellow did have python installed. One SimpleHTTPServer later and the tarball was being transferred at 100KB/sec, slower than the original download. The cables were all html anyway so let's try uncompressing the tarball and serving those. Now the server would time out instead. What do people actually use SimpleHTTPServer for?

dgulino 14y ago

There are a few scripts that facilitate the use of python on the command line: https://gist.github.com/dgulino/4750088 https://gist.github.com/dgulino/4750088 (mine) http://code.activestate.com/recipes/437932-pyline-a-grep-like-sed-like-command-line-tool/ http://code.activestate.com/recipes/437932-pyline-a-grep-lik...

alecthomas 14y ago

I recently wrote PAWK (https://github.com/alecthomas/pawk https://github.com/alecthomas/pawk) in order to give me Python at the command line. It's exactly what you think it is: Python AWK. It's served me well so far.