8 ms·
Python Ecosystem - An Introduction
- fasouto 15y agoNice article but you have a typo: updrage instead of upgrade :)
- smichael 15y agoAnd at the end, it should be "omission" not "ommision". Thanks a lot, wish I had this years ago.
- e1ven 15y agopip is a neat system and all, but I still don't see why to use that versus a system package?
- freyrs3 15y agoSystem packages install globally, pip can install globally or inside a virtualenv.
- timtadh 15y agoUpdating the source package is easier for one. Ubuntu's packages are often out of date, especially if you are running an LTS. It also integrates nicely with virtualenv which makes managing dependencies easier, via pip freeze.
- politai 15y agoSystem package's availability vary a lot with the distribution you're using. For instance I use Debian and my web hosting provider is based on CentOS and share my REQUIREMENTS.txt file.I can't really depend on the packages being available on every distro's repository, let alone the same version. Also, I wouldn't like to install some packages system-wide (a friend's personal project from a github repository) so I use pip and virtualenv in a similar manner as the article recommends to maintain a per project dependency library. That's how I see it :)
- c4urself 15y agoif you have multiple projects on a system you may have different dependency requirements. module with version 1 for one project and the same module with version 2 for the other project.
- d0mine 15y agopkg_resources.require() can handle this case. Though `virtualenv` is more explicit.
- mark_story 15y agoSystem packages are going to miss some package that exists in pypi. Using pip is also good if your company maintains their own internal toolsets, and want to be able to easily reuse those across multiple applications.
- brndnhy 15y agovirtualenv can't isolate your system's package util like it can isolate pip.
- briancurtin 15y ago> Choose Python 3 only if you need to and/or fully understand the implications. I would apply the "if you need to" part to Python 2. "3 if you can, 2 if you must"
- jholman 15y agoThe argument (for installing 2 as a default) is that a novice to the ecosystem will be disproportionately harmed by the inevitable Python3 experience: Novice: "I want to do X" Internet advice: "Use package Y" Novice: "Okay <install, install>, wtf nothing is working" <long frustrating debugging session> Novice: "Oh, wow, this doesn't support Python3 yet. Now I have to ignore all the internet advice and forge my own path, OR port all my code back to 2.7! This language sucks!" The alternative seems preferable: Stuff works, but occasionally you don't get a whiz-bang feature (you "only" get 2.7's feature set, poor you). Five years pass, you learn the language. Now you've gotta learn a bunch of new habits, which sucks, but it's not as bad because now you're pretty good at Python, so the easy parts are easy.
- briancurtin 15y agoI think they should be asking "How do I do X" before choosing a version to install. The answer might lead them towards IronPython or Jython, not a version of CPython from python.org. Some questions might even lead people straight to PyPy. More and more questions have problems that can be solved by Python 3, but it's probably a pain in the ass if you start with 3 then work out how to solve your problem.
- sixtofour 15y agoPython 3 is still an "eyes open" choice. Python.org all but comes out and says that 2 is still the safer choice. http://wiki.python.org/moin/Python2orPython3 http://wiki.python.org/moin/Python2orPython3 tl;dr "If 3 does everything you need, great. However, there's a good bit of things that still don't work with 3, in which case 2 is the safer choice. Here's a pretty long list of reasons why 2 is probably better for you. ..."
- 15y ago
- Sukotto 15y agoThank you for this article. As per the Pragmatic Programmer, I thought I would learn Python this year. It's been a tremendously frustrating experience getting a workable stack installed. I wish the famous "One -- and preferably only one -- obvious way to do it" Python design philosophy extended to actually installing everything :(
- RyanMcGreal 15y agoThis is very good. I wish a resource like this was around when I was first learning Python. The difficulty of getting things to work around the language has always been a pretty stark contrast to the ease of the language itself.
- senthil_rajasek 15y agoNice article. I am bookmarking it. Also The Zen of Python can always be accessed by this Easter egg >>> import this http://www.python.org/dev/peps/pep-0020/ http://www.python.org/dev/peps/pep-0020/
- sxephil 15y agorelax. it's a joke http://svn.python.org/view/python/trunk/Lib/this.py?view=markup http://svn.python.org/view/python/trunk/Lib/this.py?view=mar... IMHO this is more useful in the long run ... http://www.cs.yale.edu/quotes.html http://www.cs.yale.edu/quotes.html
- krosaen 15y agoThough the article claims to be targeted at users running on linux, most of the the info is still quite useful regardless of the platform - just figure out how to install python and and pip and the rest is pretty platform agnostic.
- simonw 15y agoI'd love to have one of these for Ruby. Every time I want to try out something written in Ruby I run head-first in to the packaging problem - Debian and Ubuntu don't appear to like shipping a working gem (presumably because it conflicts with how apt likes to do things) and the documentation on how to resolve the resulting inscrutable error messages isn't particularly easy to find. The Mac is a bit better, but I still run in to problems far too often. I'm pretty sure a "Ruby Ecosystem, An Introduction" guide is exactly what I need.
- sixtofour 15y agoThe way these things get written is by keeping notes as you figure out the bits and pieces ...
- mnazim 15y agoYou got it spot on. Note keeping the the way to go. The post was extracted from our internal wiki, the content in which has evolved over past 2 years and spread all over. It took me about 20-25 hours, spread over 3 weeks, to put it in the shape of a single post/tutorial.
- silversmith 15y agoI'm sitting in lecture right now, so here's a short version: - Use http://beginrescueend.com/ http://beginrescueend.com/ to install ruby itself - Use http://gembundler.com/ http://gembundler.com/ to manage project-level dependencies Both of those sites have good examples, but if no one steps up I might just write something like the linked guide when time frees up.
- imalolz 15y agoUp-vote, because it's a great idea for any language/tool out there. I happen to know ruby, but taking the first steps in anything is difficult, and usually involves looking up a lot of the things you already know from a more familiar place (setting up, using libs, etc). I'd really love to see a community resource like this - perhaps I'll start working on one :)
- 15y ago
- eccp 15y agoGreat article. Is it just me or the prepend/append examples are swapped? ie. If you want to append TO your PYTHONPATH you should do PYTHONPATH=$PYTHONPATH:/some/new/path
- sirn 15y agoIt depends on what do you mean by "append", I guess. Append the search path (e.g. "try this one first"), then /path:$PYTHONPATH is correct. If you mean append the variable then your example is correct.
- igorgue 15y agoGreat article. If you're on Ubuntu LTS you should install PIP from PyPI (easy_install pip), since the system package management version is outdated and it doesn't have the (very useful, since PyPI likes to go down) --use-mirrors install option. That would be my only recommendation.
- kmfrk 15y agoI would have loved to hear something about unit testing. Is nose or unittest2 the unit testing framework of choice for Python?
- nikcub 15y agoGreat post. I would add: * the site module, which is imported by default and is what is responsible for setting up the default sys.path. You can skip 'import site' by running python with the -S switch. the site module is written in python, so you can scan through it and understand how python starts up and inits. * PYTHONSTARTUP env variable, which points to a python file that is run (like a bashrc, or AUTOEXEC.BAT, if you prefer) on interactive prompt startup. I use this to import custom paths and modules that I want to access from the REPL, such as Google App Engine * I use pip with local repositories. clone the repos of the libs you need, and then pip install in the virtualenv from that local clone: $ pip install git+file:///Users/nik/.python-packages/tornado (note the triple slash). Or straight from GH: $ pip install git+git://nikcub@github.com/nikcub/tornado this can keep your versions in sync across all projects and virtualenvs and it means no re-downloading and you can setup and update projects while offline. * don't store the actual project inside the virtualenv. the virtualenv provides the execution context (setup and torn down using the virtualenvwrapper helper scripts). a common practice is to place all your virtualenvs into a directory like ~/.virtualenvs. you should never have to cd into this dir, access it using the wrappers and pip. (edit: also agree with comment below that you shouldn't be sudo'ing). * just a quick add, I think it is definitely worth learning how to install python from source.
- mgedmin 15y agoPYTHONSTARTUP also lets you enable things like * coloured prompt * tab-completion * persistent history Here's mine: http://mg.pov.lt/pythonrc http://mg.pov.lt/pythonrc
- mnazim 15y agoPlease keep the great feedback coming. I will try to incorporate as much as possible. I am indebted to HN community for the great feedback so far.
- d0mine 15y agoA formidable effort. It might be matter of taste but recommendations given starting from "Understanding the packages" and to "Install packages that need compiling" are almost harmful. My preference: * you should not care what is your `sys.path` looks like. You need it for debugging if something goes horribly wrong. A tutorial might mention it but things like `sys.path.insert(0,..)` should be avoided or accompanied with a big disclaimer (don't use nuclear weapons if you care about the future) * the same goes for `PYTHONPATH`. It is a hack that rarely needed * don't use `sudo pip`. System packages should be managed by a system packager. Use `pip --user` or create a `virtualenv` * `pip` can handle tarballs there is no need for `python setup.py install` in this case. "Code Like a Pythonista: Idiomatic Python" is worth mentioning http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html http://python.net/~goodger/projects/pycon/2007/idiomatic/han... Some third-party packages that could be listed (it is subjective): bpython - interactive prompt; something for tests e.g., pytest, tox, selenium; sphinx - docs; lxml - xml/html, werkzeug - if you talking about web-development; SQLAlchemy - sql; Cython - C extension, ~ Python syntax; async. libs e.g., gevent, Twisted.
- mnazim 15y agoMy indented audience was not the pure beginners. I was targeting for developers coming to python from other platforms. Over past 2 years, if I have learned one thing while training interns, trainees and experience devs in Python/Django, it is that packaging confuses people a lot. apt-get vs easy_install vs pip. That is why I choose to spend most time on package management and virtualenv etc. I think, I should have included stronger indications of this. In general, your feedback is very good and important. Idiomatic Python specifically is an important resource and should have been included. I will incorporate your feedback as soon as I can squeeze out some time. Big Thanks!
- imalolz 15y agoGreat post - I wish there was a unified resource for things like that for other languages/tools. I would only add iPython - a must for any console adventures.
- mixmastamyk 15y agobpython is much simpler, yet still helpful.
- MostAwesomeDude 15y agoipython has known issues running certain code. It is not guaranteed to function like a normal interpreter. If you insist on a fancy interpreter, use bpython.
- obtu 15y agoThese issues are not known to me, could you expand?
- MostAwesomeDude 15y agoReferences last far longer than they should, because the interpreter doesn't always let go of references to returned results. There used to be a problem with certain statements being executed as expressions and printed, but I'm told that's been fixed. The encoding is always Latin-1. Always. Hope you don't use Unicode literals. It's not Python. It's Python plus other things. That's always a Bad Thing because it precludes taking results from the interpreter and using them in plain Python contexts. web2py also has this problem.
- obtu 15y agoDefinitely; IPython is great at making Python more self-documenting, with completion, the ? help function, and the ability to embed an interpreter anywhere in your program.
- kisielk 15y agoHave you thought about combining your work with Kenneth Reitz's Python Guide? https://github.com/kennethreitz/python-guide https://github.com/kennethreitz/python-guide It looks like you're covering a lot of the same ground.
- danso 15y agoThis is fantastic. Someone could make a well-visited site by creating similar documents for all the major languages.
- detour 15y ago"While not a software tool per se, PEP 8 is a very important resource related to Python." Actually, it is :) pip install pep8
- vaksel 15y agowould make a good idea for a startup...a Python host that has a simple checkbox interface for installing all this stuff. That way you can get started with coding instead of having to install everything by yourself.
- dman 15y agohttp://enthought.com/products/epd.php http://enthought.com/products/epd.php (Disclaimer - I work for enthought)
- krupan 15y agoI mostly dabble with python. I learned a little reading this, and it raised some questions for me. Is there a reason --distribute is not the default behavior of virtualenv? Is there a plan to incorporate the stuff virtualenvwrapper does into virtualenv (virtualenvwrapper is a pretty cumbersome name, if for no other reason)?
- mardiros 15y agoI think that $ sudo apt-get install python-pip is a bad idea. You should not mix multiple packaging system on your operating system. And more you can dammage it pip provide more recent package than your distro. And if you upgrade a lib that have an incompatibility with a part of the system, you can corrupt it. I have no example to give but I am sure you can find it... Ubuntu now have many tools written in python. You should use pip inside a virtualenv only. And, fortunatelly when you create a virtualenv, pip is installed in it, and you don't need to use the --distribute to have it.
- wavetossed 15y agoWhat I tell people is that Python (as installed by apt-get, yum) is not on your system for you to develop with. It is there because some of the system tools are written in Python, including at least part of the apt packaging system. It's also there for System admins to use for writing Python scripts instead of bash scripts. But for app development, get your own Python, manage it yourself and install 'distribute' so that you have both easy_install and pip to work with. I've taken that to extreme by making a portable Python distro that comes in a tarball and runs on any Linux distro, but even if you only untar the source and run ./configure --prefix=/home/python;make That will build a default Python with support for any shared libraries for which you have a development version installed ( -dev version on debian/ubuntu, -devel version on redhat/suse) sudo make install will install it, assuming that you have write permissions on the target prefix that you specified. You can even hide it in your home directory with --prefix=~/tools/python272
- mardiros 15y agoI don't anderstand why you want to use the --distribute to have pip and easy_install installed. If you do not, you also have those tools.
- smogzer 15y agoNice list. I would replace django with web2py, it is closer to the python philosophy imho.
- darraghenright 15y agoLooks great. I've started turning my attention to python recently so skimming through this I can already see lots of stuff that'll be very useful. So much that perhaps a linked TOC at the top of the page could be an idea?
- aaronh 15y agoI'm surprised nobody has mentioned pythonbrew. https://github.com/utahta/pythonbrew https://github.com/utahta/pythonbrew It's the Python version of RVM. It is higher level than even virtualenv, and in my opinion, the most seamless way to manage Python environments.
- ryall 15y agoGreat post! Covers everything I still didn't understand after reading LPTHW. Thanks so much for sharing this.
- wavetossed 15y agoYou really shouldn't tell people to go ask how to install Python on stackoverflow.com. Instead give them a few URLs to stackoverflow questions with the answer such as this one http://stackoverflow.com/questions/7538834/how-to-create-a-python-distribution-like-activepython http://stackoverflow.com/questions/7538834/how-to-create-a-p... Or even better, give them a stackoverflow search like this one http://stackoverflow.com/search?q=%5Bpython%5D+%22install+python%22 http://stackoverflow.com/search?q=%5Bpython%5D+%22install+py... P.S. I think that your wiki page is a great idea and I'm going to write a custom one for our developer wiki.
- anacrolix 15y agoCHANGE YOUR FONT.
- sanxiyn 15y agoI am not sure why Requests gets this much press. httplib2 has all the features of Requests and more, and their APIs are not that different.
- xxiao 15y agogreat post!
- tchon 15y agoNOTE: there's brokenness lurking in urllib and email modules. The python library modules are more often than not modeled on perfect reality rather than a pragmatic one (e.g., complete violations of RFCs). BTW, if one reads re.py in the dist, you'll notice that it hasn't been touched by Fredrik Lundh since 2001! I should say I still love python. Its the most fun I've had programming next to Scheme, and well, NodeJS is kind of fun too (in an algol way).
- snthpy 15y agoThank you, thank you, thank you. I've been looking for something like this for ages. While Python the language is fantastic, I have found getting into the whole environment quite tricky so your guide is fantastic.
- axaroth 15y agoIn the section of "The Development Environment" you should mention the buildout (http://www.buildout.org/ http://www.buildout.org/). It is a very simple way to reproduce an enviroment.
- lamsg 15y agoGreat great great post!!!!!!!!!!!!!!!!!