26 ms·

Mongrel2 Says, "Goodbye Python"

246 points by spahl 16y ago

deleted 16y ago
[deleted]
aphexairlines 16y ago
what about dumping a local python install in $HOME/local/opt/mongrel-python if m2sh didn't find a version it likes? (yes, it's overkill)
- apgwoz 16y ago
  This is an insane idea. I should not have to have a whole other version of python just to run a language independent web server. The choice Zed made is a great choice--all the functionality of the original python m2sh is present in the new, and there's nothing external to support, or dump somewhere else.
  - lincolnq 16y ago
    It's not that crazy. Python is designed for embedding. The interpreter is small. Commercial software that depends on Python (almost) always embeds it.
    - apgwoz 16y ago
      I know this, however, Zed was using this for a shell command, not as an embedded scripting language.
dschobel 16y ago
Zed's pronouncements are works of art. The man is a pro. A veritable Martin Luther for our times.
- auxbuss 16y ago
  Indeed. I got to the statement: "The distros have killed Python." Said out loud, "here we go", and settled down for some classic Zed. Can't fault him on it, though.
  - ubernostrum 16y ago
    I'd go one further: distros in general haven't killed Python, Red Hat has. By tying their flagship OS to a six-year-old Python release, they've made it impossible for a lot of popular software to move on and take advantage of more recent advances in the language (since "screw everybody who uses Red Hat" simply isn't a viable option).
    - gaius 16y ago
      This is a complete non-issue. ActiveState for example installs Python, Tcl, etc in /opt.
      - jnoller 16y ago
        Except no one installs ActiveState. For most, most of the market, the RedHat supplied version of python is python.
        joshfinnie 16y ago
        Does RedHat still have that much of a market share in linux to demand such? I guess I am out of the loop, but I hear way more about ubuntu these days than RedHat...
        gaius 16y ago
        If you want to target "Linux" generically then stock Red Hat is your safest bet. I don't know anyone with 10,000 Ubuntu boxes in a datacentre.
        count 16y ago
        The US Govt can use Redhat, but not Ubuntu (Ubuntu hasn't been Common Criteria / EAL tested). That's a pretty huge market right there.
zeemonkee 16y ago
Completely agree. The Linux distros (CentOS is the one I work with most, not out of choice) have antiquated Python versions. You can get round it with virtualenv etc., and I do this as a matter of course when installing Python web applications. For a generic "plug and play" package like Mongrel2 however, where sysadmins just want something that runs straight out of apt-get or whatever, it's a pain and is holding things back. I mean WTF does yum still require ye olde Python 2.4 ?
- joelmichael 16y ago
  CentOS is about as conservative an option as you can get, though. Unusably so, in my opinion. None of the other popular distros have packages that old.
  - zeemonkee 16y ago
    Question to manager: WTF are we using CentOS ? Reply: because it's Enterprise.
    - barfoomoo 16y ago
      Which distro you think is the best to run LAMP stack with an up to date package manager?
      - deleted 16y ago
        [deleted]
      - zeemonkee 16y ago
        Some flavour of Debian would be my preference. Generally with Python WSGI applications I tend to keep things sandboxed using virtualenv and pip in any case.
        barfoomoo 16y ago
        I am complete newbie when it comes to running a server but have a bit of experience using Ubuntu in my local box. Is ubuntu a good choice? Since most use a flavor of Red Hat like CentOS I am a bit hesitant to use Ubuntu in production.
deleted 16y ago
[deleted]
nailer 16y ago
I understand what Zed's saying here - particularly as I came to programming from a sys admin background. Just like Perl before it, Python (and to some extent, Ruby if you use Puppet) are part of the OS (Linux OS or OS X that is). Shit can and will break if you mess with your OS. You never got this with PHP because people didn't use it in their OSs. But there's a simple solution: not only does the Ministry of Packaging want you to use virtualenv, your OS maker does too. The faster pip and virtualenv become standard the better.
- dagw 16y ago
  Shit can and will break if you mess with your OS. Updating a program I use should never constitute "messing" with my OS, it should constitute "using". Why should I as an OS user have to constantly worry about upsetting the OS maker? Why should I have to jump through obscure hoops just to install some software without breaking my OS? Wouldn't a better solution be for the OS makers to wall off their magic python behind some wall where I cannot see, touch nor use it, and let me install whatever python I want without fear.
  - nailer 16y ago
    Firstly: no hoops are required to install any version of python or any required Python module your app needs: * Just use pip and virtualenv. * Your OS remains intact, and you get better deployment for free! Replacing a component of your OS (Python 2.x) with another one (python 2.x +1 ) affects: * The OS installer (Anaconda on RHEL, Ubuntu'sequivalent) * The packaging system (yum) * Many GNOME system admin apps. * The config management tools (puppet sits on ruby)
    - dagw 16y ago
      My point is that /usr/bin/python probably shouldn't be a core OS component. /usr/bin/python should be whatever I want it to be. The core OS-component should be something like /sys/bin/centos-python which I will never directly use, touch or even necessarily know about. The bottom line I guess is that mixing core OS components and user installed software in the same directory structure with no way of differentiating them is simply bad design.
antileet 16y ago
Hi Zed, Can you please elaborate with some details about how "broken" python is on systems, and the situation with python 2.4? According to the article, "recent Ubuntu releases" had broken/deprecated python installations. A quick search reveals that python 2.5 has been shipping as the default "python" meta-package(?) http://packages.ubuntu.com/search?keywords=python http://packages.ubuntu.com/search?keywords=python Can you also please elaborate any "features" that the install was missing - are these specific modules ?(I think python's sqlite3 is missing by default on Ubuntu. I'm not too sure.) I'm not questioning you or anything, I'm just slightly concerned as we had plans to ship a Python application, and when someone experienced enough has a valid complaint, it makes sense to understand the basis behind that.
- Twisol 16y ago
  Speaking as someone who knows practically nothing here, I think the issue is existing systems, not brand new systems. The barrier to updating an existing system becomes greater the more and the longer things depend on it.
  - deleted 16y ago
    [deleted]
- acqq 16y ago
  I'd very much like to know why it wasn't easier for him to simply write a Python code that doesn't use anything not existing in 2.4. Just because he downloaded to his computer whatever later version he likes he can't expect from all distros and all users to have exactly the same version across of all the currently running computers, it's more than unrealistic, it's simply impossible and always will be.
  - ubernostrum 16y ago
    I'd very much like to know why it wasn't easier for him to simply write a Python code that doesn't use anything not existing in 2.4. This isn't an "oh, a brand-new version just now came out and it's not in the distros yet" problem. Python 2.4 is six years out of date at this point. Python 2.3, which Red Hat will support until 2012, is even older (and for the longest time Red Hat didn't even use a 2.x Python at all -- they sat on 1.5 forever). That's simply unacceptable, and creates endless headaches for people who want to distribute software written in Python.
aphexairlines 16y ago
"older Debian"... but Python 2.5 has been packaged in Debian stable since at least Etch.
apgwoz 16y ago
Hilariously enough, I took the time to make sure the new config file format for m2sh was nearly perfectly backwards compatible with the old file. This is pretty funny because I predict nobody will blink an eye at the config file, even though it's basically Python. This is an interesting point, especially given the amount of code (relatively small) actually needed to parse the subset of Python the config format uses. That said, the subset is fairly "generic scripting language" since there aren't any declarations, only assignments, lists, etc (at least at a cursory overview).
joelmichael 16y ago
I'm not sure why he concerns himself with old distros. Are they the customers for Mongrel2? The description on the website emphasizes modern web technologies. Using Rails 3 is best with Ruby 1.9.2, and yet the only popular distros that install that via their package management system are the rolling release ones, Arch Linux and Gentoo (where it is masked). Most people willing to use this brand new software are also willing to compile their own necessary bleeding edge dependencies, and probably have a generally up-to-date distro preference (like Ubuntu). Even the latest Debian stable, a notoriously conservative distribution, which is a year and a half old and due to be upgraded soon, uses Python 2.6. I'm not sure it's reasonable to support much past that.
- vidarh 16y ago
  Upgrading Ruby to 1.9.2 won't break your OS. Upgrading Python on some distros can even break package management. That's the difference. And if you have a well tested, rock stable environment you generally don't want to mess with it if you don't have to, or may not easily be able to. You want to not support older than Lenny. Well, we have machines that are still on Sarge, though slowly being upgraded. Some of the machines haven't been rebooted since around the time Etch came out and that's the main reason they haven't been upgraded yet.
  - mfukar 16y ago
    That's only because package management "on some distros" (I won't name them, we all know which they are and I'm sure we're talking about the same one) is fubar. I'm not talking about the package manager alone, but also the over-the-top package dependencies, incoherent dependency resolver, let alone the sheer stubborness of some developers working on package management..
  - joelmichael 16y ago
    I'm sure older machines are out there and are doing fine for their purpose. But are you planning to install Mongrel2 on them?
mfukar 16y ago
Python, as per Guido Van Rossum's wishes, is pushing for suspension of all language syntax and built-in changes and work towards preserving semantics, precisely for giving time to alternative implementations time to catch up with CPython, as well as the various distributions to catch up with Python's recent progress. In short: "...have killed Python" is a hyperbole for saying "I was tired to hear people nagging about dependency on Python". Still, it's a very good thing what Shaw did.
- dschobel 16y ago
  I've never heard the distro argument used by any of the python maintainers and I would be surprised if they had. The solution to distros which are bundling three year old software is not to halt your progress for three years, that should be obvious.
  - mfukar 16y ago
    I was mentioning it as a side-effect, the main reason was to give time to alternative Python implementations to reach Python 3.1.
  - ubernostrum 16y ago
    The solution to distros which are bundling three year old software is not to halt your progress for three years, that should be obvious. Speaking as Django's release manager: Red Hat's continued use and support of ancient 2.x Python versions has been a factor in the schedule we're developing for migration to Python 3.x. We can't simply pull the rug out from under anyone who's using Django on RHEL.
    - dschobel 16y ago
      Seeing RHEL/CentOS keep coming up led me to some googling and the first result for "RHEL python" is http://www.python.org/download/linux/ http://www.python.org/download/linux/ (about as official as it gets) which says The IUS Community Project maintains recent versions of Python for Red Hat Enterprise Linux (RHEL) and CentOS. Installations are parallel installed next to stock versions of python therefore don't disrupt the functionality of critical python utilities on the system. Doesn't that solve the issue and keep you from being hamstrung by those distros?
doki_pen 16y ago
Including Gentoo is wrong. Gentoo uses a slotted ebuild for python. You can have many concurrent versions running. Not only that, but the latest portage works fine on the latest 2.6 release of python. If you don't want to learn how to use your OS, pick a different one. I hear that Macs "Just Work".
tzury 16y ago
I guess it was just about time for me to try mongrel on a fresh new project. I feel I am blessed for that only. Seeing that 0MQ batteries are included is a great bonus. If all would be fine, perhaps my network appliance would be shipped with mongrel2 which would be fantastic.
- zedshaw 16y ago
  Cool give it a shot. We've got a bunch of stuff we're working on for the 2.0 push but what you have in 1.1 should be stable and usable. If not, let me know by dropping a ticket.
donaq 16y ago
It's sad and ironic (for Python) that Zed actually had to switch from Python to C to make Mongrel more portable. Disclaimer: I love Python, but I hate that anything but the most trivial code won't run across all versions.
- rbanffy 16y ago
  > I hate that anything but the most trivial code won't run across all versions. Please, define "all versions" and "most trivial code". I have very little problems with Pythons ranging from the ancient 2.4 to modern 2.7. Of course, I am careful with what I do. I know if I use a dictionary comprehension I will be limited to 2.7+, so I try not to. I am quite sure a lot of the code I write could run under any 2.x Python with little modification. As for 1.x, I will agree that things get more complicated.
  - donaq 16y ago
    Well, it's a matter of opinion, but I would define "all versions" as versions sharing a major number, i.e. code targeting 2.2 should run on anything starting with 2. As for trivial code, just look at the string operations part of the documentation and search for "new in" and "changed in". My connection is rather slow right now, to the point that the page I mentioned is still loading, so I won't look for more examples, but string processing should IMO be trivial parts of any language. Don't get me wrong, the changes have mostly been useful ones for 2.x versions, but I have been caught testing scripts on my own box that end up not working after deploying. Since then, I have been, as you say, more careful, but I feel I have a valid gripe when I expect string and io operations to be consistent across major versions without having to do a fine print and version check. :)
  - zedshaw 16y ago
    generators
    - rbanffy 16y ago
      I know that if I really need generators, then I really need to install something newer than the 2.4 that comes with RHEL/CentOS 4. Generators is 2.6+, right? BTW, an ugly approach I used (more or less as a joke) was to check for generator availability and, in case I can't use them, use a list comprehension instead. There is a performance/memory hit but, depending on what you are doing, it's a usable alternative. And a nice place to hang humorous comments. But I agree that, if your users don't want to install an alien (from the package manager POV) Python in order to run Mongrel, getting rid of python code is the right thing to do. It's like having a weird dependency, kind of requiring a Fortran 77 compiler in order to run Perl...
deleted 16y ago
[deleted]
rbanffy 16y ago
It's funny how, for Zed, anything that doesn't work the way he thinks it's supposed to work is "broken" and the result of some boneheaded decision and that it "sucks". I too am frustrated by ancient versions of server OSs that use equally ancient versions of Python, but that's why virtualenv was invented. And source distributions. I am happy using it when required. Being "stable" in OS terms is not changing APIs very often. One of the good things is that I know that code I wrote for RHEL 4 the day it was launched will run flawlessly on RHEL 4 today. The primary goal of such a Linux distro is that "it has to work". I won't touch the system parts.
- Confusion 16y ago
  He doesn't blame it on 'some boneheaded decision'. As far as I can see, he doesn't blame anyone, he just notes the current state of affairs, in which the situation is indeed pretty much broken. Not just for him, because he is far from the only one that thinks it should work a certain way. that's why virtualenv was invented Yes and shouldn't it also be the OSes that use it to isolate their default Python install?
  - rbanffy 16y ago
    It's a design decision. They have a default Python install they support, validate and keep working that you shouldn't mess with. I am pretty sure you can't ruin a RHEL just by installing RPMs from RH's repos. If the Python the OS vendor provides is somewhat inadequate to you, it's your job to provide another one or to use another distro. However, if you choose to write your code against the distro's Python (something that's not that hard, really - 2.4 is a modern language) you can rest assured your code will work for as long as the distro is supported. It's not "broken", not a boneheaded decision. It's just the way it is. If you require 2.7 or 3.x goodness, you can always set-up different environments separate from the OS's Python and just be happy with it. And, BTW, I never put boneheaded in quotes ;-)
    - sophacles 16y ago
      It is completely possible, easy and reasonable to have a multiple versions of python installed. Many OSes do this, and the convention is to have them named like python2.4 python2.5 and so on. Any system scripts that require 2.4 can then change their shebang to say /usr/bin/python2.4. At the very same time, packages for newer pythons can be made available for ease-of-use for the rest of the world. This breaks nothing.
gfunk911 16y ago
This thread is a classic example of something that happens over and over. Someone says something is hard to use. 1/3 of the responses agree. 1/3 of the responses disagree, say it's simple. 1/3 of the responses say it's simple, all you have to do is use additional software package Y to manage X and it works great. This is usually a sign that there is some merit to the original assertion.
- dschobel 16y ago
  We're talking about technology, not politics so consensus be damned. There is a fact of the matter independent of perception and lack of consensus in and of itself is not a valid attack on a position. In this case maybe we can listen to those who claim the python situation isn't that bad and have provided specific technical solutions for evaluation.
  - gfunk911 16y ago
    If the question is "Do a significant number of users run into problems with a piece of software," then the consensus is absolutely relevant.
    - dschobel 16y ago
      The question isn't whether there is a problem but whether there is a solution. The virtualenv/pip people seem to think so.
      - masklinn 16y ago
        > The virtualenv/pip people seem to think so. No. Unless those people are morons, but the guys who actually code them know they're tools for python developers, and maybe for sysadmins of python solutions. Pip and virtualenv for software using Python as a config file format? not a good idea.
      - zedshaw 16y ago
        Yes, pip/virtualenv works for installing things like Django. It totally fails for something like m2sh which has to live in the system PATH so that you can run it from wherever you have your configs. But you know, I wonder if the various distros could sort of invert this and they start using pip/virtualenv instead of everyone else working around them?
Tichy 16y ago
Shouldn't python just be a library dependency like any other? I was hoping the problem of libraries with different versions was solved somehow. Apparently not :-(
terra_t 16y ago
Python brought this on itself by having no respect for compatibility between versions. There's really no such thing as the Python language, only "Python 2.2", "Python 2.4", "Python 3". If you want to run Python scripts, you need to have three or four different Python runtimes installed, and that's asking a lot of both the distros and of users who need to keep everything straight, and probably even hack the #! at the beginning of scripts so that they work correctly.
- jnoller 16y ago
  Bullshit. We (python-core) are obsessed with backwards compatibility. Adding new features that aren't in old version is the source of this problem. Python 3 is by far, the biggest intentional deviation from the "don't break existing scripts" mantra that's been in place for years.
  - terra_t 16y ago
    Why didn't the official bittorrent client run on Red Hat Linux (w/o downgrading your Python) for at least three years? Why was NLTK stuck on Python 2.4 for years? I might use Python to use NLTK, but why do I have to choose an old Python to get it to work... And what if I want to use it w/ software that needs Python 2.6?
- BrandonM 16y ago
  When you add features as useful as set() datatypes, if-else expressions, generators, and comprehensions, people are going to use them. Python 2.2 code nearly always runs without modification on Python 2.3+ (the only exception would be collisions between variable names and new keywords). But someone targeting Python 2.6 has a huge hassle when deploying to older versions. This means that Python is actually adding useful new functionality; that is, the incompatibility is a symptom of a good thing.
- stevelosh 16y ago
  Mercurial runs unchanged on 2.4 to 2.7, and it's not a trivial little piece of software. Pretty much the only thing you need to do to make your Python code run on 2.x+ is "don't use features released after 2.x".
jedbrown 16y ago
This is a major problem. PETSc uses Python for configuration and we have to be careful not to use any features requiring >2.3. The last release (this spring) required 2.2 because RHEL3 is still supported (in production phase 3, see http://www.redhat.com/security/updates/errata/ http://www.redhat.com/security/updates/errata/) until Oct 31, 2010. Python is easy to install, but it's a lot to ask of users when they just want to get your software up and running. Note that RHEL4 sees end of life Feb 29, 2012 and RHEL5 not until Mar 31, 2014. So Python-2.4 support will have to live for a very long time. I hate this, but don't see a decent alternative. Many of our users are building in batch environments of varying ages and just want to get their science done. Shell/bash is not very portable, it would be painful to write all the configuration in C (and confusing because people would need to both native compilers and cross-compilers).
- igravious 16y ago
  This is very interesting and is what RVM in the Ruby world was designed to solve in some way. So maybe we need a widespread and useful PVM. This feeds into an observation of mine I made over five years ago but that seems to be ever more relevant as time goes by. Linux and BSD seem to have "solved" the packaging problem through mega-repositories with dependencies and access to source code. However your favorite language and especially the scripting ones and virtual machines one will have there own repositories that are _not_ linked into the systems way of doing things. So at the system level we have RPM and Yum and Ports and Portage and Macports and Fink and Homebrew and AptGet and Conary and yada yada yada but at the language level we have (deep breath) Perl and CPAN, Ruby and GEMS, Python and EGGS, and Emacs and ? and Java and Eclipse plugins and on and on. I would switch IN AN INSTANT to the distro that integrated all this so that I would only need to go to ONE place to upgrade system packages and language packages. Honestly if nobody does it one of these years I might even be forced to do it myself and make mega $ € ¥. Hope this makes sense to somebody somewhere. :)
  - jedbrown 16y ago
    have there own repositories that are _not_ linked into the systems way of doing things One approach is to have a system that packages everything in the language-specific repository for the distro. Haskell on Archlinux is an example of this. I'm not a fan of actually using the language-specific packages because I like the rolling release model and end up having to do some manual work to reinstall all my packages when the distro upgrades it's copy (or switch to the next 6-month release with Ubuntu and similar).
deleted 16y ago
[deleted]
mahmud 16y ago
Instead of inventing your own syntax, why didn't you use a portable syntax like XML, JSON, YAML, or better yet, libconfig[1]? Please compare the libconfig test file to the mongrel configuration file: http://www.hyperrealm.com/libconfig/test.cfg.txt http://www.hyperrealm.com/libconfig/test.cfg.txt http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-190003.3 http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-190003.... The mongrel conf is a subset of libconfig; it doesn't use the more advanced features. Plus libconfig has bindings and versions of it in nearly every mainstream language. [1]http://www.hyperrealm.com/libconfig/libconfig.html http://www.hyperrealm.com/libconfig/libconfig.html
- alexgartrell 16y ago
  The mongrel2 configuration architecture makes it pretty easy to write a libconfig configuration shell. All it would need to do is translate the libconfig stuff to sqlite. If using libconfig is a good idea, it'll win. P.s. imho, this is the cool thing about the Mongrel2 stuff. The good ideas can always win, because there really just isn't a whole lot to Mongrel2-core, and everything else is decoupled.
  - deleted 16y ago
    [deleted]
- zedshaw 16y ago
  I thought about something like that, but wanted to duplicate the existing config file format so that people's configs would keep working (mostly). It actually had some cool advantages, like a nice psuedo variable system letting you setup variables for reuse in different parts of the config file. I haven't really seen that in other config formats, and it is damn handy.
  - mahmud 16y ago
    libconfig has include directives, is far more useful to multi-instance setups. Plus you get a full type system, all in a 33k binary.
  - 16y ago
fauigerzigerk 16y ago
He's absolutely right. I have come to the conclusion that we must do one thing: Stop sharing. Shared libraries and language runtimes are a relic of the past when memory and storage were orders of magnitude smaller. Bundle all dependencies and throw out the package managers!
- jedbrown 16y ago
  And reinstall your entire operating system when a core library needs a security patch.
  - fauigerzigerk 16y ago
    Why the entire OS? All applications using a particular library would have to be updated individually, which is a drawback. So be it. Every solution has its drawbacks. I suspect that a system with a labyrinthine dependency graph is less robust and less secure. Don't forget that sharing a library doesn't just mean sharing bug fixes, it also means sharing bugs.
  - aidenn0 16y ago
    You mean upgrade applications that depend on that core library. Which you have to do anyway since a patched library will not be bug-compatible with the old one. That's the biggest issue, if you test with library version X and run with library version Y, then any sufficiently complicated program will have bugs that would have been found by testing with the same version of the library you run with.
    - jedbrown 16y ago
      Responsible library maintainers ensure that security fixes and subminor releases are drop-in replacements. No distro rebuilds the world under these circumstances.
bcl 16y ago
FYI - Fedora 13 shipped with Python 2.6, Fedora 14 is going to ship with Python 2.7 and 3.1 in parallel. Not every distribution ships ancient versions. I also recommend using virtualenv and pip, these tools make it much easier to create an easily reproducable Python environment.
- ubernostrum 16y ago
  Not every distribution ships ancient versions. RHEL, meanwhile, does not ship 3.1, 2.7 or 2.6. And to be perfectly honest, if you're seriously distributing Python software you have to take RHEL into account -- "Fedora ships something newer" doesn't help.
  - bcl 16y ago
    The EPEL5 repository has Python 2.6 available - http://download.fedora.redhat.com/pub/epel/5/SRPMS/repoview/python26.html http://download.fedora.redhat.com/pub/epel/5/SRPMS/repoview/...
    - nailer 16y ago
      +1. If you use RHEL, a 'python26' package from a known good source like EPEL is the best place to get a new python that doesn't mess with your existing OS python.
  - jrockway 16y ago
    I don't use Python at all. Despite that, I still somehow have Python 2.6.4 installed on my RHEL4 box at work. (Incidentally, the Linux version number is only slightly higher -- 2.6.9!!!) So it seems like it's possible to get 2.6 on RHEL. I did it, and I hate Python.
    - gizmomagico 16y ago
      I can't help but note that hating Python seems irrational.
- 16y ago
doki_pen 16y ago
It sounds like the correct solution to this problem is a package maintainer for the Mongrel2 package on different OSs. The package maintainers job is to make sure a package builds and runs on a given OS. The reason we have them at all is that it is often a difficult job. Mongrol2 systems like a system application, no?
ZitchDog 16y ago
I definitely would have used Lua in this situation. The Lua runtime is tiny and lightweight. You can write pretty nice config files in Lua which look pretty similar to Zed's too. http://windrealm.com/tutorials/reading-a-lua-configuration-file-from-c.php http://windrealm.com/tutorials/reading-a-lua-configuration-f...
zokier 16y ago
Doesn't every non-stale language suffer from the same thing? So either your choices are a) using language that hasn't evolved or b) using an old version of language which has evolved. I'm not sure that alternative b is that much worse than a.
mcantelon 16y ago
Python needs something like the "Go PHP 5" campaign (which pressured projects/hosting companies to drop support of PHP 4: http://gophp5.org/ http://gophp5.org/).
- briancurtin 16y ago
  What version should be promoted? As much as I like, use, and work on Python 3, a campaign to get distros to use 3.x would leave a lot of projects in the dust, broken and unusable, or resulting in a rushed conversion. On the other side of the coin, a campaign to promote a newer 2.x version is counterproductive to eventually getting everyone on 3.x.
  - mcantelon 16y ago
    The latest v2 would be the easiest step to take.
deleted 16y ago
[deleted]
FraaJad 16y ago
Zed Should have used Lua(the entire lua source distro weighs couple of hundred kilo bytes). Lua is superb embedded language and even better configuration language. No one needs to know it is Lua, (which also happens to be a marketing problem for Lua, heh). His decision not to use Lua the second time around is perplexing because he is not averse to reusing third-party libraries and he is already using the code from Lua for URL pattern matching (IIRC).
- JulianMorrison 16y ago
  Well, he said he made the new parser read the unchanged config file format. Presumably, lua would have needed a change of syntax. Also possibly he decided to step outside the language wars because it's less hassle to write your own parser than wrangle language partisans.
  - zedshaw 16y ago
    Exactly. It was part technical and part social.
- compay 16y ago
  Josh Simmons, one of the main Mongrel2 hackers other than Zed, is already working on that. The idea behind Mongrel2 is that you can write your own configuration system in any language you want, as long as the final result is an SQLite3 database. The one in C that ships with Mongrel2 is "just" a default - if you don't like it, it's not hard to roll your own. Here's an early peek Josh's Lua config tool, and a sample config file: http://gist.github.com/578726 http://gist.github.com/578726
c00p3r 16y ago
Narcissism and self-promotion as usual. People who have any trouble with a system perl or python or java are shipping their own versions within the package. Sap Bussiness Object, for example, comes with its own perl 5.004 and Sun JDK 1.4. It is also possible to provide an up-to-date package for most used OSes. You cannot replace system's default python interpreter, because to much things in distros are depended on it, put it could be /opt/python btw, it is only Debian-based systems and RHEL who are lagging behind. Fedora is always up to date. I'm considering this post as just a flame-generator. There is no problem at all.
points 16y ago
Who cares?
jgalvez 16y ago
I have found that the one system that doesn't give me any trouble running Python is Ubuntu. I'm using Ubuntu Server 10.4, specifically, which is the latest LTS (Long Term Support) version. Ubuntu 10.4 has Python 2.6 but I wanted Python 2.7 (which is the LAST 2.x version). So here's what I did: I installed all common dependencies (such as OpenSSL with "apt-get build-dep python2.6", since they're all the same for Python 2.7. Then I installed Python 2.7 with "make altinstall" without disrupting the system's base 2.6 install. Then, I got pip and virtualenv installed. I've never been happier doing Python development. pip never failed to install a single package I needed, and Python 2.7 is fast and gives me a safe path of migration towards Python 3, whenever the whole byte/str issue in the network libraries (or at least WSGI) is 'resolved'.
- zedshaw 16y ago
  Oh yeah, that's no trouble at all. <shakes head>
  - jgalvez 16y ago
    Oh well, you know, it's definitely not trouble enough to drive me away from Python. Despite the whole version madness across Linux distributions, Python is still my preferred language for web development, and even if that means going through a bit of trouble getting it properly installed on a particular system, hopefully that is something I'll only have to do and document once every few years. I surely hope all distros eventually catch up with Python 2.7 so we can all stop worrying about this. Thanks for taking the time to make some noise about it ;)
    - zedshaw 16y ago
      It's trouble enough to drive people away from Mongrel2. Also, I find it weird that you take me saying "we ditched python in mongrel2 because of Linux" to mean "Python SUCKS! Stop using it." Never said the latter, only the former.
      - jgalvez 16y ago
metamemetics 16y ago
So what can we learn from this for designing future languages? Make it standard idiom to have every file of code declare its version-dialect at the top? Default to throwing warning if interpreter and code version-dialect mismatch? Have the main code interpreter called by all code files simply be a host for routing code to the correct dialect-version interpreter? Perhaps include an optional "slim" download where the host only includes the interpreter for one dialect-version, but make this non-default\opt-in.
- points 16y ago
  > "So what can we learn from this for designing future languages?" We can learn that it's inevitable that your language will come into fashion, be hyped up to be "the language", then will fall out of fashion and be declared 'dead'. The idea of a '100 year language' is solid as long as developers don't choose languages like they choose what to wear. Unfortunately though, most developers seem to choose based on current fashion rather than anything else.
  - metamemetics 16y ago
    Let's try to be constructive here! Come up with solutions and ways to avoid the problems mentioned in the future. There's always fashions and trends sure, but I think most people chose to program their last web app in Python or Ruby over assembly for more practical reasons...
    - points 16y ago
      heh true, although you sure can learn a lot doing things the 'unaccepted' way. So I'd actually advise everyone to try writing a webapp in assembly. If most people tell you what you're doing is crazy, it's often a big sign that you should do it.
      - regularfry 16y ago
        "They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."
baq 16y ago
zed can be a very good troll. not this time, though. what he's saying is the very reason virtualenv exists.
- zedshaw 16y ago
  virtualenv doesn't work for a tool like m2sh, and isn't usable for people using other languages.
kfool 16y ago
I would be very curious to learn why you had not packaged Mongler2 with cx_freeze, which allows you to provide your own copy of Python with any modules you may want to add to it. Doing so would have made your software immune to the broken Python installations.
known 16y ago
Is anybody using Google's http://en.wikipedia.org/wiki/Go_%28programming_language%29 http://en.wikipedia.org/wiki/Go_%28programming_language%29
jrockway 16y ago
All in all, the C code looks pretty good. I browsed around in a few files to see whether he used strn* or strl*, but it turns out he uses a proper string library instead. Very rare, but very nice to see!
nwmcsween 16y ago
Every unix based OS deals with package resolution in a stupid way. Didn't the old lisp machines before Linux, FreeBSD, etc solve this with self contained state in a single object? Everything old and solved is the new and hard problem.
forgottenpaswrd 16y ago
If "python for linux sucks" I don't know what to say about python for mac or windows. I have a lot of python programs that use OpenGL, matplotlib, and a lot other libraries. No problem or very little problems on Linux. On mac 10.6 you have a limited python version and is very difficult to add matplotlib and other libs, with Windows is a similar pain in the ass, unless you buy it from a commercial scientific distro.
JabavuAdams 16y ago
After several issues with Python, Lisp, Haskell, etc., I'm coming around to the philosophy of just generating C or C++ code from Python or Lisp. Productivity is a function of (familiarity, language, libraries, configuration, runtime). Other languages may be great due to (language, library), but configuration can be a show-stopper. Especially since most devs seem to see library/wrapper writing as a chance to pull in all their favourite cruft dependencies.

246 points by spahl 16y ago

deleted 16y ago

[deleted]

what about dumping a local python install in $HOME/local/opt/mongrel-python if m2sh didn't find a version it likes? (yes, it's overkill)

apgwoz 16y ago
This is an insane idea. I should not have to have a whole other version of python just to run a language independent web server. The choice Zed made is a great choice--all the functionality of the original python m2sh is present in the new, and there's nothing external to support, or dump somewhere else.
- lincolnq 16y ago
  It's not that crazy. Python is designed for embedding. The interpreter is small. Commercial software that depends on Python (almost) always embeds it.
  - apgwoz 16y ago
    I know this, however, Zed was using this for a shell command, not as an embedded scripting language.

dschobel 16y ago

Zed's pronouncements are works of art. The man is a pro. A veritable Martin Luther for our times.

auxbuss 16y ago
Indeed. I got to the statement: "The distros have killed Python." Said out loud, "here we go", and settled down for some classic Zed. Can't fault him on it, though.
- ubernostrum 16y ago
  I'd go one further: distros in general haven't killed Python, Red Hat has. By tying their flagship OS to a six-year-old Python release, they've made it impossible for a lot of popular software to move on and take advantage of more recent advances in the language (since "screw everybody who uses Red Hat" simply isn't a viable option).
  - gaius 16y ago
    This is a complete non-issue. ActiveState for example installs Python, Tcl, etc in /opt.
    - jnoller 16y ago
      Except no one installs ActiveState. For most, most of the market, the RedHat supplied version of python is python.
      - joshfinnie 16y ago
        Does RedHat still have that much of a market share in linux to demand such? I guess I am out of the loop, but I hear way more about ubuntu these days than RedHat...
        gaius 16y ago
        If you want to target "Linux" generically then stock Red Hat is your safest bet. I don't know anyone with 10,000 Ubuntu boxes in a datacentre.
        count 16y ago
        The US Govt can use Redhat, but not Ubuntu (Ubuntu hasn't been Common Criteria / EAL tested). That's a pretty huge market right there.

zeemonkee 16y ago

Completely agree. The Linux distros (CentOS is the one I work with most, not out of choice) have antiquated Python versions. You can get round it with virtualenv etc., and I do this as a matter of course when installing Python web applications. For a generic "plug and play" package like Mongrel2 however, where sysadmins just want something that runs straight out of apt-get or whatever, it's a pain and is holding things back. I mean WTF does yum still require ye olde Python 2.4 ?

joelmichael 16y ago
CentOS is about as conservative an option as you can get, though. Unusably so, in my opinion. None of the other popular distros have packages that old.
- zeemonkee 16y ago
  Question to manager: WTF are we using CentOS ? Reply: because it's Enterprise.
  - barfoomoo 16y ago
    Which distro you think is the best to run LAMP stack with an up to date package manager?
    - deleted 16y ago
      [deleted]
    - zeemonkee 16y ago
      Some flavour of Debian would be my preference. Generally with Python WSGI applications I tend to keep things sandboxed using virtualenv and pip in any case.
      - barfoomoo 16y ago
        I am complete newbie when it comes to running a server but have a bit of experience using Ubuntu in my local box. Is ubuntu a good choice? Since most use a flavor of Red Hat like CentOS I am a bit hesitant to use Ubuntu in production.

deleted 16y ago

[deleted]

nailer 16y ago

I understand what Zed's saying here - particularly as I came to programming from a sys admin background. Just like Perl before it, Python (and to some extent, Ruby if you use Puppet) are part of the OS (Linux OS or OS X that is). Shit can and will break if you mess with your OS. You never got this with PHP because people didn't use it in their OSs. But there's a simple solution: not only does the Ministry of Packaging want you to use virtualenv, your OS maker does too. The faster pip and virtualenv become standard the better.

dagw 16y ago
Shit can and will break if you mess with your OS. Updating a program I use should never constitute "messing" with my OS, it should constitute "using". Why should I as an OS user have to constantly worry about upsetting the OS maker? Why should I have to jump through obscure hoops just to install some software without breaking my OS? Wouldn't a better solution be for the OS makers to wall off their magic python behind some wall where I cannot see, touch nor use it, and let me install whatever python I want without fear.
- nailer 16y ago
  Firstly: no hoops are required to install any version of python or any required Python module your app needs: * Just use pip and virtualenv. * Your OS remains intact, and you get better deployment for free! Replacing a component of your OS (Python 2.x) with another one (python 2.x +1 ) affects: * The OS installer (Anaconda on RHEL, Ubuntu'sequivalent) * The packaging system (yum) * Many GNOME system admin apps. * The config management tools (puppet sits on ruby)
  - dagw 16y ago
    My point is that /usr/bin/python probably shouldn't be a core OS component. /usr/bin/python should be whatever I want it to be. The core OS-component should be something like /sys/bin/centos-python which I will never directly use, touch or even necessarily know about. The bottom line I guess is that mixing core OS components and user installed software in the same directory structure with no way of differentiating them is simply bad design.

antileet 16y ago

Hi Zed, Can you please elaborate with some details about how "broken" python is on systems, and the situation with python 2.4? According to the article, "recent Ubuntu releases" had broken/deprecated python installations. A quick search reveals that python 2.5 has been shipping as the default "python" meta-package(?) http://packages.ubuntu.com/search?keywords=python http://packages.ubuntu.com/search?keywords=python Can you also please elaborate any "features" that the install was missing - are these specific modules ?(I think python's sqlite3 is missing by default on Ubuntu. I'm not too sure.) I'm not questioning you or anything, I'm just slightly concerned as we had plans to ship a Python application, and when someone experienced enough has a valid complaint, it makes sense to understand the basis behind that.

Twisol 16y ago
Speaking as someone who knows practically nothing here, I think the issue is existing systems, not brand new systems. The barrier to updating an existing system becomes greater the more and the longer things depend on it.
- deleted 16y ago
  [deleted]
acqq 16y ago
I'd very much like to know why it wasn't easier for him to simply write a Python code that doesn't use anything not existing in 2.4. Just because he downloaded to his computer whatever later version he likes he can't expect from all distros and all users to have exactly the same version across of all the currently running computers, it's more than unrealistic, it's simply impossible and always will be.
- ubernostrum 16y ago
  I'd very much like to know why it wasn't easier for him to simply write a Python code that doesn't use anything not existing in 2.4. This isn't an "oh, a brand-new version just now came out and it's not in the distros yet" problem. Python 2.4 is six years out of date at this point. Python 2.3, which Red Hat will support until 2012, is even older (and for the longest time Red Hat didn't even use a 2.x Python at all -- they sat on 1.5 forever). That's simply unacceptable, and creates endless headaches for people who want to distribute software written in Python.

aphexairlines 16y ago

"older Debian"... but Python 2.5 has been packaged in Debian stable since at least Etch.

apgwoz 16y ago

Hilariously enough, I took the time to make sure the new config file format for m2sh was nearly perfectly backwards compatible with the old file. This is pretty funny because I predict nobody will blink an eye at the config file, even though it's basically Python. This is an interesting point, especially given the amount of code (relatively small) actually needed to parse the subset of Python the config format uses. That said, the subset is fairly "generic scripting language" since there aren't any declarations, only assignments, lists, etc (at least at a cursory overview).

joelmichael 16y ago

I'm not sure why he concerns himself with old distros. Are they the customers for Mongrel2? The description on the website emphasizes modern web technologies. Using Rails 3 is best with Ruby 1.9.2, and yet the only popular distros that install that via their package management system are the rolling release ones, Arch Linux and Gentoo (where it is masked). Most people willing to use this brand new software are also willing to compile their own necessary bleeding edge dependencies, and probably have a generally up-to-date distro preference (like Ubuntu). Even the latest Debian stable, a notoriously conservative distribution, which is a year and a half old and due to be upgraded soon, uses Python 2.6. I'm not sure it's reasonable to support much past that.

vidarh 16y ago
Upgrading Ruby to 1.9.2 won't break your OS. Upgrading Python on some distros can even break package management. That's the difference. And if you have a well tested, rock stable environment you generally don't want to mess with it if you don't have to, or may not easily be able to. You want to not support older than Lenny. Well, we have machines that are still on Sarge, though slowly being upgraded. Some of the machines haven't been rebooted since around the time Etch came out and that's the main reason they haven't been upgraded yet.
- mfukar 16y ago
  That's only because package management "on some distros" (I won't name them, we all know which they are and I'm sure we're talking about the same one) is fubar. I'm not talking about the package manager alone, but also the over-the-top package dependencies, incoherent dependency resolver, let alone the sheer stubborness of some developers working on package management..
- joelmichael 16y ago
  I'm sure older machines are out there and are doing fine for their purpose. But are you planning to install Mongrel2 on them?

mfukar 16y ago

Python, as per Guido Van Rossum's wishes, is pushing for suspension of all language syntax and built-in changes and work towards preserving semantics, precisely for giving time to alternative implementations time to catch up with CPython, as well as the various distributions to catch up with Python's recent progress. In short: "...have killed Python" is a hyperbole for saying "I was tired to hear people nagging about dependency on Python". Still, it's a very good thing what Shaw did.

dschobel 16y ago
I've never heard the distro argument used by any of the python maintainers and I would be surprised if they had. The solution to distros which are bundling three year old software is not to halt your progress for three years, that should be obvious.
- mfukar 16y ago
  I was mentioning it as a side-effect, the main reason was to give time to alternative Python implementations to reach Python 3.1.
- ubernostrum 16y ago
  The solution to distros which are bundling three year old software is not to halt your progress for three years, that should be obvious. Speaking as Django's release manager: Red Hat's continued use and support of ancient 2.x Python versions has been a factor in the schedule we're developing for migration to Python 3.x. We can't simply pull the rug out from under anyone who's using Django on RHEL.
  - dschobel 16y ago
    Seeing RHEL/CentOS keep coming up led me to some googling and the first result for "RHEL python" is http://www.python.org/download/linux/ http://www.python.org/download/linux/ (about as official as it gets) which says The IUS Community Project maintains recent versions of Python for Red Hat Enterprise Linux (RHEL) and CentOS. Installations are parallel installed next to stock versions of python therefore don't disrupt the functionality of critical python utilities on the system. Doesn't that solve the issue and keep you from being hamstrung by those distros?

doki_pen 16y ago

Including Gentoo is wrong. Gentoo uses a slotted ebuild for python. You can have many concurrent versions running. Not only that, but the latest portage works fine on the latest 2.6 release of python. If you don't want to learn how to use your OS, pick a different one. I hear that Macs "Just Work".

tzury 16y ago

I guess it was just about time for me to try mongrel on a fresh new project. I feel I am blessed for that only. Seeing that 0MQ batteries are included is a great bonus. If all would be fine, perhaps my network appliance would be shipped with mongrel2 which would be fantastic.

zedshaw 16y ago
Cool give it a shot. We've got a bunch of stuff we're working on for the 2.0 push but what you have in 1.1 should be stable and usable. If not, let me know by dropping a ticket.

donaq 16y ago

It's sad and ironic (for Python) that Zed actually had to switch from Python to C to make Mongrel more portable. Disclaimer: I love Python, but I hate that anything but the most trivial code won't run across all versions.

rbanffy 16y ago
> I hate that anything but the most trivial code won't run across all versions. Please, define "all versions" and "most trivial code". I have very little problems with Pythons ranging from the ancient 2.4 to modern 2.7. Of course, I am careful with what I do. I know if I use a dictionary comprehension I will be limited to 2.7+, so I try not to. I am quite sure a lot of the code I write could run under any 2.x Python with little modification. As for 1.x, I will agree that things get more complicated.
- donaq 16y ago
  Well, it's a matter of opinion, but I would define "all versions" as versions sharing a major number, i.e. code targeting 2.2 should run on anything starting with 2. As for trivial code, just look at the string operations part of the documentation and search for "new in" and "changed in". My connection is rather slow right now, to the point that the page I mentioned is still loading, so I won't look for more examples, but string processing should IMO be trivial parts of any language. Don't get me wrong, the changes have mostly been useful ones for 2.x versions, but I have been caught testing scripts on my own box that end up not working after deploying. Since then, I have been, as you say, more careful, but I feel I have a valid gripe when I expect string and io operations to be consistent across major versions without having to do a fine print and version check. :)
- zedshaw 16y ago
  generators
  - rbanffy 16y ago
    I know that if I really need generators, then I really need to install something newer than the 2.4 that comes with RHEL/CentOS 4. Generators is 2.6+, right? BTW, an ugly approach I used (more or less as a joke) was to check for generator availability and, in case I can't use them, use a list comprehension instead. There is a performance/memory hit but, depending on what you are doing, it's a usable alternative. And a nice place to hang humorous comments. But I agree that, if your users don't want to install an alien (from the package manager POV) Python in order to run Mongrel, getting rid of python code is the right thing to do. It's like having a weird dependency, kind of requiring a Fortran 77 compiler in order to run Perl...

deleted 16y ago

[deleted]

rbanffy 16y ago

It's funny how, for Zed, anything that doesn't work the way he thinks it's supposed to work is "broken" and the result of some boneheaded decision and that it "sucks". I too am frustrated by ancient versions of server OSs that use equally ancient versions of Python, but that's why virtualenv was invented. And source distributions. I am happy using it when required. Being "stable" in OS terms is not changing APIs very often. One of the good things is that I know that code I wrote for RHEL 4 the day it was launched will run flawlessly on RHEL 4 today. The primary goal of such a Linux distro is that "it has to work". I won't touch the system parts.

Confusion 16y ago
He doesn't blame it on 'some boneheaded decision'. As far as I can see, he doesn't blame anyone, he just notes the current state of affairs, in which the situation is indeed pretty much broken. Not just for him, because he is far from the only one that thinks it should work a certain way. that's why virtualenv was invented Yes and shouldn't it also be the OSes that use it to isolate their default Python install?
- rbanffy 16y ago
  It's a design decision. They have a default Python install they support, validate and keep working that you shouldn't mess with. I am pretty sure you can't ruin a RHEL just by installing RPMs from RH's repos. If the Python the OS vendor provides is somewhat inadequate to you, it's your job to provide another one or to use another distro. However, if you choose to write your code against the distro's Python (something that's not that hard, really - 2.4 is a modern language) you can rest assured your code will work for as long as the distro is supported. It's not "broken", not a boneheaded decision. It's just the way it is. If you require 2.7 or 3.x goodness, you can always set-up different environments separate from the OS's Python and just be happy with it. And, BTW, I never put boneheaded in quotes ;-)
  - sophacles 16y ago
    It is completely possible, easy and reasonable to have a multiple versions of python installed. Many OSes do this, and the convention is to have them named like python2.4 python2.5 and so on. Any system scripts that require 2.4 can then change their shebang to say /usr/bin/python2.4. At the very same time, packages for newer pythons can be made available for ease-of-use for the rest of the world. This breaks nothing.

gfunk911 16y ago

This thread is a classic example of something that happens over and over. Someone says something is hard to use. 1/3 of the responses agree. 1/3 of the responses disagree, say it's simple. 1/3 of the responses say it's simple, all you have to do is use additional software package Y to manage X and it works great. This is usually a sign that there is some merit to the original assertion.

dschobel 16y ago
We're talking about technology, not politics so consensus be damned. There is a fact of the matter independent of perception and lack of consensus in and of itself is not a valid attack on a position. In this case maybe we can listen to those who claim the python situation isn't that bad and have provided specific technical solutions for evaluation.
- gfunk911 16y ago
  If the question is "Do a significant number of users run into problems with a piece of software," then the consensus is absolutely relevant.
  - dschobel 16y ago
    The question isn't whether there is a problem but whether there is a solution. The virtualenv/pip people seem to think so.
    - masklinn 16y ago
      > The virtualenv/pip people seem to think so. No. Unless those people are morons, but the guys who actually code them know they're tools for python developers, and maybe for sysadmins of python solutions. Pip and virtualenv for software using Python as a config file format? not a good idea.
    - zedshaw 16y ago
      Yes, pip/virtualenv works for installing things like Django. It totally fails for something like m2sh which has to live in the system PATH so that you can run it from wherever you have your configs. But you know, I wonder if the various distros could sort of invert this and they start using pip/virtualenv instead of everyone else working around them?

Tichy 16y ago

Shouldn't python just be a library dependency like any other? I was hoping the problem of libraries with different versions was solved somehow. Apparently not :-(

terra_t 16y ago

Python brought this on itself by having no respect for compatibility between versions. There's really no such thing as the Python language, only "Python 2.2", "Python 2.4", "Python 3". If you want to run Python scripts, you need to have three or four different Python runtimes installed, and that's asking a lot of both the distros and of users who need to keep everything straight, and probably even hack the #! at the beginning of scripts so that they work correctly.

jnoller 16y ago
Bullshit. We (python-core) are obsessed with backwards compatibility. Adding new features that aren't in old version is the source of this problem. Python 3 is by far, the biggest intentional deviation from the "don't break existing scripts" mantra that's been in place for years.
- terra_t 16y ago
  Why didn't the official bittorrent client run on Red Hat Linux (w/o downgrading your Python) for at least three years? Why was NLTK stuck on Python 2.4 for years? I might use Python to use NLTK, but why do I have to choose an old Python to get it to work... And what if I want to use it w/ software that needs Python 2.6?
BrandonM 16y ago
When you add features as useful as set() datatypes, if-else expressions, generators, and comprehensions, people are going to use them. Python 2.2 code nearly always runs without modification on Python 2.3+ (the only exception would be collisions between variable names and new keywords). But someone targeting Python 2.6 has a huge hassle when deploying to older versions. This means that Python is actually adding useful new functionality; that is, the incompatibility is a symptom of a good thing.
stevelosh 16y ago
Mercurial runs unchanged on 2.4 to 2.7, and it's not a trivial little piece of software. Pretty much the only thing you need to do to make your Python code run on 2.x+ is "don't use features released after 2.x".

jedbrown 16y ago

This is a major problem. PETSc uses Python for configuration and we have to be careful not to use any features requiring >2.3. The last release (this spring) required 2.2 because RHEL3 is still supported (in production phase 3, see http://www.redhat.com/security/updates/errata/ http://www.redhat.com/security/updates/errata/) until Oct 31, 2010. Python is easy to install, but it's a lot to ask of users when they just want to get your software up and running. Note that RHEL4 sees end of life Feb 29, 2012 and RHEL5 not until Mar 31, 2014. So Python-2.4 support will have to live for a very long time. I hate this, but don't see a decent alternative. Many of our users are building in batch environments of varying ages and just want to get their science done. Shell/bash is not very portable, it would be painful to write all the configuration in C (and confusing because people would need to both native compilers and cross-compilers).

igravious 16y ago
This is very interesting and is what RVM in the Ruby world was designed to solve in some way. So maybe we need a widespread and useful PVM. This feeds into an observation of mine I made over five years ago but that seems to be ever more relevant as time goes by. Linux and BSD seem to have "solved" the packaging problem through mega-repositories with dependencies and access to source code. However your favorite language and especially the scripting ones and virtual machines one will have there own repositories that are _not_ linked into the systems way of doing things. So at the system level we have RPM and Yum and Ports and Portage and Macports and Fink and Homebrew and AptGet and Conary and yada yada yada but at the language level we have (deep breath) Perl and CPAN, Ruby and GEMS, Python and EGGS, and Emacs and ? and Java and Eclipse plugins and on and on. I would switch IN AN INSTANT to the distro that integrated all this so that I would only need to go to ONE place to upgrade system packages and language packages. Honestly if nobody does it one of these years I might even be forced to do it myself and make mega $ € ¥. Hope this makes sense to somebody somewhere. :)
- jedbrown 16y ago
  have there own repositories that are _not_ linked into the systems way of doing things One approach is to have a system that packages everything in the language-specific repository for the distro. Haskell on Archlinux is an example of this. I'm not a fan of actually using the language-specific packages because I like the rolling release model and end up having to do some manual work to reinstall all my packages when the distro upgrades it's copy (or switch to the next 6-month release with Ubuntu and similar).

deleted 16y ago

[deleted]

mahmud 16y ago

Instead of inventing your own syntax, why didn't you use a portable syntax like XML, JSON, YAML, or better yet, libconfig[1]? Please compare the libconfig test file to the mongrel configuration file: http://www.hyperrealm.com/libconfig/test.cfg.txt http://www.hyperrealm.com/libconfig/test.cfg.txt http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-190003.3 http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-190003.... The mongrel conf is a subset of libconfig; it doesn't use the more advanced features. Plus libconfig has bindings and versions of it in nearly every mainstream language. [1]http://www.hyperrealm.com/libconfig/libconfig.html http://www.hyperrealm.com/libconfig/libconfig.html

alexgartrell 16y ago
The mongrel2 configuration architecture makes it pretty easy to write a libconfig configuration shell. All it would need to do is translate the libconfig stuff to sqlite. If using libconfig is a good idea, it'll win. P.s. imho, this is the cool thing about the Mongrel2 stuff. The good ideas can always win, because there really just isn't a whole lot to Mongrel2-core, and everything else is decoupled.
- deleted 16y ago
  [deleted]
zedshaw 16y ago
I thought about something like that, but wanted to duplicate the existing config file format so that people's configs would keep working (mostly). It actually had some cool advantages, like a nice psuedo variable system letting you setup variables for reuse in different parts of the config file. I haven't really seen that in other config formats, and it is damn handy.
- mahmud 16y ago
  libconfig has include directives, is far more useful to multi-instance setups. Plus you get a full type system, all in a 33k binary.
- 16y ago

fauigerzigerk 16y ago

He's absolutely right. I have come to the conclusion that we must do one thing: Stop sharing. Shared libraries and language runtimes are a relic of the past when memory and storage were orders of magnitude smaller. Bundle all dependencies and throw out the package managers!

jedbrown 16y ago
And reinstall your entire operating system when a core library needs a security patch.
- fauigerzigerk 16y ago
  Why the entire OS? All applications using a particular library would have to be updated individually, which is a drawback. So be it. Every solution has its drawbacks. I suspect that a system with a labyrinthine dependency graph is less robust and less secure. Don't forget that sharing a library doesn't just mean sharing bug fixes, it also means sharing bugs.
- aidenn0 16y ago
  You mean upgrade applications that depend on that core library. Which you have to do anyway since a patched library will not be bug-compatible with the old one. That's the biggest issue, if you test with library version X and run with library version Y, then any sufficiently complicated program will have bugs that would have been found by testing with the same version of the library you run with.
  - jedbrown 16y ago
    Responsible library maintainers ensure that security fixes and subminor releases are drop-in replacements. No distro rebuilds the world under these circumstances.

bcl 16y ago

FYI - Fedora 13 shipped with Python 2.6, Fedora 14 is going to ship with Python 2.7 and 3.1 in parallel. Not every distribution ships ancient versions. I also recommend using virtualenv and pip, these tools make it much easier to create an easily reproducable Python environment.

ubernostrum 16y ago
Not every distribution ships ancient versions. RHEL, meanwhile, does not ship 3.1, 2.7 or 2.6. And to be perfectly honest, if you're seriously distributing Python software you have to take RHEL into account -- "Fedora ships something newer" doesn't help.
- bcl 16y ago
  The EPEL5 repository has Python 2.6 available - http://download.fedora.redhat.com/pub/epel/5/SRPMS/repoview/python26.html http://download.fedora.redhat.com/pub/epel/5/SRPMS/repoview/...
  - nailer 16y ago
    +1. If you use RHEL, a 'python26' package from a known good source like EPEL is the best place to get a new python that doesn't mess with your existing OS python.
- jrockway 16y ago
  I don't use Python at all. Despite that, I still somehow have Python 2.6.4 installed on my RHEL4 box at work. (Incidentally, the Linux version number is only slightly higher -- 2.6.9!!!) So it seems like it's possible to get 2.6 on RHEL. I did it, and I hate Python.
  - gizmomagico 16y ago
    I can't help but note that hating Python seems irrational.
16y ago

doki_pen 16y ago

It sounds like the correct solution to this problem is a package maintainer for the Mongrel2 package on different OSs. The package maintainers job is to make sure a package builds and runs on a given OS. The reason we have them at all is that it is often a difficult job. Mongrol2 systems like a system application, no?

ZitchDog 16y ago

I definitely would have used Lua in this situation. The Lua runtime is tiny and lightweight. You can write pretty nice config files in Lua which look pretty similar to Zed's too. http://windrealm.com/tutorials/reading-a-lua-configuration-file-from-c.php http://windrealm.com/tutorials/reading-a-lua-configuration-f...

zokier 16y ago

Doesn't every non-stale language suffer from the same thing? So either your choices are a) using language that hasn't evolved or b) using an old version of language which has evolved. I'm not sure that alternative b is that much worse than a.

mcantelon 16y ago

Python needs something like the "Go PHP 5" campaign (which pressured projects/hosting companies to drop support of PHP 4: http://gophp5.org/ http://gophp5.org/).

briancurtin 16y ago
What version should be promoted? As much as I like, use, and work on Python 3, a campaign to get distros to use 3.x would leave a lot of projects in the dust, broken and unusable, or resulting in a rushed conversion. On the other side of the coin, a campaign to promote a newer 2.x version is counterproductive to eventually getting everyone on 3.x.
- mcantelon 16y ago
  The latest v2 would be the easiest step to take.

deleted 16y ago

[deleted]

FraaJad 16y ago

Zed Should have used Lua(the entire lua source distro weighs couple of hundred kilo bytes). Lua is superb embedded language and even better configuration language. No one needs to know it is Lua, (which also happens to be a marketing problem for Lua, heh). His decision not to use Lua the second time around is perplexing because he is not averse to reusing third-party libraries and he is already using the code from Lua for URL pattern matching (IIRC).

JulianMorrison 16y ago
Well, he said he made the new parser read the unchanged config file format. Presumably, lua would have needed a change of syntax. Also possibly he decided to step outside the language wars because it's less hassle to write your own parser than wrangle language partisans.
- zedshaw 16y ago
  Exactly. It was part technical and part social.
compay 16y ago
Josh Simmons, one of the main Mongrel2 hackers other than Zed, is already working on that. The idea behind Mongrel2 is that you can write your own configuration system in any language you want, as long as the final result is an SQLite3 database. The one in C that ships with Mongrel2 is "just" a default - if you don't like it, it's not hard to roll your own. Here's an early peek Josh's Lua config tool, and a sample config file: http://gist.github.com/578726 http://gist.github.com/578726

c00p3r 16y ago

Narcissism and self-promotion as usual. People who have any trouble with a system perl or python or java are shipping their own versions within the package. Sap Bussiness Object, for example, comes with its own perl 5.004 and Sun JDK 1.4. It is also possible to provide an up-to-date package for most used OSes. You cannot replace system's default python interpreter, because to much things in distros are depended on it, put it could be /opt/python btw, it is only Debian-based systems and RHEL who are lagging behind. Fedora is always up to date. I'm considering this post as just a flame-generator. There is no problem at all.

points 16y ago

Who cares?

jgalvez 16y ago

I have found that the one system that doesn't give me any trouble running Python is Ubuntu. I'm using Ubuntu Server 10.4, specifically, which is the latest LTS (Long Term Support) version. Ubuntu 10.4 has Python 2.6 but I wanted Python 2.7 (which is the LAST 2.x version). So here's what I did: I installed all common dependencies (such as OpenSSL with "apt-get build-dep python2.6", since they're all the same for Python 2.7. Then I installed Python 2.7 with "make altinstall" without disrupting the system's base 2.6 install. Then, I got pip and virtualenv installed. I've never been happier doing Python development. pip never failed to install a single package I needed, and Python 2.7 is fast and gives me a safe path of migration towards Python 3, whenever the whole byte/str issue in the network libraries (or at least WSGI) is 'resolved'.

zedshaw 16y ago
Oh yeah, that's no trouble at all. <shakes head>
- jgalvez 16y ago
  Oh well, you know, it's definitely not trouble enough to drive me away from Python. Despite the whole version madness across Linux distributions, Python is still my preferred language for web development, and even if that means going through a bit of trouble getting it properly installed on a particular system, hopefully that is something I'll only have to do and document once every few years. I surely hope all distros eventually catch up with Python 2.7 so we can all stop worrying about this. Thanks for taking the time to make some noise about it ;)
  - zedshaw 16y ago
    It's trouble enough to drive people away from Mongrel2. Also, I find it weird that you take me saying "we ditched python in mongrel2 because of Linux" to mean "Python SUCKS! Stop using it." Never said the latter, only the former.
    - jgalvez 16y ago

metamemetics 16y ago

So what can we learn from this for designing future languages? Make it standard idiom to have every file of code declare its version-dialect at the top? Default to throwing warning if interpreter and code version-dialect mismatch? Have the main code interpreter called by all code files simply be a host for routing code to the correct dialect-version interpreter? Perhaps include an optional "slim" download where the host only includes the interpreter for one dialect-version, but make this non-default\opt-in.

points 16y ago
> "So what can we learn from this for designing future languages?" We can learn that it's inevitable that your language will come into fashion, be hyped up to be "the language", then will fall out of fashion and be declared 'dead'. The idea of a '100 year language' is solid as long as developers don't choose languages like they choose what to wear. Unfortunately though, most developers seem to choose based on current fashion rather than anything else.
- metamemetics 16y ago
  Let's try to be constructive here! Come up with solutions and ways to avoid the problems mentioned in the future. There's always fashions and trends sure, but I think most people chose to program their last web app in Python or Ruby over assembly for more practical reasons...
  - points 16y ago
    heh true, although you sure can learn a lot doing things the 'unaccepted' way. So I'd actually advise everyone to try writing a webapp in assembly. If most people tell you what you're doing is crazy, it's often a big sign that you should do it.
    - regularfry 16y ago
      "They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."

baq 16y ago

zed can be a very good troll. not this time, though. what he's saying is the very reason virtualenv exists.

zedshaw 16y ago
virtualenv doesn't work for a tool like m2sh, and isn't usable for people using other languages.

kfool 16y ago

I would be very curious to learn why you had not packaged Mongler2 with cx_freeze, which allows you to provide your own copy of Python with any modules you may want to add to it. Doing so would have made your software immune to the broken Python installations.

known 16y ago

Is anybody using Google's http://en.wikipedia.org/wiki/Go_%28programming_language%29 http://en.wikipedia.org/wiki/Go_%28programming_language%29

jrockway 16y ago

All in all, the C code looks pretty good. I browsed around in a few files to see whether he used strn* or strl*, but it turns out he uses a proper string library instead. Very rare, but very nice to see!

nwmcsween 16y ago

Every unix based OS deals with package resolution in a stupid way. Didn't the old lisp machines before Linux, FreeBSD, etc solve this with self contained state in a single object? Everything old and solved is the new and hard problem.

forgottenpaswrd 16y ago

If "python for linux sucks" I don't know what to say about python for mac or windows. I have a lot of python programs that use OpenGL, matplotlib, and a lot other libraries. No problem or very little problems on Linux. On mac 10.6 you have a limited python version and is very difficult to add matplotlib and other libs, with Windows is a similar pain in the ass, unless you buy it from a commercial scientific distro.

JabavuAdams 16y ago

After several issues with Python, Lisp, Haskell, etc., I'm coming around to the philosophy of just generating C or C++ code from Python or Lisp. Productivity is a function of (familiarity, language, libraries, configuration, runtime). Other languages may be great due to (language, library), but configuration can be a show-stopper. Especially since most devs seem to see library/wrapper writing as a chance to pull in all their favourite cruft dependencies.