3 ms·

Perl, really? Just out of curiousity, what were your reasons? The close ties to shell scripting?

by aston 19y ago

staunch 19y ago
Why are you surprised that someone would choose Perl to build the ultimate sysadmin-oriented application? It's the obvious choice for anyone making this decision without regard to what's trendy.
- aston 19y ago
  I disagree. It's not obvious to me at all. Which is why I asked for their reasons.
  - SwellJoe 19y ago
    What do you believe would be the obvious choice for a web-based application that administers everything on a UNIX system? Keep in mind: It runs as root. Directly parses and edits every configuration file that is managed--this is not a "generate config files from a database plus templates" tool--it has to understand the configuration files and edit them cleanly. Strong regex support is vital. It is internationalized to dozens of languages...it needs to support Unicode. It has to be reasonably fast...it serves dozens of users simultaneously, often doing rather complicated tasks. It needs to be a widely available and stable language--we support dozens of platforms (nearly all Linux distros, Mac OS X, the BSDs, and even Windows to some degree). All of them have different versions of Perl, Python, Ruby, PHP etc. and varying degrees of library support "out of the box". We need to know that whatever reasonably recent version exists will work fine now and in the future.
    - aston 19y ago
      I would've probably picked Python, mostly because you get all of that stuff with (almost) uniformly better syntax. But I certainly don't blame you, especially in light of the details from above.
      - SwellJoe 19y ago
        If we were starting Webmin, Virtualmin and Usermin today, it could very well be in Python. We both like it as a language, and have worked in it on pretty large projects. And the syntax is certainly cleaner and simpler. I'm also quite fond of Ruby, though its standard implementation leaves a lot to be desired (performance, Unicode, lack of libraries). But, we'd have to spend some time with it to know for sure...as I mentioned in the longer explanation in this thread, I've done quite a bit of sysadmin tasks in Python (modifying others code, generally), and in every single case I've felt like the implementation was longer than it needed to be, and more complex. In a couple of cases, I even talked it over with seriously good Python guys and did comparative implementations to try and improve my own grasp of Python, and the response was generally something like: "Sure, the Python implementation is twice as long, and runs slower, but I can't read the Perl implementation!" Which is a valid response, of course, but I think just showed that they had never worked in Perl and had little comprehension of its idioms. But, to be fair, I've been pleasantly surprised by some of the Python code I've had to modify. Perhaps the single most satisfying patch I ever wrote for someone else's code was to add authentication support back into yum after they moved from urllib to urllib2. It was the first Python code I'd ever touched, and it was a very pleasant learning experience. Really nice code.
SwellJoe 19y ago
Several reasons: The most compelling, and inarguable, being that our codebase is approaching ten years old (Webmin, with about 300k lines of code, is the core of our product). Ten years ago, Perl was the clear choice for web applications and system administration applications, so it was a double win. Several "competing" Open Source and proprietary products in other languages, including Python and PHP, have come and gone during that time, and we expect more to come and go in the future. Re-writes of a large codebase merely to follow language fashion are idiotic, and we're not idiots. Jamie and I have frequently worked in other languages, with Python having been used heavily by both of us. I'm a bit of a language geek and so I spend a few days every month with something new (I'm playing with Haskell right now). We're not Perl-only developers, but in hindsight Perl has served Webmin very well, and it continues to do so. It has some very nice functional constructs that make the kinds of data processing we do simple and short...we make heavy use of map, grep, first class functions, etc. and it's possible to treat Perl arrays like Lisp lists, which can be very powerful. We'll both admit to liking Python syntax. We might even admit to liking it better than Perl, after we've had to dig into one of the older modules in Webmin and interpret some of the squiggles Jamie used back then. I definitely like Ruby syntax better. But, although I've said that we aren't Perl-only developers, it is worth mentioning that Jamie (the primary developer on all of our products) is fantastically proficient with Perl. It's a language he's been working with full-time for his entire professional career, and it shows. He can produce, in hours, what I've witnessed other projects spend weeks or months creating badly. Perl is extremely expressive, quite concise, is fast enough for almost everything we do, has great Unicode support, threads and concurrency that works, and an extremely mature and stable implementation. We can count on our software running on just about any recent or future version of Perl. This is not true of Python, Ruby, or PHP (with PHP being the most egregious example of backward-compatibility failure--I've done enough work in all three to know that Perl wins hands down in this area). Java is pretty stable, but would require a lot more code, and it isn't as widely available on Linux systems. "close ties to shell scripting" doesn't really play much of a role, though the close ties to system administration does. Perl evolved out of system administration and text processing tasks, so it has a lot of great features for those tasks. As mentioned, Perl's grep is great for that kind of job, as are map, split, etc. It's nice to have iterators that already exist for the exact tasks we're doing. Sure, we could write our own in other languages (assuming they have the constructs needed to do it cleanly...so Python and Ruby, but probably not PHP), but we don't have to in Perl. I've worked on system administration tools in Python in the past, having modified yum, trac, Subversions post-commit mailer.py script, and Mailman for various custom tasks, and I generally come out the other side thinking, "I could have done that faster, and cleaner, in Perl". This reflects my own weakness in Python nearly as much as the strengths of Perl over Python, but I don't believe it is entirely my lack of proficiency. Perl just does some things better, and system administration is one of them. I'm actually in the midst of choosing a language for new hosted applications, and so I'm thinking a lot about whether to stick with Perl or branch out some. We are no longer really tied to Perl--even with things that go out to customers--as Webmin has an XML-RPC interface, which can be used from any language with reasonable support for XML and sockets (pretty much all modern languages). But for hosted apps we're entirely free to choose. So, the UI portion of it will be implemented in PHP, to ease integration with our Joomla-based website...since I thoroughly dislike working in PHP, the other components of those apps will be in another language...probably Perl, but I'm toying with Erlang and Haskell. We'll see. In short, saying "Perl, really?", kind of indicates a bias against Perl that is unjustified. Sure, many have trouble with the syntax. It can be intimidating. I've been working with it for years, and I generally can't write more than 10 lines without a syntax error! And, much Perl code in the wild is ugly...even some of Webmin's older code that we have to work with (and some of the newer code that I write that Jamie hasn't fixed yet!). But, it's among the most powerful mainstream languages. Because the folks making decisions about Perl don't have Guido's or Matz' devotion to one particular development model, Perl gets everything--every cool idea in software development ever thought of has an implementation in Perl. You can code entirely in a functional style in Perl. You can do OO. You've got first class functions and closures. TMTOWTDI is a double edged sword, we can all agree, but if used wisely, it cuts through an awful lot of crap really fast. ;-)

Perl, really? Just out of curiousity, what were your reasons? The close ties to shell scripting?

by aston 19y ago

Perl, really? Just out of curiousity, what were your reasons? The close ties to shell scripting?

Why are you surprised that someone would choose Perl to build the ultimate sysadmin-oriented application? It's the obvious choice for anyone making this decision without regard to what's trendy.

aston 19y ago
I disagree. It's not obvious to me at all. Which is why I asked for their reasons.
- SwellJoe 19y ago
  What do you believe would be the obvious choice for a web-based application that administers everything on a UNIX system? Keep in mind: It runs as root. Directly parses and edits every configuration file that is managed--this is not a "generate config files from a database plus templates" tool--it has to understand the configuration files and edit them cleanly. Strong regex support is vital. It is internationalized to dozens of languages...it needs to support Unicode. It has to be reasonably fast...it serves dozens of users simultaneously, often doing rather complicated tasks. It needs to be a widely available and stable language--we support dozens of platforms (nearly all Linux distros, Mac OS X, the BSDs, and even Windows to some degree). All of them have different versions of Perl, Python, Ruby, PHP etc. and varying degrees of library support "out of the box". We need to know that whatever reasonably recent version exists will work fine now and in the future.
  - aston 19y ago
    I would've probably picked Python, mostly because you get all of that stuff with (almost) uniformly better syntax. But I certainly don't blame you, especially in light of the details from above.
    - SwellJoe 19y ago
      If we were starting Webmin, Virtualmin and Usermin today, it could very well be in Python. We both like it as a language, and have worked in it on pretty large projects. And the syntax is certainly cleaner and simpler. I'm also quite fond of Ruby, though its standard implementation leaves a lot to be desired (performance, Unicode, lack of libraries). But, we'd have to spend some time with it to know for sure...as I mentioned in the longer explanation in this thread, I've done quite a bit of sysadmin tasks in Python (modifying others code, generally), and in every single case I've felt like the implementation was longer than it needed to be, and more complex. In a couple of cases, I even talked it over with seriously good Python guys and did comparative implementations to try and improve my own grasp of Python, and the response was generally something like: "Sure, the Python implementation is twice as long, and runs slower, but I can't read the Perl implementation!" Which is a valid response, of course, but I think just showed that they had never worked in Perl and had little comprehension of its idioms. But, to be fair, I've been pleasantly surprised by some of the Python code I've had to modify. Perhaps the single most satisfying patch I ever wrote for someone else's code was to add authentication support back into yum after they moved from urllib to urllib2. It was the first Python code I'd ever touched, and it was a very pleasant learning experience. Really nice code.

SwellJoe 19y ago

Several reasons: The most compelling, and inarguable, being that our codebase is approaching ten years old (Webmin, with about 300k lines of code, is the core of our product). Ten years ago, Perl was the clear choice for web applications and system administration applications, so it was a double win. Several "competing" Open Source and proprietary products in other languages, including Python and PHP, have come and gone during that time, and we expect more to come and go in the future. Re-writes of a large codebase merely to follow language fashion are idiotic, and we're not idiots. Jamie and I have frequently worked in other languages, with Python having been used heavily by both of us. I'm a bit of a language geek and so I spend a few days every month with something new (I'm playing with Haskell right now). We're not Perl-only developers, but in hindsight Perl has served Webmin very well, and it continues to do so. It has some very nice functional constructs that make the kinds of data processing we do simple and short...we make heavy use of map, grep, first class functions, etc. and it's possible to treat Perl arrays like Lisp lists, which can be very powerful. We'll both admit to liking Python syntax. We might even admit to liking it better than Perl, after we've had to dig into one of the older modules in Webmin and interpret some of the squiggles Jamie used back then. I definitely like Ruby syntax better. But, although I've said that we aren't Perl-only developers, it is worth mentioning that Jamie (the primary developer on all of our products) is fantastically proficient with Perl. It's a language he's been working with full-time for his entire professional career, and it shows. He can produce, in hours, what I've witnessed other projects spend weeks or months creating badly. Perl is extremely expressive, quite concise, is fast enough for almost everything we do, has great Unicode support, threads and concurrency that works, and an extremely mature and stable implementation. We can count on our software running on just about any recent or future version of Perl. This is not true of Python, Ruby, or PHP (with PHP being the most egregious example of backward-compatibility failure--I've done enough work in all three to know that Perl wins hands down in this area). Java is pretty stable, but would require a lot more code, and it isn't as widely available on Linux systems. "close ties to shell scripting" doesn't really play much of a role, though the close ties to system administration does. Perl evolved out of system administration and text processing tasks, so it has a lot of great features for those tasks. As mentioned, Perl's grep is great for that kind of job, as are map, split, etc. It's nice to have iterators that already exist for the exact tasks we're doing. Sure, we could write our own in other languages (assuming they have the constructs needed to do it cleanly...so Python and Ruby, but probably not PHP), but we don't have to in Perl. I've worked on system administration tools in Python in the past, having modified yum, trac, Subversions post-commit mailer.py script, and Mailman for various custom tasks, and I generally come out the other side thinking, "I could have done that faster, and cleaner, in Perl". This reflects my own weakness in Python nearly as much as the strengths of Perl over Python, but I don't believe it is entirely my lack of proficiency. Perl just does some things better, and system administration is one of them. I'm actually in the midst of choosing a language for new hosted applications, and so I'm thinking a lot about whether to stick with Perl or branch out some. We are no longer really tied to Perl--even with things that go out to customers--as Webmin has an XML-RPC interface, which can be used from any language with reasonable support for XML and sockets (pretty much all modern languages). But for hosted apps we're entirely free to choose. So, the UI portion of it will be implemented in PHP, to ease integration with our Joomla-based website...since I thoroughly dislike working in PHP, the other components of those apps will be in another language...probably Perl, but I'm toying with Erlang and Haskell. We'll see. In short, saying "Perl, really?", kind of indicates a bias against Perl that is unjustified. Sure, many have trouble with the syntax. It can be intimidating. I've been working with it for years, and I generally can't write more than 10 lines without a syntax error! And, much Perl code in the wild is ugly...even some of Webmin's older code that we have to work with (and some of the newer code that I write that Jamie hasn't fixed yet!). But, it's among the most powerful mainstream languages. Because the folks making decisions about Perl don't have Guido's or Matz' devotion to one particular development model, Perl gets everything--every cool idea in software development ever thought of has an implementation in Perl. You can code entirely in a functional style in Perl. You can do OO. You've got first class functions and closures. TMTOWTDI is a double edged sword, we can all agree, but if used wisely, it cuts through an awful lot of crap really fast. ;-)