6 ms·

I switched to writing python as my main backend/scripting language with my new job. Previously I was using Ruby for most of this stuff with some Go. I must say

by K0nserv 10y ago

I switched to writing python as my main backend/scripting language with my new job. Previously I was using Ruby for most of this stuff with some Go. I must say that so far Python has been a vastly inferior experience for me. The languages are fairly similar all though I prefer the more talkative/english style of ruby and the use of functions over list and dict comprehension. The thing that really stands out to me though is the poor state of python tooling and the library ecosystem. Having used Rubygems and bundler pip feels like taking an enormous step back. It's much less expressive. It doesn't handle the difference between production only dependencies, regular dependencies and development only dependencies in a good way. It's difficult to differentiate between locked dependencies vs desired dependencies(the Gemfile/Gemfile.lock distinction). PyPi and especially using private PyPi registers is more complex than it is in Ruby. There seems to be fewer nice libraries and they seem to be spread across the web whereas Ruby centralises around GitHub. I also find that Python libraries have lacking or hard to find documentation in many cases. I wonder how much of this is due to the community spending a lot of time on the unproductive Python2/Python3 debacle instead of moving the language and the ecosystem forward. It seems like Ruby and Python would be in more similar positions had it not been for the problems around the Python 2/Python 3 upgrade. Ruby had the risk of a similar problems in the 1.8 to 1.9 upgrade, but it seems to have worked out better than the Python 2/Python 3 upgrade.

korijn 10y ago
Check out conda sometime.
- K0nserv 10y ago
  That looks interesting, I've also tried out pip-tools[0] which is really nice. However nice these tools are it seems to a bit at odds with the Zen of Python. Ideally there should be just one single good way to deal with packages. Ironically this is the case for Ruby, but not at all for Python. Ultimately you can always use some alternative method that solves it better, but you then have to take on the burden of convincing your co-contributors that it is a better idea than using pip and you also have to explain it to anyone that joins your project. 0: https://github.com/nvie/pip-tools https://github.com/nvie/pip-tools
- IndianAstronaut 10y ago
  It can conflict with any native Python installations. So if you are running any pre built Python libraries which point to any system wide Python, it can be a nightmare. Hue is one example I ran into.
curiousssly 10y ago
"There seems to be fewer nice libraries and they seem to be spread across the web whereas Ruby centralises around GitHub. I also find that Python libraries have lacking or hard to find documentation in many cases." Could you clarify a bit more on this with the real world examples ?
- K0nserv 10y ago
  Well for the case of "nicer libraries" I suppose I mean high quality libraries. Also good to note that I am referring to libraries in the web development/backend sphere here. I have a hard time thinking of some real world examples of there being a larger quantity of quality libraries for ruby, it's more a feeling that I get. Around documentation and usage I think it comes down to the fact that for Python libraries a google search usually takes you to the modules page on the PyPi registry, to a project website or a read the docs instance. I usually find that where ever I end up doesn't provide all the information I want, e.g what's the latest version, what are some typical use cases for this library. As a concrete example let's compare Flask and Sinatra. For both Flask and Sinatra a google search gives me their respective websites as the top relevant result. In this case I find both websites don't really provide all the information I am after. When I navigate to their GitHub page however the difference is more stark. Sinatra uses GitHub flavoured markdown to render a nice guide with increasingly more complex use cases. Flask just has some non markdown text pointing me in a few different directions. I could probably find all the information I need to setup and use Sinatra from just the GitHub Readme whereas for Flask I'd have to go navigate their read the docs page where all the documentation and usage instructions are spread over multiple pages and sub pages. I guess it comes down to my preference for the more concise format of a long markdown README.md vs the experience with a read the docs like set of pages. I recon this is a bit rambling, it's kinda hard to put into words what my feelings around this are and why I hold the opinions I do.
euroclydon 10y ago
I wish Ruby had the math and scientific library support of Python. Also, I've been using Jupyter lately to brush up on data structures and algorithms. It's been pretty sweet to write code in a web notebook and render output including digraphs beneath each block of code. Is there anything like that for Ruby?
- K0nserv 10y ago
  Not to my knowledge and this is definitely an area where Python is a lot better than Ruby from my experience. My comment pertains more to web development than scientific computing.
- _joel 10y ago
  You can run a Ruby kernel in Jupyter - https://github.com/SciRuby/iruby https://github.com/SciRuby/iruby
- _Wintermute 10y ago
  You can use other kernels in jupyter notebooks. I've never tried ruby, but you could try this: https://github.com/SciRuby/iruby https://github.com/SciRuby/iruby
- LyndsySimon 10y ago
  Not only does Jupyter have Ruby support, but you can go back and forth between languages in the same notebook while sharing a context.
danso 10y ago
You bring up a good point. I'm in the same boat as you in terms of switching from Ruby to Python. Unlike you I've really enjoyed the switch. But also unlike you I've chosen to ignore (i.e. stick my head in the ground) the issues with the dev-tooling. Not that I ever had a strong understanding of how Rubygems worked, other than knowing why I needed to do `bundle exec rake [etc]`. But I've largely delegated the pain of Python package management to Anaconda. Also, I haven't had to deploy any production public-facing apps in Python. I don't share your experience on the documentation of libraries though. I've generally had no problems working with the major third-party libraries (requests, python-dateutils, Flask). And I also am reluctant to feel strongly against how the 2v3 chasm has been drawn out, though it's easy to say that as someone who didn't have to make the jump. The main difference from Ruby 1.8->1.9, AFAICT, is that the Ruby community has a near-monolithic devotion to Rails. When Rails left 1.8 behind, not making the jump would basically be career-suicide for a Ruby dev. The downside is that it shows how dependent Ruby is on Rails' momentum. Whereas in Python, a scientist (among several other use-cases) can live on 2.x for a very long time no matter what all moves to 3.x.
- rtpg 10y ago
  python had the opposite problem where so many major libraries didn't take make the jump to Python 3, so even if you really wanted to use Python 3 it was extremely restricting to do so.
barrkel 10y ago
I find chained enumerator / block style code much easier to write and understand than list / dict comprehensions too. It's probably the single biggest thing that I don't like about Python; lambdas are simply too awkward, but lambdas + monadic transformations are probably my primary mental abstractions for writing software these days.
- dood 10y ago
  Could you give an example of that?
  - barrkel 10y ago
    Think: bash shell pipes. Transforming input to output through successive filters and projections, applied in a concatenated fashion and developed interactively in a REPL. Model algorithms as transformations or compositions of the same, where possible, applied to lazy or eager iterators or consumers (the dual of an iterator, that acts as a sink for output) as required. Model data sources and sinks (like databases and network streams) as iterators and consumers. Program by building a data flow graph out of increasingly sophisticated (by composition) stateless widgets, rather than imperative loops etc.
  - K0nserv 10y ago
    Imagine you have a list of products with prices and you want to calculate the cost of all products that are of a certain type. In ruby you can write this as products.select { |product| product.type == 'x' }.map(&:price).reduce(:+)
    - minikomi 10y ago
      And in Python you can write something like: sum([p.price for p in products if p.type == "x"]) Excuse untested code from phone, but I like it better than the ruby version.
      - K0nserv 10y ago
        I guess it's a personal style choice, the Python code is more performant(less iterations over the data set), but I like the ruby version better.
deleted 10y ago
[deleted]
kozikow 10y ago
> Having used Rubygems and bundler pip feels like taking an enormous step back. Personally I moved away from relying on language specific tools to do more than just "install library X" and manage my deployment and dependencies using docker. So far it works great for me. > There seems to be fewer nice libraries Depends on your domain. Ruby is very good at web development. Python is good at almost everything. > Python libraries have lacking or hard to find documentation in many cases. I didn't use Ruby, but JVM languages or Javascript don't come close to Python in the documentation coverage. Especially if you consider SO as a documentation, googling "how do I do X" in python usually gets you on the right track to the high quality solution. Javascript or Java SO post are more like a mine field in comparison.
- zemanel 10y ago
  Did you take a look at pip-tools ? It allows dependency locking
  - StavrosK 10y ago
    What do you guys mean by dependency locking? What I do with pip is add a requirements.txt with things like: Django~=1.10.0 And it keeps the env autoupdated with the latest patch revision but not the latest minor revision. pip-tools doesn't seem to do much related to that, no?
    - K0nserv 10y ago
      The problem with that is that you no longer have reproducible builds. If you checkout a year old commit and built deployment artifacts with it you want the resulting artifact the be the same as when it was created. This is what you get with Bundler and the Gemfile/Gemfile.lock split You put gem 'rails', '~> 4.2.7' in your Gemfile and when you run `bundler install` the exact version of rails you ended up resolving with the above is locked in your Gemfile.lock. Thus if you have a version in Gemfile.lock it is the source of truth and you have reproducible builds. If you want to update rails given the pattern in your gemfile you just issue `bundle update rails` and it will update to the latest conforming version and update your Gemfile.lock The fact that pip doesn't support this in a good way is a major limitation imo. You can have either reproducible builds or "auto upgrading", but not both. pip-tools solves this, but requires you to use a different tool than pip and by extension you have to teach and explain it to anyone working on your project.
rtpg 10y ago
I think a lot of this is due to these problems being solved through standardised processes in python. For example, with dependencies, production dependencies and dev dependencies are defined through two requirements files: requirements.txt and requirements-dev.txt. requirements-test.txt for testing. No need for tooling, just pass those into pip. Most python tooling is pretty limited in scope but works well. From my side, I've had more trouble with "turing complete tooling" that ruby has offered me. I have not had to use ruby a great deal though. I think the difference in philosophy between the two communities is that python stuff will usually value ease of debugging/implementation over "ease of use". Most things in pyhton and its echosystem are straightforward. None of this "Library A adding methods to classes of Library B" stuff (not that it's impossible). Of course, you then end up with more boilerplate. Py2/Py3 has definitely been a major pain.
- K0nserv 10y ago
  Yeah the requirements-{group}.txt convention kind of works, but what happens if you want to save only your frozen test dependencies for example? If you are using a single virtual env you end up freeezing everything in that env not only the test dependencies and its transitive dependencies.
kennell 10y ago
I prefer Python over Ruby as a language, but you are correct gem ist superior to pip. Take a look at "pip-tools" if you haven't already, it eases some of the pain. For web-dev Rails is more stable, more modern and has better maintained library ecosystem than Django. Django is okay-ish but can't live up to Rails. Django still lacks some stuff that i feel it should bring with it out of the box.... dev/test/production setups, asset compilation etc. On the other hand, if you don't need a full blown web framework and are looking for something more minimalistic, Flask is awesome.
Stubb 10y ago
Amazing that Python 2 vs. Python 3 has still not yet been sorted out. I dipped my toe into Python a couple years ago when looking for a general-purpose scripting language and quickly gave up in frustration—Python 3 was by far the preferable flavor, but many libraries simply didn't work. I picked up Ruby and haven't looked back.
- busterarm 10y ago
  Likewise...and when I have to work in Python it's a frustrating pain because the tooling is so much better on the Ruby side. Heck, the tooling is better on the PHP side even...
- lastofus 10y ago
  A lot has improved since a couple years ago. A majority of major libraries have both Python 2 and 3 support these days. I've since started all new projects in Python 3.
- justifier 10y ago
  i started with python when 3 dropped in ~2008.917 ignorant of any prior ecosystem i just thought i was getting into the most current i came for the prebaked arbitrary precision floating point arithmetic and now i use it for prototyping all of my projects, and all the other stuff that i would call general scripting this 2to3 complaint made sense to me from an employed standpoint:i work as X and use 2.7's Y library; but you said general scripting i wonder what libraries were you hoping to use but were unable, and for what general scripting purpose?
  - Stubb 10y ago
    This was back in late 2013. My general scripting had to do with automating wireless network simulations to explore parameter spaces and slicing & dicing results stored in MySQL or SQLite databases. I recall several showstoppers in Python v3 including integration into Emacs, accessing a MySQL database, and basic visualization. The library system was also a mess. It didn't seem possible to develop on v3 and expect that the libraries you needed would be available. Or that anyone else would care since they were all using v2.7.

by K0nserv 10y ago

korijn 10y ago

Check out conda sometime.

K0nserv 10y ago
That looks interesting, I've also tried out pip-tools[0] which is really nice. However nice these tools are it seems to a bit at odds with the Zen of Python. Ideally there should be just one single good way to deal with packages. Ironically this is the case for Ruby, but not at all for Python. Ultimately you can always use some alternative method that solves it better, but you then have to take on the burden of convincing your co-contributors that it is a better idea than using pip and you also have to explain it to anyone that joins your project. 0: https://github.com/nvie/pip-tools https://github.com/nvie/pip-tools
IndianAstronaut 10y ago
It can conflict with any native Python installations. So if you are running any pre built Python libraries which point to any system wide Python, it can be a nightmare. Hue is one example I ran into.

curiousssly 10y ago

"There seems to be fewer nice libraries and they seem to be spread across the web whereas Ruby centralises around GitHub. I also find that Python libraries have lacking or hard to find documentation in many cases." Could you clarify a bit more on this with the real world examples ?

K0nserv 10y ago
Well for the case of "nicer libraries" I suppose I mean high quality libraries. Also good to note that I am referring to libraries in the web development/backend sphere here. I have a hard time thinking of some real world examples of there being a larger quantity of quality libraries for ruby, it's more a feeling that I get. Around documentation and usage I think it comes down to the fact that for Python libraries a google search usually takes you to the modules page on the PyPi registry, to a project website or a read the docs instance. I usually find that where ever I end up doesn't provide all the information I want, e.g what's the latest version, what are some typical use cases for this library. As a concrete example let's compare Flask and Sinatra. For both Flask and Sinatra a google search gives me their respective websites as the top relevant result. In this case I find both websites don't really provide all the information I am after. When I navigate to their GitHub page however the difference is more stark. Sinatra uses GitHub flavoured markdown to render a nice guide with increasingly more complex use cases. Flask just has some non markdown text pointing me in a few different directions. I could probably find all the information I need to setup and use Sinatra from just the GitHub Readme whereas for Flask I'd have to go navigate their read the docs page where all the documentation and usage instructions are spread over multiple pages and sub pages. I guess it comes down to my preference for the more concise format of a long markdown README.md vs the experience with a read the docs like set of pages. I recon this is a bit rambling, it's kinda hard to put into words what my feelings around this are and why I hold the opinions I do.

euroclydon 10y ago

I wish Ruby had the math and scientific library support of Python. Also, I've been using Jupyter lately to brush up on data structures and algorithms. It's been pretty sweet to write code in a web notebook and render output including digraphs beneath each block of code. Is there anything like that for Ruby?

K0nserv 10y ago
Not to my knowledge and this is definitely an area where Python is a lot better than Ruby from my experience. My comment pertains more to web development than scientific computing.
_joel 10y ago
You can run a Ruby kernel in Jupyter - https://github.com/SciRuby/iruby https://github.com/SciRuby/iruby
_Wintermute 10y ago
You can use other kernels in jupyter notebooks. I've never tried ruby, but you could try this: https://github.com/SciRuby/iruby https://github.com/SciRuby/iruby
LyndsySimon 10y ago
Not only does Jupyter have Ruby support, but you can go back and forth between languages in the same notebook while sharing a context.

danso 10y ago

You bring up a good point. I'm in the same boat as you in terms of switching from Ruby to Python. Unlike you I've really enjoyed the switch. But also unlike you I've chosen to ignore (i.e. stick my head in the ground) the issues with the dev-tooling. Not that I ever had a strong understanding of how Rubygems worked, other than knowing why I needed to do `bundle exec rake [etc]`. But I've largely delegated the pain of Python package management to Anaconda. Also, I haven't had to deploy any production public-facing apps in Python. I don't share your experience on the documentation of libraries though. I've generally had no problems working with the major third-party libraries (requests, python-dateutils, Flask). And I also am reluctant to feel strongly against how the 2v3 chasm has been drawn out, though it's easy to say that as someone who didn't have to make the jump. The main difference from Ruby 1.8->1.9, AFAICT, is that the Ruby community has a near-monolithic devotion to Rails. When Rails left 1.8 behind, not making the jump would basically be career-suicide for a Ruby dev. The downside is that it shows how dependent Ruby is on Rails' momentum. Whereas in Python, a scientist (among several other use-cases) can live on 2.x for a very long time no matter what all moves to 3.x.

rtpg 10y ago
python had the opposite problem where so many major libraries didn't take make the jump to Python 3, so even if you really wanted to use Python 3 it was extremely restricting to do so.

barrkel 10y ago

I find chained enumerator / block style code much easier to write and understand than list / dict comprehensions too. It's probably the single biggest thing that I don't like about Python; lambdas are simply too awkward, but lambdas + monadic transformations are probably my primary mental abstractions for writing software these days.

dood 10y ago
Could you give an example of that?
- barrkel 10y ago
  Think: bash shell pipes. Transforming input to output through successive filters and projections, applied in a concatenated fashion and developed interactively in a REPL. Model algorithms as transformations or compositions of the same, where possible, applied to lazy or eager iterators or consumers (the dual of an iterator, that acts as a sink for output) as required. Model data sources and sinks (like databases and network streams) as iterators and consumers. Program by building a data flow graph out of increasingly sophisticated (by composition) stateless widgets, rather than imperative loops etc.
- K0nserv 10y ago
  Imagine you have a list of products with prices and you want to calculate the cost of all products that are of a certain type. In ruby you can write this as products.select { |product| product.type == 'x' }.map(&:price).reduce(:+)
  - minikomi 10y ago
    And in Python you can write something like: sum([p.price for p in products if p.type == "x"]) Excuse untested code from phone, but I like it better than the ruby version.
    - K0nserv 10y ago
      I guess it's a personal style choice, the Python code is more performant(less iterations over the data set), but I like the ruby version better.

deleted 10y ago

[deleted]

kozikow 10y ago

> Having used Rubygems and bundler pip feels like taking an enormous step back. Personally I moved away from relying on language specific tools to do more than just "install library X" and manage my deployment and dependencies using docker. So far it works great for me. > There seems to be fewer nice libraries Depends on your domain. Ruby is very good at web development. Python is good at almost everything. > Python libraries have lacking or hard to find documentation in many cases. I didn't use Ruby, but JVM languages or Javascript don't come close to Python in the documentation coverage. Especially if you consider SO as a documentation, googling "how do I do X" in python usually gets you on the right track to the high quality solution. Javascript or Java SO post are more like a mine field in comparison.

zemanel 10y ago
Did you take a look at pip-tools ? It allows dependency locking
- StavrosK 10y ago
  What do you guys mean by dependency locking? What I do with pip is add a requirements.txt with things like: Django~=1.10.0 And it keeps the env autoupdated with the latest patch revision but not the latest minor revision. pip-tools doesn't seem to do much related to that, no?
  - K0nserv 10y ago
    The problem with that is that you no longer have reproducible builds. If you checkout a year old commit and built deployment artifacts with it you want the resulting artifact the be the same as when it was created. This is what you get with Bundler and the Gemfile/Gemfile.lock split You put gem 'rails', '~> 4.2.7' in your Gemfile and when you run `bundler install` the exact version of rails you ended up resolving with the above is locked in your Gemfile.lock. Thus if you have a version in Gemfile.lock it is the source of truth and you have reproducible builds. If you want to update rails given the pattern in your gemfile you just issue `bundle update rails` and it will update to the latest conforming version and update your Gemfile.lock The fact that pip doesn't support this in a good way is a major limitation imo. You can have either reproducible builds or "auto upgrading", but not both. pip-tools solves this, but requires you to use a different tool than pip and by extension you have to teach and explain it to anyone working on your project.

rtpg 10y ago

I think a lot of this is due to these problems being solved through standardised processes in python. For example, with dependencies, production dependencies and dev dependencies are defined through two requirements files: requirements.txt and requirements-dev.txt. requirements-test.txt for testing. No need for tooling, just pass those into pip. Most python tooling is pretty limited in scope but works well. From my side, I've had more trouble with "turing complete tooling" that ruby has offered me. I have not had to use ruby a great deal though. I think the difference in philosophy between the two communities is that python stuff will usually value ease of debugging/implementation over "ease of use". Most things in pyhton and its echosystem are straightforward. None of this "Library A adding methods to classes of Library B" stuff (not that it's impossible). Of course, you then end up with more boilerplate. Py2/Py3 has definitely been a major pain.

K0nserv 10y ago
Yeah the requirements-{group}.txt convention kind of works, but what happens if you want to save only your frozen test dependencies for example? If you are using a single virtual env you end up freeezing everything in that env not only the test dependencies and its transitive dependencies.

kennell 10y ago

I prefer Python over Ruby as a language, but you are correct gem ist superior to pip. Take a look at "pip-tools" if you haven't already, it eases some of the pain. For web-dev Rails is more stable, more modern and has better maintained library ecosystem than Django. Django is okay-ish but can't live up to Rails. Django still lacks some stuff that i feel it should bring with it out of the box.... dev/test/production setups, asset compilation etc. On the other hand, if you don't need a full blown web framework and are looking for something more minimalistic, Flask is awesome.

Stubb 10y ago

Amazing that Python 2 vs. Python 3 has still not yet been sorted out. I dipped my toe into Python a couple years ago when looking for a general-purpose scripting language and quickly gave up in frustration—Python 3 was by far the preferable flavor, but many libraries simply didn't work. I picked up Ruby and haven't looked back.

busterarm 10y ago
Likewise...and when I have to work in Python it's a frustrating pain because the tooling is so much better on the Ruby side. Heck, the tooling is better on the PHP side even...
lastofus 10y ago
A lot has improved since a couple years ago. A majority of major libraries have both Python 2 and 3 support these days. I've since started all new projects in Python 3.
justifier 10y ago
i started with python when 3 dropped in ~2008.917 ignorant of any prior ecosystem i just thought i was getting into the most current i came for the prebaked arbitrary precision floating point arithmetic and now i use it for prototyping all of my projects, and all the other stuff that i would call general scripting this 2to3 complaint made sense to me from an employed standpoint:i work as X and use 2.7's Y library; but you said general scripting i wonder what libraries were you hoping to use but were unable, and for what general scripting purpose?
- Stubb 10y ago
  This was back in late 2013. My general scripting had to do with automating wireless network simulations to explore parameter spaces and slicing & dicing results stored in MySQL or SQLite databases. I recall several showstoppers in Python v3 including integration into Emacs, accessing a MySQL database, and basic visualization. The library system was also a mess. It didn't seem possible to develop on v3 and expect that the libraries you needed would be available. Or that anyone else would care since they were all using v2.7.