6 ms·

Good points. Personally, it's a bit puzzling to me why the ML community ran with Python and why there isn't a popular alternative for Ruby. It's very capable i

Good points. Personally, it's a bit puzzling to me why the ML community ran with Python and why there isn't a popular alternative for Ruby. It's very capable in terms of making DSLs. My hunch is that the number-crunching libraries were better in Python (numpy etc).

pdimitar 7y ago
I spoke with several data scientists and students in the last 2 years and the consensus was basically two central points: 1. NumPy is really fast and has a lot of features. Even if they don't need all of them they feel more at ease knowing that they can use them should the need arise. 2. Python is very familiar to them, their professors know it, they know it, it's easy to teach to new people in the team so everybody just goes with the flow. As much as I think the bigger IT area is in a desperate need of innovation... I can actually agree with those two points. Tinkering should be left to us the programmers and sysadmins. Everybody else has a job to do and they cannot and won't spend a lot of time evaluating all tech possibilities. That's okay. It's our job to tempt them with something irresistible. :)
- Gravityloss 7y ago
  But those are reasons that are valid afterwards. Why weren't the good numerical libraries developed for Ruby? Maybe the watershed moment was when MIT switched from Scheme to Python? I don't know.
  - pdimitar 7y ago
    Really good question. And I have no clue. But I believe (can't prove) that you are correct on the outset: some big org switched, then a lot of people went like "oh well, Org X can't be wrong", and the rest is history.
    - simtel20 7y ago
      This isn't how I remember it. I remember that prior to the scientific python stack, you had to pay for Matlab or mathmatica licenses. The scientific python stack meant that for essentially free, you no longer had to justify licenses at any financial or educational institutions. You could just load up your code and run it against your data. In academia this meant that every student could do their homework on their desktop or laptop (if they were motivated). Even in financial companies, the decision was often made to limit the number of licenses - and in most cases they weren't used to actually run production jobs. Quants had to translate their work to another language with the help of their developers. With the scientific python stack you entered a new world with unlimited seats, you could at least try to run the same experimental/exploratory code with minimal changes in a production environment, etc. So in short it didn't need some tentpole user/customer/exemplar. In almost any situation it was free and it started making money as soon as it was downloaded installed and run.
    - moomin 7y ago
      Well, I believe that the native memory model of Python is relatively welcoming to just doing your own thing, but also Python just got there first: NumPy was first released the same year as Ruby on Rails. So Western developers were discovering Ruby while Python was merging two existing scientific programming projects.
  - simtel20 7y ago
    My recollection is that Perl had numerical libraries that were crash-prone, and in at least the finance company where I worked that was the backbone for some models for a while. In the early-mid 2000s numpy/scipy got a lot of commercial support, and as I recall the ffi support for this was broader and more unified than anything else - the unifying idea of the numpy array and the ability to load it with data and pass it to everything in the ecosystem just made it work. At the same time, I saw no substantial interest in persuing the same dedicated effort at the same scale in any other ecosystem. Rails was what everyone did in Ruby (aside: recalling that the ffi changed between 1.x and 2.x and it wasn't a trivial API change, and how the only documentation seemed to be the code, would have broken any prior work... I can't imagine putting long-term effort in this direction), java and jvm hosted languages didn't seem to gain any traction from users for these problem spaces (maybe the inability to easily dynamically allocate more heap on demand in user code made this unfeasible?) Looking back I think the fact that numerical work has found success being built on a huge foundation of stable libraries that are glued into a dynamic interpreted language and which is then often treated as a finished artifact, I can't think of another major language that is more appropriate than python. Specifically relative to Ruby: at its core Ruby is matz's place to experiment about ideas in programming languages. If you are going to build your business around a few million lines of glue code that relies on the internals of a language, you want to have strong guarantees about the stability of what you're building on. Looking back I think the python 2 to 3 transition period actually made this better for python by providing a very stable foundation in python2 while experiments and destabilizing changes were mostly pushed to 3, which kept ahead. When finally transitioning to 3, the needs of the scientific python community and ecosystem were so well known and represented that it was unlikely to simply get forgotten, left out, or left behind.
    - pdimitar 7y ago
      Pretty good analysis and recollection. Thank you.
  - matheusmoreira 7y ago
    Ruby was relatively unknown until Rails made it popular. Python had a head start.
  - riffraff 7y ago
    Numpy was working fine and effectively for a long time, ruby had narray but it wasn't as good (less documentation and such). Then scipy was the same. Then pandas etc etc.
  - marcinzm 7y ago
    Looking at Tiobe, in the early 2000s Python was a lot more popular than Ruby although neither was very popular. Then RoR took off but that only made Ruby on parity with Python as Python's popularity was also growing. Or in other words, 90% of Ruby talk became about RoR while Python wasn't. If you're making a quick local program or library, would you pick the language where people only seem to do web dev or one that seems to have broader adoption? edit: Which lines up with my recollection, Ruby was synonymous with Rails and web dev while Python wasn't type cast yet.
  - mdasen 7y ago
    Ultimately, it's probably that Python is older and it's use in the English-speaking world is substantially older. Python is around 5 years older than Ruby and the first book about Ruby in English came 10 years after Python existed. It's 1995. Python has been around for 5 years. Guido is working with the scientific community and adding better complex number, array, and matrix support to Python. Ruby has just started and everything is in Japanese. Scientists and Engineers at US universities and FFRDCs (Federally Funded R&D Centers) keep plugging away at Python for scientific purposes. Finally in late 2000, the first book about Ruby is published in English. Ruby is still pretty unknown. In 2005 Rails hits the scene and the English-speaking world notices Ruby. But this is 10 years after the scientific community starts investing in Python and 15 years after the Python language came out. By this time, the community had Numeric, numarray, and finally NumPy to unify their efforts. Ultimately, Python's BDFL started working with the scientific/numeric community before Ruby existed. It would be 10-15 years after Python that Ruby would start making inroads in the English-speaking world and by that time there was already good numeric/scientific tools in Python. Scientific communities weren't looking to learn new languages for the sake of new languages. Many still just use MATLAB. MIT hadn't switched to Python when a lot of this happened. I think it's more that Ruby wasn't widely known to exist until 2005 and was "brand new" to the English-speaking world then. Python had been plugging away at numerical libraries for a decade by then and NumPy (which had learned from previous numeric/scientific libraries) came out basically when Rails came out. Once all these numeric libraries exist for Python, are you getting grad students excited to work on an open-source numeric system for Ruby? Are you getting FFRDCs interested in creating libraries for a language that's quite similar to Python without any significant advantage? How do you justify that work? How do you get people excited about it when the reaction of most users is going to be "we're already happy with what we've got"? Rails burst onto the scene at a time when people were often unhappy with their tools. Rails was a breath of fresh air with clean separation of concerns and a lot of helpful stuff. This is back when Java meant XML sit-ups, C# was closed-source and this is before most of what the community knows today existed, Scala had just appeared, Python was still in the Plone/Zope era (remember when an object DB in Python seemed good?), PHP was pretty dominant but often ended up a mess and didn't have the same framework that Rails was offering. Rails offered something compelling and new which meant attention. Many other communities followed Rails - Django, .NET MVC, JAX-RS, etc. Still, Rails was there first and got a lot of mind-share. Similarly, Python was there first in scientific/numeric computing and people haven't provided enough to compel people to move off it.
marcinzm 7y ago
I remember Python being fairly popular for quick work in research 10+ years ago. It was simple and easy to start with but not that different from the c/c++ which they needed for performant bits. >It's very capable in terms of making DSLs. Which is a double edged sword. Too many DSLs means you have to learn and remember a lot more to write code.
jpz 7y ago
Numerical computation is generally done on scripting platforms - Matlab for instance. After a few different options such as Octave, numerical computation in the free software space coalesced around Python. The Python ecosystem - it's change and release processes (PIPs etc) - thorough API documentation - make for a good target language, over other languages that simply do not have that infrastructure. I don't know what the state of Ruby documentation is these days for instance, but back in the day, it was simply incomplete.
groundCode 7y ago
One reason I can think of is, certainly early in the Ruby/rails days it was quite difficult to support/install ruby on Windows. Python was much easier to install and get going with so it maybe got a foothold just because more people had access to it.
cbzbc 7y ago
Another reason is that in the timeframe where people might conceivably written these, ruby was going through the 1.X->2 transition with all the pain that entailed. ISTM (supporting a system written in Ruby at the time) that a large number of the libraries were broken when used in concert with each other -- and that language experts knew how to monkey patch their way around these issues, but didn't bother documented these fixes. I've been told that things are much more mature and stable now -- but that experience put me off the language.
- pdimitar 7y ago
  These are pretty normal growing pains and one should ascertain for themselves if they are willing to shoulder the risk of using an emerging technology. That being said, some of the "break stuff and monkey patch later" mentality is still in the community's DNA. I am not trash-talking that phenomena but I also found it off-putting and eventually moved away from Ruby/Rails after 6 years with them.
int_19h 7y ago
I remember looking at both Python and Ruby when I was getting into dynamic languages - this was the time when Rails just showed up, and Ruby was suddenly getting a lot of attention. I vaguely recall that Python documentation was generally much better at the time, and especially so the parts on writing extension modules. With Ruby, it felt like the English reference docs were largely an afterthought - you pretty much had to get a book on it to get far, and the more advanced sections (like extensibility) were especially lacking.

pdimitar 7y ago

I spoke with several data scientists and students in the last 2 years and the consensus was basically two central points: 1. NumPy is really fast and has a lot of features. Even if they don't need all of them they feel more at ease knowing that they can use them should the need arise. 2. Python is very familiar to them, their professors know it, they know it, it's easy to teach to new people in the team so everybody just goes with the flow. As much as I think the bigger IT area is in a desperate need of innovation... I can actually agree with those two points. Tinkering should be left to us the programmers and sysadmins. Everybody else has a job to do and they cannot and won't spend a lot of time evaluating all tech possibilities. That's okay. It's our job to tempt them with something irresistible. :)

Gravityloss 7y ago
But those are reasons that are valid afterwards. Why weren't the good numerical libraries developed for Ruby? Maybe the watershed moment was when MIT switched from Scheme to Python? I don't know.
- pdimitar 7y ago
  Really good question. And I have no clue. But I believe (can't prove) that you are correct on the outset: some big org switched, then a lot of people went like "oh well, Org X can't be wrong", and the rest is history.
  - simtel20 7y ago
    This isn't how I remember it. I remember that prior to the scientific python stack, you had to pay for Matlab or mathmatica licenses. The scientific python stack meant that for essentially free, you no longer had to justify licenses at any financial or educational institutions. You could just load up your code and run it against your data. In academia this meant that every student could do their homework on their desktop or laptop (if they were motivated). Even in financial companies, the decision was often made to limit the number of licenses - and in most cases they weren't used to actually run production jobs. Quants had to translate their work to another language with the help of their developers. With the scientific python stack you entered a new world with unlimited seats, you could at least try to run the same experimental/exploratory code with minimal changes in a production environment, etc. So in short it didn't need some tentpole user/customer/exemplar. In almost any situation it was free and it started making money as soon as it was downloaded installed and run.
  - moomin 7y ago
    Well, I believe that the native memory model of Python is relatively welcoming to just doing your own thing, but also Python just got there first: NumPy was first released the same year as Ruby on Rails. So Western developers were discovering Ruby while Python was merging two existing scientific programming projects.
- simtel20 7y ago
  My recollection is that Perl had numerical libraries that were crash-prone, and in at least the finance company where I worked that was the backbone for some models for a while. In the early-mid 2000s numpy/scipy got a lot of commercial support, and as I recall the ffi support for this was broader and more unified than anything else - the unifying idea of the numpy array and the ability to load it with data and pass it to everything in the ecosystem just made it work. At the same time, I saw no substantial interest in persuing the same dedicated effort at the same scale in any other ecosystem. Rails was what everyone did in Ruby (aside: recalling that the ffi changed between 1.x and 2.x and it wasn't a trivial API change, and how the only documentation seemed to be the code, would have broken any prior work... I can't imagine putting long-term effort in this direction), java and jvm hosted languages didn't seem to gain any traction from users for these problem spaces (maybe the inability to easily dynamically allocate more heap on demand in user code made this unfeasible?) Looking back I think the fact that numerical work has found success being built on a huge foundation of stable libraries that are glued into a dynamic interpreted language and which is then often treated as a finished artifact, I can't think of another major language that is more appropriate than python. Specifically relative to Ruby: at its core Ruby is matz's place to experiment about ideas in programming languages. If you are going to build your business around a few million lines of glue code that relies on the internals of a language, you want to have strong guarantees about the stability of what you're building on. Looking back I think the python 2 to 3 transition period actually made this better for python by providing a very stable foundation in python2 while experiments and destabilizing changes were mostly pushed to 3, which kept ahead. When finally transitioning to 3, the needs of the scientific python community and ecosystem were so well known and represented that it was unlikely to simply get forgotten, left out, or left behind.
  - pdimitar 7y ago
    Pretty good analysis and recollection. Thank you.
- matheusmoreira 7y ago
  Ruby was relatively unknown until Rails made it popular. Python had a head start.
- riffraff 7y ago
  Numpy was working fine and effectively for a long time, ruby had narray but it wasn't as good (less documentation and such). Then scipy was the same. Then pandas etc etc.
- marcinzm 7y ago
  Looking at Tiobe, in the early 2000s Python was a lot more popular than Ruby although neither was very popular. Then RoR took off but that only made Ruby on parity with Python as Python's popularity was also growing. Or in other words, 90% of Ruby talk became about RoR while Python wasn't. If you're making a quick local program or library, would you pick the language where people only seem to do web dev or one that seems to have broader adoption? edit: Which lines up with my recollection, Ruby was synonymous with Rails and web dev while Python wasn't type cast yet.
- mdasen 7y ago
  Ultimately, it's probably that Python is older and it's use in the English-speaking world is substantially older. Python is around 5 years older than Ruby and the first book about Ruby in English came 10 years after Python existed. It's 1995. Python has been around for 5 years. Guido is working with the scientific community and adding better complex number, array, and matrix support to Python. Ruby has just started and everything is in Japanese. Scientists and Engineers at US universities and FFRDCs (Federally Funded R&D Centers) keep plugging away at Python for scientific purposes. Finally in late 2000, the first book about Ruby is published in English. Ruby is still pretty unknown. In 2005 Rails hits the scene and the English-speaking world notices Ruby. But this is 10 years after the scientific community starts investing in Python and 15 years after the Python language came out. By this time, the community had Numeric, numarray, and finally NumPy to unify their efforts. Ultimately, Python's BDFL started working with the scientific/numeric community before Ruby existed. It would be 10-15 years after Python that Ruby would start making inroads in the English-speaking world and by that time there was already good numeric/scientific tools in Python. Scientific communities weren't looking to learn new languages for the sake of new languages. Many still just use MATLAB. MIT hadn't switched to Python when a lot of this happened. I think it's more that Ruby wasn't widely known to exist until 2005 and was "brand new" to the English-speaking world then. Python had been plugging away at numerical libraries for a decade by then and NumPy (which had learned from previous numeric/scientific libraries) came out basically when Rails came out. Once all these numeric libraries exist for Python, are you getting grad students excited to work on an open-source numeric system for Ruby? Are you getting FFRDCs interested in creating libraries for a language that's quite similar to Python without any significant advantage? How do you justify that work? How do you get people excited about it when the reaction of most users is going to be "we're already happy with what we've got"? Rails burst onto the scene at a time when people were often unhappy with their tools. Rails was a breath of fresh air with clean separation of concerns and a lot of helpful stuff. This is back when Java meant XML sit-ups, C# was closed-source and this is before most of what the community knows today existed, Scala had just appeared, Python was still in the Plone/Zope era (remember when an object DB in Python seemed good?), PHP was pretty dominant but often ended up a mess and didn't have the same framework that Rails was offering. Rails offered something compelling and new which meant attention. Many other communities followed Rails - Django, .NET MVC, JAX-RS, etc. Still, Rails was there first and got a lot of mind-share. Similarly, Python was there first in scientific/numeric computing and people haven't provided enough to compel people to move off it.

marcinzm 7y ago

I remember Python being fairly popular for quick work in research 10+ years ago. It was simple and easy to start with but not that different from the c/c++ which they needed for performant bits. >It's very capable in terms of making DSLs. Which is a double edged sword. Too many DSLs means you have to learn and remember a lot more to write code.

jpz 7y ago

Numerical computation is generally done on scripting platforms - Matlab for instance. After a few different options such as Octave, numerical computation in the free software space coalesced around Python. The Python ecosystem - it's change and release processes (PIPs etc) - thorough API documentation - make for a good target language, over other languages that simply do not have that infrastructure. I don't know what the state of Ruby documentation is these days for instance, but back in the day, it was simply incomplete.

groundCode 7y ago

One reason I can think of is, certainly early in the Ruby/rails days it was quite difficult to support/install ruby on Windows. Python was much easier to install and get going with so it maybe got a foothold just because more people had access to it.

cbzbc 7y ago

Another reason is that in the timeframe where people might conceivably written these, ruby was going through the 1.X->2 transition with all the pain that entailed. ISTM (supporting a system written in Ruby at the time) that a large number of the libraries were broken when used in concert with each other -- and that language experts knew how to monkey patch their way around these issues, but didn't bother documented these fixes. I've been told that things are much more mature and stable now -- but that experience put me off the language.

pdimitar 7y ago
These are pretty normal growing pains and one should ascertain for themselves if they are willing to shoulder the risk of using an emerging technology. That being said, some of the "break stuff and monkey patch later" mentality is still in the community's DNA. I am not trash-talking that phenomena but I also found it off-putting and eventually moved away from Ruby/Rails after 6 years with them.

int_19h 7y ago

I remember looking at both Python and Ruby when I was getting into dynamic languages - this was the time when Rails just showed up, and Ruby was suddenly getting a lot of attention. I vaguely recall that Python documentation was generally much better at the time, and especially so the parts on writing extension modules. With Ruby, it felt like the English reference docs were largely an afterthought - you pretty much had to get a book on it to get far, and the more advanced sections (like extensibility) were especially lacking.