5 ms·

For all its virtues, I feel introductions to Python tiptoe around a very important topic: yes, Python is nice and easy, but you'll have to program the Python wa

For all its virtues, I feel introductions to Python tiptoe around a very important topic: yes, Python is nice and easy, but you'll have to program the Python way too. Try running a nested loop on a non-trivial example, and you can end up spending minutes in what would take milliseconds in any other language. If you want to program in Python, you must get used to the functional paradigm. Not "should", "must". Now, I'm all in for bringing more functional programming into daily life, but I find a bit dishonest that beginners are not told about this. My first serious program took about a week for a task, and shaving that time down involved a lot of pain changing simple nested loops into multiplications of Numpy matrices. There was a lot of caching too. I have no idea how someone without a CS degree would have dealt with this.

technomancy 10y ago
Weird, I thought that Python was trying to discourage functional programming with its extremely limited lambdas and the proposal to move map and reduce out of the core language.
- brianwawok 10y ago
  Kinda. Many things you can do with a map can also be done with a list comprehension or generator, so that seems to be the way to go. Moving from Scala to Python it was annoying at first.. but honestly the code is easier to read in most cases, so no complaints here.
- LyndsySimon 10y ago
  IIRC, the limitations on lambdas has to do with the way whitespace is parsed rather than being an explicitly conscious decision.
  - andybak 10y ago
    Guido also maintains that by the time you need multiline lambda's, the extra cost of giving the function a name is more than outweighed by the improved readability. I kinda agree. Certainly it guards against some of the code you see in overly nested javascript. The downside is that the code is slightly removed from it's call site - but rarely by much. Overall the increased complexity it would bring to the language has not been justified by anyone arguing for it's benefits. Giving the function a name isn't the worst thing in the world.
- andybak 10y ago
  The important thing is to fully support first class functions - not the fact that they are anonymous - so I find it difficult to see how the lack of multiline anonymous functions is really a blow to functional Python I think the argument against map and reduce was that there was better and more Pythonic ways to do both - not because they were 'too functional'.
jnbiche 10y ago
> Try running a nested loop on a non-trivial example, and you can end up spending minutes in what would take milliseconds in any other language. Huh? Can you provide an example? There's nothing about writing nested loops in Python that is qualitatively different from other languages. > If you want to program in Python, you must get used to the functional paradigm. Not "should", "must". This is incorrect, much to my chagrin. Python's trend over the past few years has been not only not to encourage functional programming, but to explicitly discourage functional programming. For example, starting in Python 3, `reduce` has been banished from the language proper to the standard library.
- dagw 10y ago
  Huh? Can you provide an example? Well he did mention numpy and obviously doing something that can be done in numpy with nested loops will be much much slower. However that is as much a case of numpy being really fast as python being slow. For example, just tested elementwise multiplication of two 10kX10k matrices and with numpy and numpy arrays it took ~350 ms vs ~15 seconds with a nested for loops and python lists. Still minutes vs. milliseconds seems like extreme hyperbole. edit: Figured I'd test how long the naive nested loop approach would take in a 'fast' language like Julia, and much to my surprise it took over 50 seconds. edit2: cleaned up all my code to try to make it as similar as possible across both languages: Nested for loops python 2.7: ~30 seconds Nested for loops Julia 4.6; ~50 seconds Nested for loops PyPy 5.3.1 ~0.9 seconds Make of these numbers what you will
  - ced 10y ago
    Could you post the Julia code? Unfortunately, you do have to make sure that the types are correctly inferred by the compiler in high-performance loops. Or maybe you used a global. http://docs.julialang.org/en/release-0.4/manual/performance-tips/ http://docs.julialang.org/en/release-0.4/manual/performance-...
    - dagw 10y ago
      I'll admit I don't really know that much Julia and basically wrote naive MATLAB code (and I wanted to make it as close as possible to my python code): s=10000; a=ones(s,s); b=ones(s,s); c=zeros(s,s); tic(); for i in 1:s for j in 1:s c[i,j]=a[i,j]*b[i,j] end end toc(); Obviously in real code I'd simple write c=a.*b and get basically the same performance as numpy
andybak 10y ago
Curious what you mean about nested loops? Do you specifically mean nested list comprehensions? If nested they can sometime be less readable than explicit for loops so my advice to a struggling newbie would be "if it hurts stop doing it" Or did you mean something else?
- probably_wrong 10y ago
  I don't mean it as "readable", but as in "thanks to type inference, it will take forever". I linked to this question [1] somewhere else, which is something that AFAIK happens only in Python. [1] http://stackoverflow.com/questions/8097408/why-python-is-so-slow-for-a-simple-for-loop#8097669 http://stackoverflow.com/questions/8097408/why-python-is-so-...

technomancy 10y ago

Weird, I thought that Python was trying to discourage functional programming with its extremely limited lambdas and the proposal to move map and reduce out of the core language.

brianwawok 10y ago
Kinda. Many things you can do with a map can also be done with a list comprehension or generator, so that seems to be the way to go. Moving from Scala to Python it was annoying at first.. but honestly the code is easier to read in most cases, so no complaints here.
LyndsySimon 10y ago
IIRC, the limitations on lambdas has to do with the way whitespace is parsed rather than being an explicitly conscious decision.
- andybak 10y ago
  Guido also maintains that by the time you need multiline lambda's, the extra cost of giving the function a name is more than outweighed by the improved readability. I kinda agree. Certainly it guards against some of the code you see in overly nested javascript. The downside is that the code is slightly removed from it's call site - but rarely by much. Overall the increased complexity it would bring to the language has not been justified by anyone arguing for it's benefits. Giving the function a name isn't the worst thing in the world.
andybak 10y ago
The important thing is to fully support first class functions - not the fact that they are anonymous - so I find it difficult to see how the lack of multiline anonymous functions is really a blow to functional Python I think the argument against map and reduce was that there was better and more Pythonic ways to do both - not because they were 'too functional'.

jnbiche 10y ago

> Try running a nested loop on a non-trivial example, and you can end up spending minutes in what would take milliseconds in any other language. Huh? Can you provide an example? There's nothing about writing nested loops in Python that is qualitatively different from other languages. > If you want to program in Python, you must get used to the functional paradigm. Not "should", "must". This is incorrect, much to my chagrin. Python's trend over the past few years has been not only not to encourage functional programming, but to explicitly discourage functional programming. For example, starting in Python 3, `reduce` has been banished from the language proper to the standard library.

dagw 10y ago
Huh? Can you provide an example? Well he did mention numpy and obviously doing something that can be done in numpy with nested loops will be much much slower. However that is as much a case of numpy being really fast as python being slow. For example, just tested elementwise multiplication of two 10kX10k matrices and with numpy and numpy arrays it took ~350 ms vs ~15 seconds with a nested for loops and python lists. Still minutes vs. milliseconds seems like extreme hyperbole. edit: Figured I'd test how long the naive nested loop approach would take in a 'fast' language like Julia, and much to my surprise it took over 50 seconds. edit2: cleaned up all my code to try to make it as similar as possible across both languages: Nested for loops python 2.7: ~30 seconds Nested for loops Julia 4.6; ~50 seconds Nested for loops PyPy 5.3.1 ~0.9 seconds Make of these numbers what you will
- ced 10y ago
  Could you post the Julia code? Unfortunately, you do have to make sure that the types are correctly inferred by the compiler in high-performance loops. Or maybe you used a global. http://docs.julialang.org/en/release-0.4/manual/performance-tips/ http://docs.julialang.org/en/release-0.4/manual/performance-...
  - dagw 10y ago
    I'll admit I don't really know that much Julia and basically wrote naive MATLAB code (and I wanted to make it as close as possible to my python code): s=10000; a=ones(s,s); b=ones(s,s); c=zeros(s,s); tic(); for i in 1:s for j in 1:s c[i,j]=a[i,j]*b[i,j] end end toc(); Obviously in real code I'd simple write c=a.*b and get basically the same performance as numpy

andybak 10y ago

Curious what you mean about nested loops? Do you specifically mean nested list comprehensions? If nested they can sometime be less readable than explicit for loops so my advice to a struggling newbie would be "if it hurts stop doing it" Or did you mean something else?

probably_wrong 10y ago
I don't mean it as "readable", but as in "thanks to type inference, it will take forever". I linked to this question [1] somewhere else, which is something that AFAIK happens only in Python. [1] http://stackoverflow.com/questions/8097408/why-python-is-so-slow-for-a-simple-for-loop#8097669 http://stackoverflow.com/questions/8097408/why-python-is-so-...