5 ms·

The article makes a mistake in comparing lambda vs operator.add performance. The lambda example is recreating the function each time. $ python -m timeit "

The article makes a mistake in comparing lambda vs operator.add performance. The lambda example is recreating the function each time. $ python -m timeit "(lambda x,y: x + y)(12, 15)" 10000000 loops, best of 3: 0.124 usec per loop $ python -m timeit -s "from operator import add" "add(12, 15)" 10000000 loops, best of 3: 0.0565 usec per loop Instead, it should create a function in the setup and use that for the loop: $ python -m timeit -s "func = lambda x,y: x + y" "func(12, 15)" 10000000 loops, best of 3: 0.0779 usec per loop

blester 5y ago
Generally it is un-pythonic to assign lambda to a variable. Lots of linters will complain if you do this.
- sharikous 5y ago
  So do a "def" instead, it can be even inlined. def add_fun(x,y): return x+y But I wouldn't mind irritating the linter with an assigned lambda. I fail to see what's the problem with it
  - ViViDboarder 5y ago
    But if you’re doing that, why not just import the identical function?
    - sharikous 5y ago
      One major point of OP was that the functions defined in operators are faster. Grandparent comment argued that there is a measurement error that invalidates this point, since properly defined lambdas (or defs) are just as fast. Of course the point regarding the simplicity of importing a function instead of defining it still stands. I personally wouldn't mind defining those functions as they are very immediate. So from the article I would take only the part about the more exotic "operator".
      - klyrs 5y ago
        The timings are certainly closer, but the built-in function imported from operator is still faster than the user-defined function. Not a big deal if you're doing simple stuff, but if this code is in a hot loop that 30% can make a measurable difference.
  - kevin_thibedeau 5y ago
    Lambdas don't know their name and weaken debug info.
    - elteto 5y ago
      It seems like an implementation issue of the interpreter no? There’s no reason lambdas _have_ to be this way.
      - jonathrg 5y ago
        lambdas don't have names, how could the interpreter possibly improve in this regard?
        zodiakzz 5y ago
        JS arrow functions don't have names either, it's inferred from the left hand side variable name. Can't Python do something similar?
        jonathrg 5y ago
        If you assign a lambda to a variable, you might as well use a def. A more typical use of a lambda is as an arguments to a function call. What would be the name of, say, the first argument to map(lambda x: x*2, numbers)?
- ris 5y ago
  Linters will complain about all sorts of nonsense, and I'll have to continue to remind people that pep8 was only supposed to be guidance, and never something that was enforced. In short: being picked up by a linter does not mean that something is un-pythonic.
- elteto 5y ago
  The reasoning that I’ve seen for this is that you lose context information (file, line, etc) when using lambdas. Is that it? If so it seems like a an interpreter implementation issue.
- jonathrg 5y ago
  The point is that the function is created only once, like when you pass a lambda to map/filter/reduce or any function that takes a callback. The fact that it is assigned here is just a neccessary step for the timeit command to do the right thing
randallsquared 5y ago
Is that a mistake, or is it just the reason why the lambda style is slower? If you're assigning the lambda, then as @sharikous mentions below, you might as well just be using a def, which is exactly what is done in the operator module: https://github.com/python/cpython/blob/3.9/Lib/operator.py#L75 https://github.com/python/cpython/blob/3.9/Lib/operator.py#L... For someone familiar with the operator module, there is a final small argument for using it instead of defining your own, if the result will be the same: that's a definition that doesn't have to be understood by the reader.
- kzrdude 5y ago
  You're reading the pure-python version of operator.py. You get the builtin versions of the same functions - not the python functions when you use it - see the import of _operator further down the file. Also verify the type info on the object here: In [1]: import operator In [5]: operator.add?? Signature: operator.add(a, b, /) Docstring: Same as a + b. Type: builtin_function_or_method
- JonathonW 5y ago
  'operator' actually maps to C implementations if possible; you're still calling Python functions, but the implementations in 'operator' should be a bit faster than defining your own in pure Python (or the pure Python implementations that get used if the _operator C module is not available for some reason): https://github.com/python/cpython/blob/3.9/Modules/_operator.c https://github.com/python/cpython/blob/3.9/Modules/_operator...
- stormbrew 5y ago
  I mean, it's the reason the benchmark is slower, because the benchmark is comparing doing 2 things with only doing one of those things -- this is virtually tautological. So I agree that it is why it's slower, but I also agree that it's a mistake. This is the "hard part" about synthetic benchmarks. This one is measuring something people rarely do: creating an anonymous function and calling it in a tight loop. It's extremely unlikely you're ever going to run into a situation where creating this lambda is in a hot path, and if you do the thing you did wrong is very unlikely to be whether you used a standard library function or one you defined yourself. It's that you have a loop where you did almost nothing but create a lambda every iteration instead of, say, passing it to a map/fold function or whatever.

blester 5y ago

Generally it is un-pythonic to assign lambda to a variable. Lots of linters will complain if you do this.

sharikous 5y ago
So do a "def" instead, it can be even inlined. def add_fun(x,y): return x+y But I wouldn't mind irritating the linter with an assigned lambda. I fail to see what's the problem with it
- ViViDboarder 5y ago
  But if you’re doing that, why not just import the identical function?
  - sharikous 5y ago
    One major point of OP was that the functions defined in operators are faster. Grandparent comment argued that there is a measurement error that invalidates this point, since properly defined lambdas (or defs) are just as fast. Of course the point regarding the simplicity of importing a function instead of defining it still stands. I personally wouldn't mind defining those functions as they are very immediate. So from the article I would take only the part about the more exotic "operator".
    - klyrs 5y ago
      The timings are certainly closer, but the built-in function imported from operator is still faster than the user-defined function. Not a big deal if you're doing simple stuff, but if this code is in a hot loop that 30% can make a measurable difference.
- kevin_thibedeau 5y ago
  Lambdas don't know their name and weaken debug info.
  - elteto 5y ago
    It seems like an implementation issue of the interpreter no? There’s no reason lambdas _have_ to be this way.
    - jonathrg 5y ago
      lambdas don't have names, how could the interpreter possibly improve in this regard?
      - zodiakzz 5y ago
        JS arrow functions don't have names either, it's inferred from the left hand side variable name. Can't Python do something similar?
        jonathrg 5y ago
        If you assign a lambda to a variable, you might as well use a def. A more typical use of a lambda is as an arguments to a function call. What would be the name of, say, the first argument to map(lambda x: x*2, numbers)?
ris 5y ago
Linters will complain about all sorts of nonsense, and I'll have to continue to remind people that pep8 was only supposed to be guidance, and never something that was enforced. In short: being picked up by a linter does not mean that something is un-pythonic.
elteto 5y ago
The reasoning that I’ve seen for this is that you lose context information (file, line, etc) when using lambdas. Is that it? If so it seems like a an interpreter implementation issue.
jonathrg 5y ago
The point is that the function is created only once, like when you pass a lambda to map/filter/reduce or any function that takes a callback. The fact that it is assigned here is just a neccessary step for the timeit command to do the right thing

randallsquared 5y ago

Is that a mistake, or is it just the reason why the lambda style is slower? If you're assigning the lambda, then as @sharikous mentions below, you might as well just be using a def, which is exactly what is done in the operator module: https://github.com/python/cpython/blob/3.9/Lib/operator.py#L75 https://github.com/python/cpython/blob/3.9/Lib/operator.py#L... For someone familiar with the operator module, there is a final small argument for using it instead of defining your own, if the result will be the same: that's a definition that doesn't have to be understood by the reader.

kzrdude 5y ago
You're reading the pure-python version of operator.py. You get the builtin versions of the same functions - not the python functions when you use it - see the import of _operator further down the file. Also verify the type info on the object here: In [1]: import operator In [5]: operator.add?? Signature: operator.add(a, b, /) Docstring: Same as a + b. Type: builtin_function_or_method
JonathonW 5y ago
'operator' actually maps to C implementations if possible; you're still calling Python functions, but the implementations in 'operator' should be a bit faster than defining your own in pure Python (or the pure Python implementations that get used if the _operator C module is not available for some reason): https://github.com/python/cpython/blob/3.9/Modules/_operator.c https://github.com/python/cpython/blob/3.9/Modules/_operator...
stormbrew 5y ago
I mean, it's the reason the benchmark is slower, because the benchmark is comparing doing 2 things with only doing one of those things -- this is virtually tautological. So I agree that it is why it's slower, but I also agree that it's a mistake. This is the "hard part" about synthetic benchmarks. This one is measuring something people rarely do: creating an anonymous function and calling it in a tight loop. It's extremely unlikely you're ever going to run into a situation where creating this lambda is in a hot path, and if you do the thing you did wrong is very unlikely to be whether you used a standard library function or one you defined yourself. It's that you have a loop where you did almost nothing but create a lambda every iteration instead of, say, passing it to a map/fold function or whatever.