8 ms·
The pythonic way of creating a Point like data structure is not point = {'x': 0, 'y': 0} but instead, this Point = namedtuple('Point', ['x', 'y'], verbos
by kashif 13y ago
The pythonic way of creating a Point like data structure is not
point = {'x': 0, 'y': 0}
but instead, this
Point = namedtuple('Point', ['x', 'y'], verbose=True)
The resulting class, I believe to be more performant than using a dictionary - but I haven't actually measured this.
- a13xnet 13y agoHehe, came here to say exactly that.
- ramchip 13y agoIt's mentioned in the article, albeit at the end.
- adnam 13y agoWould't it be more like this? points = [(0, 0), (3, 4), ...] sum_x = sum([x for x,_ in points]) sum_y = sum([y for _,y in points])
- NigelTufnel 13y agoI did a stupid microbenchmark and it turned out that simple class with __slots__ was the fastest method.
- michaelmior 13y agoCurious if you included ctypes.Structure in your benchmark.
- NigelTufnel 13y agoI just did. And either a) I'm doing something terribly wrong or b) ctypes.Structure is slower than Python class with __slots__. I would bet on a) - I've never worked with ctypes before. EDIT: I bet on a) and I was wrong: ctypes offers no speed advantage.
- copx 13y agoWhile we are at stupid microbenchmarks. The following loop in C (line being a struct, length is int): for (int i = 1; i <= 400000000; ++i) { ++line.length; } Compiles to the following ASM: @8: inc eax dec edx jne @8 ..as expected. The compiler realized that there is no need to actually copy the intermediate values to line.length and does everything in registers. Now here is the same loop in Lua (everything dynamically typed, line.length must be hashed (in theory) just like in Python): for i = 1, 400000000 do line.length = line.length + 1 end LuaJIT 2.1 generates the following ASM: ->LOOP: addsd xmm7, xmm0 movsd [rax], xmm7 add edi, +0x01 cmp edi, 0x17d78400 jle 0x7fee13effe0 ->LOOP The C program executes in ~0.3s, the Lua one in ~0.5s .. and those 0.5s include LuaJIT's startup, profiling, and JIT compilation time. So for a longer running program the difference would be even smaller. Tl;dr: modern JIT compilers are amazing and can optimize away the things mentioned in the article.
- StillBored 13y agoI think you forgot to turn on your optimizer. GCC will optimize that loop into something along the lines of line.length+=400000000; That is because it can predict the loop result during compile time. For example, this is the entire program with your loop and a line to print the result (gcc 4.3.4/linux -O3), mov $0x17d78401,%esi mov $0x400654,%edi xor %eax,%eax jmpq 0x400430 <printf@plt> In fact without the printf all you get is "reqz ret" (not counting .init overhead in the binary). That is because the compiler detects that line.length is not used and fails to even set it. #include <stdio.h> int main(int argc,char *argv[]) { struct lines { char *somedata; int length; } line; int i; line.length=0; for (i=0;i<=400000000;++i) { ++line.length; } printf("Line len %d\n",line.length); }
- copx 13y ago>I think you forgot to turn on your optimizer. I did not. I used Pelles C to compile it (with the optimizer turned on). I am not surprised that GCC managed to eliminate the pointless loop entirely in this situation. In fact I was happy that both Pelles C and LuaJIT did not realize that the whole loop was pointless and thus I did not have to come up with a more complex example. The primary point of this was to show that JIT compilers can optimize away hash table access and dynamic typing in hot loops, not a code generation competition between LuaJIT and GCC. I am a pretty big LuaJIT fanboy and not even I would claim that Lua compiled with LuaJIT can compete against C compiled by current versions of GCC with all optimizations turned on, at least not in most real-world cases. However, it does get amazingly close, most of the time within an order of magnitude, which means that I personally do not need to use C anymore.
- tostitos1979 13y agoI'm curious where one would go to learn specifically about performance issues in Python - patterns to avoid, and those to stick to. Any recommendations?
- mouhtasi 13y agohttps://wiki.python.org/moin/PythonSpeed/PerformanceTips https://wiki.python.org/moin/PythonSpeed/PerformanceTips is a good start
- mjschultz 13y agoI'm probably doing it wrong but I measured exactly this yesterday and the dict version was faster: $ python -m timeit --setup "from collections import namedtuple; Point = namedtuple('Point', ['x', 'y']); p = Point(x=0, y=0)" "p.x + p.y" 1000000 loops, best of 3: 0.284 usec per loop vs. $ python -m timeit --setup "p = {'x': 0, 'y': 0}" "p['x'] + p['y']" 10000000 loops, best of 3: 0.0737 usec per loop Maybe the use isn't right because I agree with your belief that namedtuple is suppose to be more performant.
- mapleoin 13y agoI think you're timing the import in there, too. Plus, creating an object of a class rather than using a literal will always be slower.
- mjschultz 13y agoDoes the --setup statement really get timed? My reading of the docs[1] has always led me to believe that it doesn't. For example, "It is possible to provide a setup statement that is executed only once at the beginning" [1]: http://docs.python.org/2/library/timeit.html http://docs.python.org/2/library/timeit.html
- deleted 13y ago[deleted]
- the_mitsuhiko 13y agoYep. Because the dictionary literal gets created via opcodes in the interpreter loop whereas the named tuple pays the price of calling a function and setting up an interpreter frame.
- freyrs3 13y ago> The resulting class, I believe to be more performant than using a dictionary - but I haven't actually measured this. If you didn't measure it then how did you come to believe this?