3 ms·
Python (and probably every other dynamically typed language that does use the tagged pointers trick) has the same issue, even worse on 64-bit. Allocating [sys.m
by Erwin 17y ago
Python (and probably every other dynamically typed language that does use the tagged pointers trick) has the same issue, even worse on 64-bit. Allocating [sys.maxint-1 for x in xrange (1000 * 1000)] takes up 36 megabytes of memory.
Allocating dict((x, float(sys.maxint-1)) for x in xrange (1000 * 1000)), which is closer to PHP's combined arrays-dictionaries and values which I believe are always floats (unless I'm thinking of Perl or Javascript?), takes up 97 megabytes.
Of course if you really need 1 million floating point values, that are not going to be None and that are contigously indexed 0 to 999999, you could do: array.array('d',(float(sys.maxint-1) for x in xrange(1000 * 1000))) which has no overhead.
But for a random collection of objects with attributes and lists and dictionary, the overhead exists. I've got 15 gigabytes of Python objects on this box across a couple dozens of processes and sometimes I dream of a time where every bit was accounted for, where for every bit I could trace back to where it was set and why and accounted for it.
As for the "tagged pointers" trick -- basically taking advantage of that pointers to objects are going to always have the last bottom bits set to 0 and saying that if they are set to 01 for example, that the top 32 or 62 bits are an immediate integer -- I think someone tried it out in Python a few years ago and the result were that it was no win overall, though it might have decreased memory usage occasionally.
- deleted 17y ago[deleted]
- lanstein 17y agoI was thinking more about the overhead of building arrays of small strings. Depending on how you measure memory usage, Python is 3-4x more efficient at building lists of one-byte strings than PHP is wrt memory usage.
- vilya 17y agoPython's float type is a double.