6 ms·

> The PHP interpreter is also quite memory hungry, as interpreters go. Any PHP value in your program uses 68 bytes of overhead [6]. An array of a million values

> The PHP interpreter is also quite memory hungry, as interpreters go. Any PHP value in your program uses 68 bytes of overhead [6]. An array of a million values takes over 68 MB. Well, that is obscene.

Erwin 17y ago
Python (and probably every other dynamically typed language that does use the tagged pointers trick) has the same issue, even worse on 64-bit. Allocating [sys.maxint-1 for x in xrange (1000 * 1000)] takes up 36 megabytes of memory. Allocating dict((x, float(sys.maxint-1)) for x in xrange (1000 * 1000)), which is closer to PHP's combined arrays-dictionaries and values which I believe are always floats (unless I'm thinking of Perl or Javascript?), takes up 97 megabytes. Of course if you really need 1 million floating point values, that are not going to be None and that are contigously indexed 0 to 999999, you could do: array.array('d',(float(sys.maxint-1) for x in xrange(1000 * 1000))) which has no overhead. But for a random collection of objects with attributes and lists and dictionary, the overhead exists. I've got 15 gigabytes of Python objects on this box across a couple dozens of processes and sometimes I dream of a time where every bit was accounted for, where for every bit I could trace back to where it was set and why and accounted for it. As for the "tagged pointers" trick -- basically taking advantage of that pointers to objects are going to always have the last bottom bits set to 0 and saying that if they are set to 01 for example, that the top 32 or 62 bits are an immediate integer -- I think someone tried it out in Python a few years ago and the result were that it was no win overall, though it might have decreased memory usage occasionally.
- deleted 17y ago
  [deleted]
- lanstein 17y ago
  I was thinking more about the overhead of building arrays of small strings. Depending on how you measure memory usage, Python is 3-4x more efficient at building lists of one-byte strings than PHP is wrt memory usage.
- vilya 17y ago
  Python's float type is a double.
jemfinch 17y ago
I'd really like to see the source of that overhead. I'm looking at the Zend source and only seeing 16-20 bytes of overhead per `zval`, depending on alignment.
- pbiggar 17y ago
  OK, I misspoke. I'll quote from someone who knows this stuff way better than me: "As a result of the function-local symbol-table, each PHP variable uses a great amount of space. The zval itself is 16 bytes long. However, the symbol-table is a hashtable with a 36 byte bucket. Combined with memory allocation overhead, each variable occupies 68 bytes on a 32-bit platform [?], and nearly double that on a 64-bit platform. " - me, sometime within the last two years.
  - jemfinch 17y ago
    Ah, great -- thanks. I found the fuller email whence that came at http://www.mail-archive.com/internals@lists.php.net/msg35235.html http://www.mail-archive.com/internals@lists.php.net/msg35235... , in case anyone else is interested.

Erwin 17y ago

Python (and probably every other dynamically typed language that does use the tagged pointers trick) has the same issue, even worse on 64-bit. Allocating [sys.maxint-1 for x in xrange (1000 * 1000)] takes up 36 megabytes of memory. Allocating dict((x, float(sys.maxint-1)) for x in xrange (1000 * 1000)), which is closer to PHP's combined arrays-dictionaries and values which I believe are always floats (unless I'm thinking of Perl or Javascript?), takes up 97 megabytes. Of course if you really need 1 million floating point values, that are not going to be None and that are contigously indexed 0 to 999999, you could do: array.array('d',(float(sys.maxint-1) for x in xrange(1000 * 1000))) which has no overhead. But for a random collection of objects with attributes and lists and dictionary, the overhead exists. I've got 15 gigabytes of Python objects on this box across a couple dozens of processes and sometimes I dream of a time where every bit was accounted for, where for every bit I could trace back to where it was set and why and accounted for it. As for the "tagged pointers" trick -- basically taking advantage of that pointers to objects are going to always have the last bottom bits set to 0 and saying that if they are set to 01 for example, that the top 32 or 62 bits are an immediate integer -- I think someone tried it out in Python a few years ago and the result were that it was no win overall, though it might have decreased memory usage occasionally.

deleted 17y ago
[deleted]
lanstein 17y ago
I was thinking more about the overhead of building arrays of small strings. Depending on how you measure memory usage, Python is 3-4x more efficient at building lists of one-byte strings than PHP is wrt memory usage.
vilya 17y ago
Python's float type is a double.

jemfinch 17y ago

I'd really like to see the source of that overhead. I'm looking at the Zend source and only seeing 16-20 bytes of overhead per `zval`, depending on alignment.

pbiggar 17y ago
OK, I misspoke. I'll quote from someone who knows this stuff way better than me: "As a result of the function-local symbol-table, each PHP variable uses a great amount of space. The zval itself is 16 bytes long. However, the symbol-table is a hashtable with a 36 byte bucket. Combined with memory allocation overhead, each variable occupies 68 bytes on a 32-bit platform [?], and nearly double that on a 64-bit platform. " - me, sometime within the last two years.
- jemfinch 17y ago
  Ah, great -- thanks. I found the fuller email whence that came at http://www.mail-archive.com/internals@lists.php.net/msg35235.html http://www.mail-archive.com/internals@lists.php.net/msg35235... , in case anyone else is interested.