4 ms·

Pointer packing is popular with people wanting to write efficient interpreters for programming languages. The memory use and performance gains are massive and i

by copx 11y ago

Pointer packing is popular with people wanting to write efficient interpreters for programming languages. The memory use and performance gains are massive and it was perfectly safe during the 32-bit era. LuaJIT does it for example. Interestingly enough some programmers do this without even understanding how memory allocation works. For example the author of the newish wren language: https://github.com/munificent/wren https://github.com/munificent/wren Somehow most programmers - including almost everyone who frequents reddit.com/r/programming - believe that "virtual address space" means that the addresses returned by malloc() will restart from (somewhere close to) 0x0 for every new process i.e. that it is safe to truncate pointers if you never intent to allocate more than ~2GB per process anyway.

beagle3 11y ago
LuaJIT does something weirder. It does double packing - using 48 bits (IIRC) of value inside a "NaN" double encoding. It is going to be a while before this becomes a problem, even on 64-bit systems - you need more than 128TB addressable space before this becomes a problem (and I suspect Pall will switch to a 51-bit (1024TB=1PB) pointer when that future is on the horizon).
- copx 11y ago
  No that is wrong. Currently the x64 port of LuaJIT is limited to about 1GB [1] in the best case and can fail to allocate any Lua objects at all if the lower region of the address space is already completely consumed. [1] http://lua-users.org/lists/lua-l/2010-11/msg00241.html http://lua-users.org/lists/lua-l/2010-11/msg00241.html
  - beagle3 11y ago
    Thanks. A slightly later discussion I found is here[0]. However, this is not a limit imposed by the tag structure used by LuaJIT, but rather the Lua/LuaJIT GC (from a quick reading of the mailing list discussion - I might be wrong here). Apparently, there's a GC in the works that is supposed to resolve this, but it isn't there yet. [0] http://lua-users.org/lists/lua-l/2012-04/msg00729.html http://lua-users.org/lists/lua-l/2012-04/msg00729.html

by copx 11y ago

beagle3 11y ago

LuaJIT does something weirder. It does double packing - using 48 bits (IIRC) of value inside a "NaN" double encoding. It is going to be a while before this becomes a problem, even on 64-bit systems - you need more than 128TB addressable space before this becomes a problem (and I suspect Pall will switch to a 51-bit (1024TB=1PB) pointer when that future is on the horizon).

copx 11y ago
No that is wrong. Currently the x64 port of LuaJIT is limited to about 1GB [1] in the best case and can fail to allocate any Lua objects at all if the lower region of the address space is already completely consumed. [1] http://lua-users.org/lists/lua-l/2010-11/msg00241.html http://lua-users.org/lists/lua-l/2010-11/msg00241.html
- beagle3 11y ago
  Thanks. A slightly later discussion I found is here[0]. However, this is not a limit imposed by the tag structure used by LuaJIT, but rather the Lua/LuaJIT GC (from a quick reading of the mailing list discussion - I might be wrong here). Apparently, there's a GC in the works that is supposed to resolve this, but it isn't there yet. [0] http://lua-users.org/lists/lua-l/2012-04/msg00729.html http://lua-users.org/lists/lua-l/2012-04/msg00729.html