6 ms·

Why is my x64 process getting heap address above 4GB on Windows 8?

54 points by ScottWRobinson 11y ago

the8472 11y ago
> Why are they truncating 64-bit pointers to 32-bit values? Maybe to get more compact data structures? But in that case they shouldn't rely on that and try detecting that behavior at runtime. E.g. the hotspot jvm tries to map memory in the low end of address space to compress pointers but falls back to full pointers when that fails.
- electrograv 11y ago
  > Maybe to get more compact data structures? Even if this is why, there is never a valid excuse for blindly truncating 64-bit pointer to 32-bits. It's just like saying "I'm gonna just go ahead and truncate all bank account balances to 16-bit integers to save memory in my data structures." It may be the reason, but that reason is still incredibly stupid.
  - cnvogel 11y ago
    > It may be the reason, but that reason is > still incredibly stupid. Sometimes there are stupid reasons, but sometimes even reasons which are incredibly stupid from a technical standpoint make sense in a broader context. For a short-term fix, for example, I can very well imagine to prefer this "trick" forcing sub-4G allocations if the other alternative would be to change 1000 places in undocumented legacy code doing crazy casts... And if it's for a product (or product component) on life-support only needed for a forseeable future, to me it makes perfect sense to ask the initial "crazy" question, even if it makes me cringe...
    - sjolsen 11y ago
      I'd suggest that even if it's absolutely necessary, one could at least put something like #if (ULONG_MAX < UINTPTR_MAX) && !defined (SUPPRESS_TRUNCATION_ERROR) #error Truncating pointers to long is a bad idea here #endif in some source file or another, just so you have some indication whether or not things are going to break when you change the build environment.
      - cpeterso 11y ago
        static_assert(sizeof(DWORD) == sizeof(void*), "assume 32-bit pointers");
        sjolsen 11y ago
        If we're talking about "undocumented legacy code doing crazy casts," static_assert is probably not an option.
    - cpeterso 11y ago
      Perhaps the maintainer of the legacy application can override operator new to use a custom memory allocator that uses a block of memory allocated below 4GB using VirtualAlloc.
    - xg15 11y ago
      > For a short-term fix, for example, I can very well imagine to prefer this "trick" forcing sub-4G allocations ... But that's not what this is doing - it's taking an address allocated at an arbitrary place and then pretends that it was allocated sub-4G, whether or not it actually was. If it happens that it wasn't, this will likely cause memory corruption or an access violation. Even if your product is nearing end-of-life, I'd still consider such a move a big "f* you" to your customers as you'd basically be tolerating that your product may crash without warning at any time. From what I understood, what makes this idea so bad isn't truncating to 32bit - it's truncating to 32 bit while setting the LARGEADDRESSAWARE build flag - which basically tells the OS "I'm fine with all kinds of 64bit addresses, bring them on!" The correct fix is outlined at the end of the article - and it does not involve combing through undocumented legacy code: > If there is some fundamental reason that they have to truncate pointers to 32-bit values, then they should build without /LARGEADDRESSAWARE so that the process will be given an address space of only 2GB, and then they can truncate their pointers all they want.
- cesarb 11y ago
  As mentioned on the blog post's comments, it's often caused either by storing a pointer in a DWORD variable (very common on 32-bit Windows), or by storing a pointer in a "long" variable (on most common 32-bit and 64-bit platforms, a "long" is at least as wide as a pointer; the main exception is 64-bit Windows). The correct variable type to store a pointer would be intptr_t/uintptr_t for portable software (and DWORD_PTR for Windows-only software), but the "intptr" types didn't exist until the 1999 C standard (and, as far as I could find, the default Windows compiler didn't add the corresponding header file until around 2010). Software older than that would often assume that a "long" (or a DWORD, which IIRC was just another name for "long") was big enough to store a pointer, or perhaps tried to use the C99 types and used "long" if the header wasn't found.
  - TillE 11y ago
    > storing a pointer in a DWORD variable (very common on 32-bit Windows) Is it? I mean, if you're lucky it won't cause a compiler warning on x86, but the sane thing to do is to use the types which the API docs specify. Which would usually be things like LPVOID or HANDLE. That's basically how I managed to port a service and several drivers to 64-bit Windows many years ago with almost no changes.
    - acqq 11y ago
      At the time the code was written the warnings you get now didn't exist. Then the 32-bit code worked, so nobody needed to change the it. Now with windows 8 they see it for the first time that it doesn't work. I know as I also worked on the huge codebase where the oldest pieces of the code were written around 1995.
- netheril96 11y ago
  There are tons of C/C++ codes that use `int` for every type of integer, including indexing arrays and storing pointer values. A relic of the time where everything was 32-bit. It is common on Unix system as well. When you compile C/C++ library, a Python c-extension, there are usually many warnings about integer truncation.
  - kardos 11y ago
    aka bugs!
- aardvark179 11y ago
  Hotspot is really quite subtle, and thorough in its strategy. Because of the alignment of java objects it can both shift and truncate pointers (reversing the shift when they are actually used), but can also handle the heap not being allocated in the low end of address space by storing the offset of the base address, and adding/subtracting that as needed as well.
phs2501 11y ago
This bit me in a library of mine. Some third-party code I used was using long as memory offsets which failed on Windows 8. Unfortunately I was developing on Windows 7, so it didn't get noticed until the customer stumbled over it. :( (Fortunately said third-party code had an update that switched to ptrdiff_t in all the right places, so it was an easy fix.) I'm a Unix person, so the notion of long being less than word-length seems pretty silly to me, but I understand why they did it, sort of.
- sjolsen 11y ago
  >I'm a Unix person, so the notion of long being less than word-length seems pretty silly to me It's silly for Windows to do something a certain way just because that's not how Unix does it? What's silly is assuming that long and unsigned long are more than 32 bits wide, since neither C nor C++ has ever guaranteed anything more.
  - sp332 11y ago
    If it's not specified, implementers have the option of making it wider. Matching the width of a pointer seems like a better choice.
    - cremno 11y ago
      Yeah however the C standard recommends that the integer conversion rank of size_t and ptrdiff_t is lower or equal to long. http://port70.net/~nsz/c/c11/n1570.html#7.19p4 http://port70.net/~nsz/c/c11/n1570.html#7.19p4
      - asveikau 11y ago
        From a certain angle it would seem size_t and ptrdiff_t are meant to roughly correspond to array subscript use cases. Therefore it's not unheard of for them to not be as wide as a pointer. intptr_t is for that. In your link they have examples where size_t and ptrdiff_t are 16 bits. Obviously this is absurd on today's machines. But you can imagine a machine where array subscript is most convenient at 16 bits and total addressable space is larger (16 bit x86 comes to mind). By the same token I don't think it would be too weird to have a 32 bit size_t on a 64 bit architecture. It would just make memcpy et al more annoying when your allocations are large. Anyway the quotation from your link: > The types used for size_t and ptrdiff_t should not have an integer conversion rank greater than that of signed long int unless the implementation supports objects large enough to make this necessary. I guess if longs are 32bit and you can have a 4g allocation this pretty well follows the spirit here.
copx 11y ago
Pointer packing is popular with people wanting to write efficient interpreters for programming languages. The memory use and performance gains are massive and it was perfectly safe during the 32-bit era. LuaJIT does it for example. Interestingly enough some programmers do this without even understanding how memory allocation works. For example the author of the newish wren language: https://github.com/munificent/wren https://github.com/munificent/wren Somehow most programmers - including almost everyone who frequents reddit.com/r/programming - believe that "virtual address space" means that the addresses returned by malloc() will restart from (somewhere close to) 0x0 for every new process i.e. that it is safe to truncate pointers if you never intent to allocate more than ~2GB per process anyway.
- beagle3 11y ago
  LuaJIT does something weirder. It does double packing - using 48 bits (IIRC) of value inside a "NaN" double encoding. It is going to be a while before this becomes a problem, even on 64-bit systems - you need more than 128TB addressable space before this becomes a problem (and I suspect Pall will switch to a 51-bit (1024TB=1PB) pointer when that future is on the horizon).
  - copx 11y ago
    No that is wrong. Currently the x64 port of LuaJIT is limited to about 1GB [1] in the best case and can fail to allocate any Lua objects at all if the lower region of the address space is already completely consumed. [1] http://lua-users.org/lists/lua-l/2010-11/msg00241.html http://lua-users.org/lists/lua-l/2010-11/msg00241.html
    - beagle3 11y ago
      Thanks. A slightly later discussion I found is here[0]. However, this is not a limit imposed by the tag structure used by LuaJIT, but rather the Lua/LuaJIT GC (from a quick reading of the mailing list discussion - I might be wrong here). Apparently, there's a GC in the works that is supposed to resolve this, but it isn't there yet. [0] http://lua-users.org/lists/lua-l/2012-04/msg00729.html http://lua-users.org/lists/lua-l/2012-04/msg00729.html
daniel02216 11y ago
This is one reason why OS X enforces a hard page-zero on 64-bit programs - it is a hard error to map or allocate anything in the lowest 4 GB of memory, so programmers will immediately discover pointer truncation bugs when porting 32-bit code to 64-bit.
- cokernel_hacker 11y ago
  What if you have a load command in your mach-o which asks for an address below the 32-bit boundary? Will loading the image fail?
fsloth 11y ago
Long was not 64 bits? This is why C standard has stdint.h which typedefs numeric types of specific length (for 64 bit integers use int64_t, etc). It seems Visual Studio also has it nowadays https://msdn.microsoft.com/en-us/library/hh874765.aspx https://msdn.microsoft.com/en-us/library/hh874765.aspx.

> Why are they truncating 64-bit pointers to 32-bit values? Maybe to get more compact data structures? But in that case they shouldn't rely on that and try detecting that behavior at runtime. E.g. the hotspot jvm tries to map memory in the low end of address space to compress pointers but falls back to full pointers when that fails.

electrograv 11y ago
> Maybe to get more compact data structures? Even if this is why, there is never a valid excuse for blindly truncating 64-bit pointer to 32-bits. It's just like saying "I'm gonna just go ahead and truncate all bank account balances to 16-bit integers to save memory in my data structures." It may be the reason, but that reason is still incredibly stupid.
- cnvogel 11y ago
  > It may be the reason, but that reason is > still incredibly stupid. Sometimes there are stupid reasons, but sometimes even reasons which are incredibly stupid from a technical standpoint make sense in a broader context. For a short-term fix, for example, I can very well imagine to prefer this "trick" forcing sub-4G allocations if the other alternative would be to change 1000 places in undocumented legacy code doing crazy casts... And if it's for a product (or product component) on life-support only needed for a forseeable future, to me it makes perfect sense to ask the initial "crazy" question, even if it makes me cringe...
  - sjolsen 11y ago
    I'd suggest that even if it's absolutely necessary, one could at least put something like #if (ULONG_MAX < UINTPTR_MAX) && !defined (SUPPRESS_TRUNCATION_ERROR) #error Truncating pointers to long is a bad idea here #endif in some source file or another, just so you have some indication whether or not things are going to break when you change the build environment.
    - cpeterso 11y ago
      static_assert(sizeof(DWORD) == sizeof(void*), "assume 32-bit pointers");
      - sjolsen 11y ago
        If we're talking about "undocumented legacy code doing crazy casts," static_assert is probably not an option.
  - cpeterso 11y ago
    Perhaps the maintainer of the legacy application can override operator new to use a custom memory allocator that uses a block of memory allocated below 4GB using VirtualAlloc.
  - xg15 11y ago
    > For a short-term fix, for example, I can very well imagine to prefer this "trick" forcing sub-4G allocations ... But that's not what this is doing - it's taking an address allocated at an arbitrary place and then pretends that it was allocated sub-4G, whether or not it actually was. If it happens that it wasn't, this will likely cause memory corruption or an access violation. Even if your product is nearing end-of-life, I'd still consider such a move a big "f* you" to your customers as you'd basically be tolerating that your product may crash without warning at any time. From what I understood, what makes this idea so bad isn't truncating to 32bit - it's truncating to 32 bit while setting the LARGEADDRESSAWARE build flag - which basically tells the OS "I'm fine with all kinds of 64bit addresses, bring them on!" The correct fix is outlined at the end of the article - and it does not involve combing through undocumented legacy code: > If there is some fundamental reason that they have to truncate pointers to 32-bit values, then they should build without /LARGEADDRESSAWARE so that the process will be given an address space of only 2GB, and then they can truncate their pointers all they want.
cesarb 11y ago
As mentioned on the blog post's comments, it's often caused either by storing a pointer in a DWORD variable (very common on 32-bit Windows), or by storing a pointer in a "long" variable (on most common 32-bit and 64-bit platforms, a "long" is at least as wide as a pointer; the main exception is 64-bit Windows). The correct variable type to store a pointer would be intptr_t/uintptr_t for portable software (and DWORD_PTR for Windows-only software), but the "intptr" types didn't exist until the 1999 C standard (and, as far as I could find, the default Windows compiler didn't add the corresponding header file until around 2010). Software older than that would often assume that a "long" (or a DWORD, which IIRC was just another name for "long") was big enough to store a pointer, or perhaps tried to use the C99 types and used "long" if the header wasn't found.
- TillE 11y ago
  > storing a pointer in a DWORD variable (very common on 32-bit Windows) Is it? I mean, if you're lucky it won't cause a compiler warning on x86, but the sane thing to do is to use the types which the API docs specify. Which would usually be things like LPVOID or HANDLE. That's basically how I managed to port a service and several drivers to 64-bit Windows many years ago with almost no changes.
  - acqq 11y ago
    At the time the code was written the warnings you get now didn't exist. Then the 32-bit code worked, so nobody needed to change the it. Now with windows 8 they see it for the first time that it doesn't work. I know as I also worked on the huge codebase where the oldest pieces of the code were written around 1995.
netheril96 11y ago
There are tons of C/C++ codes that use `int` for every type of integer, including indexing arrays and storing pointer values. A relic of the time where everything was 32-bit. It is common on Unix system as well. When you compile C/C++ library, a Python c-extension, there are usually many warnings about integer truncation.
- kardos 11y ago
  aka bugs!
aardvark179 11y ago
Hotspot is really quite subtle, and thorough in its strategy. Because of the alignment of java objects it can both shift and truncate pointers (reversing the shift when they are actually used), but can also handle the heap not being allocated in the low end of address space by storing the offset of the base address, and adding/subtracting that as needed as well.

phs2501 11y ago

This bit me in a library of mine. Some third-party code I used was using long as memory offsets which failed on Windows 8. Unfortunately I was developing on Windows 7, so it didn't get noticed until the customer stumbled over it. :( (Fortunately said third-party code had an update that switched to ptrdiff_t in all the right places, so it was an easy fix.) I'm a Unix person, so the notion of long being less than word-length seems pretty silly to me, but I understand why they did it, sort of.

sjolsen 11y ago
>I'm a Unix person, so the notion of long being less than word-length seems pretty silly to me It's silly for Windows to do something a certain way just because that's not how Unix does it? What's silly is assuming that long and unsigned long are more than 32 bits wide, since neither C nor C++ has ever guaranteed anything more.
- sp332 11y ago
  If it's not specified, implementers have the option of making it wider. Matching the width of a pointer seems like a better choice.
  - cremno 11y ago
    Yeah however the C standard recommends that the integer conversion rank of size_t and ptrdiff_t is lower or equal to long. http://port70.net/~nsz/c/c11/n1570.html#7.19p4 http://port70.net/~nsz/c/c11/n1570.html#7.19p4
    - asveikau 11y ago
      From a certain angle it would seem size_t and ptrdiff_t are meant to roughly correspond to array subscript use cases. Therefore it's not unheard of for them to not be as wide as a pointer. intptr_t is for that. In your link they have examples where size_t and ptrdiff_t are 16 bits. Obviously this is absurd on today's machines. But you can imagine a machine where array subscript is most convenient at 16 bits and total addressable space is larger (16 bit x86 comes to mind). By the same token I don't think it would be too weird to have a 32 bit size_t on a 64 bit architecture. It would just make memcpy et al more annoying when your allocations are large. Anyway the quotation from your link: > The types used for size_t and ptrdiff_t should not have an integer conversion rank greater than that of signed long int unless the implementation supports objects large enough to make this necessary. I guess if longs are 32bit and you can have a 4g allocation this pretty well follows the spirit here.

copx 11y ago

Pointer packing is popular with people wanting to write efficient interpreters for programming languages. The memory use and performance gains are massive and it was perfectly safe during the 32-bit era. LuaJIT does it for example. Interestingly enough some programmers do this without even understanding how memory allocation works. For example the author of the newish wren language: https://github.com/munificent/wren https://github.com/munificent/wren Somehow most programmers - including almost everyone who frequents reddit.com/r/programming - believe that "virtual address space" means that the addresses returned by malloc() will restart from (somewhere close to) 0x0 for every new process i.e. that it is safe to truncate pointers if you never intent to allocate more than ~2GB per process anyway.

beagle3 11y ago
LuaJIT does something weirder. It does double packing - using 48 bits (IIRC) of value inside a "NaN" double encoding. It is going to be a while before this becomes a problem, even on 64-bit systems - you need more than 128TB addressable space before this becomes a problem (and I suspect Pall will switch to a 51-bit (1024TB=1PB) pointer when that future is on the horizon).
- copx 11y ago
  No that is wrong. Currently the x64 port of LuaJIT is limited to about 1GB [1] in the best case and can fail to allocate any Lua objects at all if the lower region of the address space is already completely consumed. [1] http://lua-users.org/lists/lua-l/2010-11/msg00241.html http://lua-users.org/lists/lua-l/2010-11/msg00241.html
  - beagle3 11y ago
    Thanks. A slightly later discussion I found is here[0]. However, this is not a limit imposed by the tag structure used by LuaJIT, but rather the Lua/LuaJIT GC (from a quick reading of the mailing list discussion - I might be wrong here). Apparently, there's a GC in the works that is supposed to resolve this, but it isn't there yet. [0] http://lua-users.org/lists/lua-l/2012-04/msg00729.html http://lua-users.org/lists/lua-l/2012-04/msg00729.html

daniel02216 11y ago

This is one reason why OS X enforces a hard page-zero on 64-bit programs - it is a hard error to map or allocate anything in the lowest 4 GB of memory, so programmers will immediately discover pointer truncation bugs when porting 32-bit code to 64-bit.

cokernel_hacker 11y ago
What if you have a load command in your mach-o which asks for an address below the 32-bit boundary? Will loading the image fail?

fsloth 11y ago

Long was not 64 bits? This is why C standard has stdint.h which typedefs numeric types of specific length (for 64 bit integers use int64_t, etc). It seems Visual Studio also has it nowadays https://msdn.microsoft.com/en-us/library/hh874765.aspx https://msdn.microsoft.com/en-us/library/hh874765.aspx.