4 ms·

According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows

by yan 18y ago

According to the comments on that article, the people still don't understand the basics of virtual memory. "Great article! / Quite shocked to know that windows takes double the kernel memory compared to Linux." "I’m wondering, though, why does the kernel space consume 1gb? That seems like a lot.."

narag 18y ago
Address space is not much bigger than physical memory now for 32 bits. I've read that my laptop could in fact have 4GB instead of the nominal 3GB. So even if the memory is not "consumed", the default must be changed at run time, unless the kernel space literally consumes 2/3 of available memory.
- Hoff 18y ago
  Given you don't reference what operating system is in use, you're probably using Microsoft Windows here. The implementation of virtual memory has common traits, but details vary by OS and even OS version. Physical address space and virtual address space are two different aspects of system design, and it is completely feasible to have these values be the same, or to have either of the two be larger than the other. Yes, you can have a 32-bit virtual memory system and 34 or more bits of physical memory, and there can be good reasons to do that, too. It's also feasible to have address "holes" in virtual address space, and "holes" in physical address space. Address ranges or gaps that are simply not implemented by the particular processor, or that are not instantiated by the memory controller. But I digress. Save for low-end boxes, all current gear is 64-bit virtual addressing (variously with "holes"), and most current systems can have 48 bits (x86-64) or 50 bits (IA-64) or other implementation-specific ranges for physical addressing. Details vary by processor and by implementation. Available memory varies by budget - 50 bits of memory is expensive - and target market for the box. And consuming address space is different than consuming memory. On various systems, address space is an inexpensive resource. And on a virtual memory system, memory is limited by physical memory and by backing storage and by your willingness to wait for the transitions between these two resources; for memory paging to occur. And what your application thinks is in memory might be on disk. When referenced, the data is valid or is paged in and made valid. Parts of an OS might be memory-resident, and parts can be paged. Applications tend to be paged. If you're programming in an area that has performance constraints, knowing details of the particular virtual memory and physical memory implementation can be a key to success. Issues around data alignment, multiprocessor memory caching and data structure layout can all be relevant; having a large and improperly-designed data structure can lead to memory paging for each data structure reference, for instance.
  - narag 18y ago
    I really don't understand. At home I've got an Asus F6V. At work it's Asus G50V. Both with Vista/Kubuntu dual boot. I really don't know what's happening with 32/64 bits here. Both use 32 bits OSs but G50V (64 bit machine) has 4GB, so it seems it isn't OS the piece that decides memory addressing capabilities :-)
    - scott_s 18y ago
      Memory addressing capabilities are determined by the processor. A 32-bit processor can only use up to 32-bits to represent an address. That means it can address 2^32 (4,294,967,296; 4GB) distinct bytes. But the memory is managed by the OS. So it's possible to have an OS that won't use more than 4GB even though the processor can handle more than that. Also, most modern x86 processors have a hack - Physical Address Extension (PAE) - that let the 32-bit processors address up to 64 GB.
a-priori 18y ago
As narag says, it's only recently that this has become an issue. When physical memory size was orders of magnitude smaller than than the address space, it didn't matter that the kernel carved out 1GB of address space for itself. User applications still had the other 3GB, which was far more than any system had physical memory. It's only during this limbo period when 4GB of RAM is relatively cheap, yet there are still 32-bit systems kicking around, that it becomes an issue. On a 64-bit system with a 16 exabyte address space, 1GB is a drop in the bucket. Once again, it won't matter. (I suspect Linux allocates more than this for itself in 64-bit systems, but I don't know the details. At any rate, out of 16EB, even 100GB would hardly be noticed.)
- yan 18y ago
  From the way I read those comments, I gathered that the comments implied that Windows "consumed" two gigs of ram, where as Linux took 1 gig. I thought there was a dissonance between how people understood "virtual memory," which is essentially a mapping function, to "memory," which people assume to be the physical ram.
  - a-priori 18y ago
    It's easy to make that mistake at first, but it is a very important distinction. Most of the time, for most processes, most of that space is unallocated. However, since it's basically impossible to change the size of the kernel area without relinking the process binaries, and since that area is shared between all processes, the size of that area has to remain fixed. It is interesting that Linux and NT differ. Perhaps this is due to the fact that NT needs more kernel memory to house, e.g. the windowing system, whereas Linux systems put X in userspace? I'm speculating here. Something I never considered: that design difference probably makes Wine's job easier, because between the 2GB and 3GB marks is a region of memory the Windows process assumes is inaccessible, yet in Linux is in userspace. They could (and probably do) load the Wine libraries here without interfering with the Windows process. It also means that it would be very difficult to implement Wine's complement, a way of running Linux binaries on Windows.
    - 18y ago
aaronblohowiak 18y ago
New people are being born all the time!
- yters 18y ago
  What about all the old people being born?

by yan 18y ago

narag 18y ago

Address space is not much bigger than physical memory now for 32 bits. I've read that my laptop could in fact have 4GB instead of the nominal 3GB. So even if the memory is not "consumed", the default must be changed at run time, unless the kernel space literally consumes 2/3 of available memory.

Hoff 18y ago
Given you don't reference what operating system is in use, you're probably using Microsoft Windows here. The implementation of virtual memory has common traits, but details vary by OS and even OS version. Physical address space and virtual address space are two different aspects of system design, and it is completely feasible to have these values be the same, or to have either of the two be larger than the other. Yes, you can have a 32-bit virtual memory system and 34 or more bits of physical memory, and there can be good reasons to do that, too. It's also feasible to have address "holes" in virtual address space, and "holes" in physical address space. Address ranges or gaps that are simply not implemented by the particular processor, or that are not instantiated by the memory controller. But I digress. Save for low-end boxes, all current gear is 64-bit virtual addressing (variously with "holes"), and most current systems can have 48 bits (x86-64) or 50 bits (IA-64) or other implementation-specific ranges for physical addressing. Details vary by processor and by implementation. Available memory varies by budget - 50 bits of memory is expensive - and target market for the box. And consuming address space is different than consuming memory. On various systems, address space is an inexpensive resource. And on a virtual memory system, memory is limited by physical memory and by backing storage and by your willingness to wait for the transitions between these two resources; for memory paging to occur. And what your application thinks is in memory might be on disk. When referenced, the data is valid or is paged in and made valid. Parts of an OS might be memory-resident, and parts can be paged. Applications tend to be paged. If you're programming in an area that has performance constraints, knowing details of the particular virtual memory and physical memory implementation can be a key to success. Issues around data alignment, multiprocessor memory caching and data structure layout can all be relevant; having a large and improperly-designed data structure can lead to memory paging for each data structure reference, for instance.
- narag 18y ago
  I really don't understand. At home I've got an Asus F6V. At work it's Asus G50V. Both with Vista/Kubuntu dual boot. I really don't know what's happening with 32/64 bits here. Both use 32 bits OSs but G50V (64 bit machine) has 4GB, so it seems it isn't OS the piece that decides memory addressing capabilities :-)
  - scott_s 18y ago
    Memory addressing capabilities are determined by the processor. A 32-bit processor can only use up to 32-bits to represent an address. That means it can address 2^32 (4,294,967,296; 4GB) distinct bytes. But the memory is managed by the OS. So it's possible to have an OS that won't use more than 4GB even though the processor can handle more than that. Also, most modern x86 processors have a hack - Physical Address Extension (PAE) - that let the 32-bit processors address up to 64 GB.

a-priori 18y ago

As narag says, it's only recently that this has become an issue. When physical memory size was orders of magnitude smaller than than the address space, it didn't matter that the kernel carved out 1GB of address space for itself. User applications still had the other 3GB, which was far more than any system had physical memory. It's only during this limbo period when 4GB of RAM is relatively cheap, yet there are still 32-bit systems kicking around, that it becomes an issue. On a 64-bit system with a 16 exabyte address space, 1GB is a drop in the bucket. Once again, it won't matter. (I suspect Linux allocates more than this for itself in 64-bit systems, but I don't know the details. At any rate, out of 16EB, even 100GB would hardly be noticed.)

yan 18y ago
From the way I read those comments, I gathered that the comments implied that Windows "consumed" two gigs of ram, where as Linux took 1 gig. I thought there was a dissonance between how people understood "virtual memory," which is essentially a mapping function, to "memory," which people assume to be the physical ram.
- a-priori 18y ago
  It's easy to make that mistake at first, but it is a very important distinction. Most of the time, for most processes, most of that space is unallocated. However, since it's basically impossible to change the size of the kernel area without relinking the process binaries, and since that area is shared between all processes, the size of that area has to remain fixed. It is interesting that Linux and NT differ. Perhaps this is due to the fact that NT needs more kernel memory to house, e.g. the windowing system, whereas Linux systems put X in userspace? I'm speculating here. Something I never considered: that design difference probably makes Wine's job easier, because between the 2GB and 3GB marks is a region of memory the Windows process assumes is inaccessible, yet in Linux is in userspace. They could (and probably do) load the Wine libraries here without interfering with the Windows process. It also means that it would be very difficult to implement Wine's complement, a way of running Linux binaries on Windows.
  - 18y ago

aaronblohowiak 18y ago

New people are being born all the time!

yters 18y ago
What about all the old people being born?