4 ms·

There are architectures where all you have is word addressing to memory. If you want to get a specific byte out, you need to retrieve it and shift/mask yoursel

by mlyle 4y ago

There are architectures where all you have is word addressing to memory. If you want to get a specific byte out, you need to retrieve it and shift/mask yourself. In turn, a pointer to a byte is a software construct rather than something there's actual direct architectural support for.

loeg 4y ago
Do C compilers for those platforms transparently implement this for your char pointers as GP suggests? I would expect that you would need to do it manually and that native C pointers would only address the same words as the machine itself.
- mlyle 4y ago
  > Do C compilers for those platforms transparently implement this for your char pointers as GP suggests? Yes. Lots of little microcontrollers and older big machines have this "feature" and C compilers fix it for you. There are nightmarish microcontrollers with Harvard architectures and standards-compliant C compilers that fix this up all behind the scenes for you. E.g. the 8051 is ubiquitous, and it has a Harvard architecture: there are separate buses/instructions to access program memory and normal data memory. The program memory is only word addressable, and the data memory is byte addressable. So, a "pointer" in many C environments for 8051 says what bus the data is on and stashes in other bits what the byte address is, if applicable. And dereferencing the pointer involves a whole lot of conditional operations. Then there's things like the PDP-10, where there's hardware support for doing fancy things with byte pointers, but the pointers still have a different format than word pointers (e.g. they stash the byte offset in the high bits, not the low bits). The C standards makes relatively few demands upon pointers so that you can do interesting things if necessary for an architecture.
- billpg 4y ago
  Depends on how helpful the compiler is. This particular compiler had an option to switch off adding in bit shifting code when reading characters and instead set CHAR_BIT to 32, meaning strings would have each character taking up 32 bits of space. (So many zero bits, but already handles emojis.)
- yurish 4y ago
  I have seen a DSP processor that could address only 16-bit words. And C compiler did not fix it, bytes had 16 bits there.
  - loeg 4y ago
    Yeah, this is what I have heard of and was expecting. Sibling comment says it's not universal -- some C compilers for these platforms emulate byte addressing.
csense 4y ago
x86 is byte addressable, but internally, the x86 memory bus is word addressable. So an x86 CPU does the shift/mask process you're referring to internally. Which means it's actually slower to access (for example) a 32-bit value that is not aligned to a 4-byte boundary. C/C++ compilers often by default add extra bytes if necessary to make sure everything's aligned. So if you have struct X { int a; char b; int c; char d; } and struct Y { int a; int b; char c; char d; } actually X takes up more memory than Y, because X needs 6 extra bytes to align the int fields to 32-bit boundaries (or 14 bytes to align to a 64-bit boundary) while Y only needs 2 bytes (or 6 bytes for 64-bit). Meaning you can sometimes save significant amounts of memory in a C/C++ program by re-ordering struct fields [1]. [1] http://www.catb.org/esr/structure-packing/ http://www.catb.org/esr/structure-packing/
- mlyle 4y ago
  Sure, unaligned access to memory is always expensive (on architectures that allow it at all). But I'm talking about retrieving the 9th to 16th bit of a word, which is a little different. x86 does this just fine/quickly, because bytes are addressable.
- moonchild 4y ago
  > internally, the x86 memory bus is word addressable The 'memory bus' is not architectural. Different microarchitectures implement things differently, but most high-performance microarchitectures these days have relatively efficient misaligned accesses.
  - csense 4y ago
    > The 'memory bus' is not architectural Whether we classify the issue as "architectural" (whatever that means) is beside the point. Alignment has real effects on performance, and being aware of those effects is practically useful for working programmers. I do agree with you about one thing: The unaligned access penalty is probably less on the x86 CPU's of the 2020's than it was, say, on a 486 from the mid-1990's.

by mlyle 4y ago

loeg 4y ago

Do C compilers for those platforms transparently implement this for your char pointers as GP suggests? I would expect that you would need to do it manually and that native C pointers would only address the same words as the machine itself.

mlyle 4y ago
> Do C compilers for those platforms transparently implement this for your char pointers as GP suggests? Yes. Lots of little microcontrollers and older big machines have this "feature" and C compilers fix it for you. There are nightmarish microcontrollers with Harvard architectures and standards-compliant C compilers that fix this up all behind the scenes for you. E.g. the 8051 is ubiquitous, and it has a Harvard architecture: there are separate buses/instructions to access program memory and normal data memory. The program memory is only word addressable, and the data memory is byte addressable. So, a "pointer" in many C environments for 8051 says what bus the data is on and stashes in other bits what the byte address is, if applicable. And dereferencing the pointer involves a whole lot of conditional operations. Then there's things like the PDP-10, where there's hardware support for doing fancy things with byte pointers, but the pointers still have a different format than word pointers (e.g. they stash the byte offset in the high bits, not the low bits). The C standards makes relatively few demands upon pointers so that you can do interesting things if necessary for an architecture.
billpg 4y ago
Depends on how helpful the compiler is. This particular compiler had an option to switch off adding in bit shifting code when reading characters and instead set CHAR_BIT to 32, meaning strings would have each character taking up 32 bits of space. (So many zero bits, but already handles emojis.)
yurish 4y ago
I have seen a DSP processor that could address only 16-bit words. And C compiler did not fix it, bytes had 16 bits there.
- loeg 4y ago
  Yeah, this is what I have heard of and was expecting. Sibling comment says it's not universal -- some C compilers for these platforms emulate byte addressing.

csense 4y ago

x86 is byte addressable, but internally, the x86 memory bus is word addressable. So an x86 CPU does the shift/mask process you're referring to internally. Which means it's actually slower to access (for example) a 32-bit value that is not aligned to a 4-byte boundary. C/C++ compilers often by default add extra bytes if necessary to make sure everything's aligned. So if you have struct X { int a; char b; int c; char d; } and struct Y { int a; int b; char c; char d; } actually X takes up more memory than Y, because X needs 6 extra bytes to align the int fields to 32-bit boundaries (or 14 bytes to align to a 64-bit boundary) while Y only needs 2 bytes (or 6 bytes for 64-bit). Meaning you can sometimes save significant amounts of memory in a C/C++ program by re-ordering struct fields [1]. [1] http://www.catb.org/esr/structure-packing/ http://www.catb.org/esr/structure-packing/

mlyle 4y ago
Sure, unaligned access to memory is always expensive (on architectures that allow it at all). But I'm talking about retrieving the 9th to 16th bit of a word, which is a little different. x86 does this just fine/quickly, because bytes are addressable.
moonchild 4y ago
> internally, the x86 memory bus is word addressable The 'memory bus' is not architectural. Different microarchitectures implement things differently, but most high-performance microarchitectures these days have relatively efficient misaligned accesses.
- csense 4y ago
  > The 'memory bus' is not architectural Whether we classify the issue as "architectural" (whatever that means) is beside the point. Alignment has real effects on performance, and being aware of those effects is practically useful for working programmers. I do agree with you about one thing: The unaligned access penalty is probably less on the x86 CPU's of the 2020's than it was, say, on a 486 from the mid-1990's.