4 ms·

> Just don't allow casting to u24, as it makes no sense unless you define u24 to be u32 sized as I think c standard does. The reason u32->u24 casting must be w

by jjmarr 3mo ago

> Just don't allow casting to u24, as it makes no sense unless you define u24 to be u32 sized as I think c standard does. The reason u32->u24 casting must be well defined is because some hardware (e.g. many GPUs, microcontrollers) only have floating point multipliers. A 24 bit unsigned integer (stored in a 32 bit register) can be losslessly converted to a 32 bit float by the hardware, multiplied, then converted back. This is much faster than doing 32 bit multiplication in software, however, you still need to tell the compiler about this constraint.

mathisfun123 3mo ago
> many GPUs Citation please - every single GPU in the literal world supports integer arithmetic for operating on tid, gid, etc.
- jmalicki 3mo ago
  While the GP might be technically wrong in a narrow sense, GPUs are built for FP, and that's what you want to be doing if you're using them as accelerators.
  - mathisfun123 3mo ago
    You don't know what you're talking about: an enormous amount of TOPs now runs through quantized (read: integer) kernels. Many GPUs don't have even FP64 or even FP32 support.
    - jmalicki 3mo ago
      EDIT: I was completely wrong, I have mostly worked with GGUF and related quantizations that are LUTs, thank you for correcting me.
      - mathisfun123 3mo ago
        > The quantized integer kernels aren't running true integer multiplication, the quantization is it's own thing, they're basically enums not integers ELI-a-GPU-compiler-engineer-working-at-a-major-vendor (because I am). Ie I can pull up the design docs for our ALUs and literally see that you're wrong.
- jjmarr 3mo ago
  From page 175 of the AMD CDNA4 ISA: https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-cdna4-instruction-set-architecture.pdf https://www.amd.com/content/dam/amd/en/documents/instinct-te... > V_MUL_U32_U24 >,Multiply two unsigned 24-bit integer inputs and store the result as an unsigned 32-bit integer into a vector register. D0.u32 = 32'U(S0.u24) * 32'U(S1.u24) > Notes > This opcode is expected to be as efficient as basic single-precision opcodes since it utilizes the single-precision floating point multiplier. See also V_MUL_HI_U32_U24. Nvidia GPUs used to do the same thing and theres a umul24 intrinsic if you care to use it. https://stackoverflow.com/questions/5544355/cuda-umul24-function-useful-or-not https://stackoverflow.com/questions/5544355/cuda-umul24-func... This is super-super-niche since it basically only applies to 32-bit integer multiplication. You likely won't run into it unless you're doing high performance embedded systems or GPU programming on non-NVDIA cards, and for some unknowable reason, your workload does a 32-bit integer multiplication in the hot path.
  - mathisfun123 3mo ago
    That's literally only for 32bx24b (I don't remember why we did that specifically for CDNA - I'll ask someone) but as you see from V_MUL_HI_I32, V_MUL_LO_U32 there is very much vector arithmetic hardware (nevermind that we're not talking about VALU but conventional scalar ALU).
    - imtringued 3mo ago
      I think he has a point, but I am still not 100% convinced by the arguments relating to casting. There is a difference between a u24 data type inside u32 and a u24 datatype inside u24 and that is what's so frustrating here. u24 is an alignment nightmare so it will basically never exist as "u24 in u24" and only ever as "u24 in u32". For casting to make sense, the alignment must be compatible and it's not clear how you can simultaneously make arbitrary bit data types simultaneously useful for the scenario of describing bit fields in packets, where padding is inherently undesirable and performing integer arithmetic with an FPU, where padding is an acceptable cost for alignment. These appear to be mutually exclusive use cases.
ozgrakkurt 3mo ago
I am criticizing the part where they allowed [3]u8 to u24 bitCast in the first place. It doesn't make sense logically as u24 is likely not 24 bits in any targets let alone portably on every target. Interpreting u24 like it is actually 24 bits sounds like programming in crazy land since it is not 24 bits in any relevant architecture afaik. They didn't allow []u24 with a similar rationale as far as I can remember. I agree with this as someone programming at this level should be able to understand there is no real u24 layout and they should use []u32. Going with the same magical rational they went with here, compiler should generate unaligned u24 loading code when you use []u24 since it is "logically 24 bits"
- nyrikki 3mo ago
  The ease of dealing with arbitrary bit-width integers and packed structs is actually one of the 'killer features' for me in zig. Zig natively supports arbitrary bit-width integers, the ABI is defined and you could simply think it as a slice of the next larger backing integer. The[3]u8 to u24 bitCast will simply be backed by a 32bit int, using the same ABI. As you have u1 - u65535, sometimes it can be multiple words. The 24 Bits (3 Bytes) [3]u8 to u24 example is exactly related to utf-8 that covers all the languages but excludes the emojis. There are very valid use cases when you want to limit utf-8 to U+0000-U+FFFF, and it is valuable if your language allows you to make those decisions. Remember, in zig packed structs are just integers and integers are just a group of logically consecutive bits. Arrays like []u24 do not have the same ABI, arrays are not bit/byte packed, are not universally LSB across archs etc.. The compiler isn't producing unaligned code, don't confuse the abstraction with the concrete implementation. And yes [8]u1 and [8]u8 are exactly the same size and shape, even though they are arrays. My current project is parsing ELF/Macho files, I can easily have zero allocations in my hot path with zig, the same is far more challenging in C, so I am biased, especially with zig allowing methods on structs. And yes, I do use that crazy casting to 0xdeadbeef and other ascii metadata that is in those files. To be clear here, I am not trying to prove you wrong, this is one of the places zig is very different and (IMHO) useful. Especially with streaming data or where you have network ordering etc... It is so nice to only cast what you need to but it does take a little while to wrap your head around how this interacts with buffers which are not your native endianness. At least for me, once I figured out to separate the shape of those data streams from their values it was super useful.

by jjmarr 3mo ago

mathisfun123 3mo ago

> many GPUs Citation please - every single GPU in the literal world supports integer arithmetic for operating on tid, gid, etc.

jmalicki 3mo ago
While the GP might be technically wrong in a narrow sense, GPUs are built for FP, and that's what you want to be doing if you're using them as accelerators.
- mathisfun123 3mo ago
  You don't know what you're talking about: an enormous amount of TOPs now runs through quantized (read: integer) kernels. Many GPUs don't have even FP64 or even FP32 support.
  - jmalicki 3mo ago
    EDIT: I was completely wrong, I have mostly worked with GGUF and related quantizations that are LUTs, thank you for correcting me.
    - mathisfun123 3mo ago
      > The quantized integer kernels aren't running true integer multiplication, the quantization is it's own thing, they're basically enums not integers ELI-a-GPU-compiler-engineer-working-at-a-major-vendor (because I am). Ie I can pull up the design docs for our ALUs and literally see that you're wrong.
jjmarr 3mo ago
From page 175 of the AMD CDNA4 ISA: https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-cdna4-instruction-set-architecture.pdf https://www.amd.com/content/dam/amd/en/documents/instinct-te... > V_MUL_U32_U24 >,Multiply two unsigned 24-bit integer inputs and store the result as an unsigned 32-bit integer into a vector register. D0.u32 = 32'U(S0.u24) * 32'U(S1.u24) > Notes > This opcode is expected to be as efficient as basic single-precision opcodes since it utilizes the single-precision floating point multiplier. See also V_MUL_HI_U32_U24. Nvidia GPUs used to do the same thing and theres a umul24 intrinsic if you care to use it. https://stackoverflow.com/questions/5544355/cuda-umul24-function-useful-or-not https://stackoverflow.com/questions/5544355/cuda-umul24-func... This is super-super-niche since it basically only applies to 32-bit integer multiplication. You likely won't run into it unless you're doing high performance embedded systems or GPU programming on non-NVDIA cards, and for some unknowable reason, your workload does a 32-bit integer multiplication in the hot path.
- mathisfun123 3mo ago
  That's literally only for 32bx24b (I don't remember why we did that specifically for CDNA - I'll ask someone) but as you see from V_MUL_HI_I32, V_MUL_LO_U32 there is very much vector arithmetic hardware (nevermind that we're not talking about VALU but conventional scalar ALU).
  - imtringued 3mo ago
    I think he has a point, but I am still not 100% convinced by the arguments relating to casting. There is a difference between a u24 data type inside u32 and a u24 datatype inside u24 and that is what's so frustrating here. u24 is an alignment nightmare so it will basically never exist as "u24 in u24" and only ever as "u24 in u32". For casting to make sense, the alignment must be compatible and it's not clear how you can simultaneously make arbitrary bit data types simultaneously useful for the scenario of describing bit fields in packets, where padding is inherently undesirable and performing integer arithmetic with an FPU, where padding is an acceptable cost for alignment. These appear to be mutually exclusive use cases.

ozgrakkurt 3mo ago

I am criticizing the part where they allowed [3]u8 to u24 bitCast in the first place. It doesn't make sense logically as u24 is likely not 24 bits in any targets let alone portably on every target. Interpreting u24 like it is actually 24 bits sounds like programming in crazy land since it is not 24 bits in any relevant architecture afaik. They didn't allow []u24 with a similar rationale as far as I can remember. I agree with this as someone programming at this level should be able to understand there is no real u24 layout and they should use []u32. Going with the same magical rational they went with here, compiler should generate unaligned u24 loading code when you use []u24 since it is "logically 24 bits"

nyrikki 3mo ago
The ease of dealing with arbitrary bit-width integers and packed structs is actually one of the 'killer features' for me in zig. Zig natively supports arbitrary bit-width integers, the ABI is defined and you could simply think it as a slice of the next larger backing integer. The[3]u8 to u24 bitCast will simply be backed by a 32bit int, using the same ABI. As you have u1 - u65535, sometimes it can be multiple words. The 24 Bits (3 Bytes) [3]u8 to u24 example is exactly related to utf-8 that covers all the languages but excludes the emojis. There are very valid use cases when you want to limit utf-8 to U+0000-U+FFFF, and it is valuable if your language allows you to make those decisions. Remember, in zig packed structs are just integers and integers are just a group of logically consecutive bits. Arrays like []u24 do not have the same ABI, arrays are not bit/byte packed, are not universally LSB across archs etc.. The compiler isn't producing unaligned code, don't confuse the abstraction with the concrete implementation. And yes [8]u1 and [8]u8 are exactly the same size and shape, even though they are arrays. My current project is parsing ELF/Macho files, I can easily have zero allocations in my hot path with zig, the same is far more challenging in C, so I am biased, especially with zig allowing methods on structs. And yes, I do use that crazy casting to 0xdeadbeef and other ascii metadata that is in those files. To be clear here, I am not trying to prove you wrong, this is one of the places zig is very different and (IMHO) useful. Especially with streaming data or where you have network ordering etc... It is so nice to only cast what you need to but it does take a little while to wrap your head around how this interacts with buffers which are not your native endianness. At least for me, once I figured out to separate the shape of those data streams from their values it was super useful.