4 ms·

> This suggests the hardware is superscalar, with more 16-bit ALUs than 32-bit ALUs To me, it sounds like it might mean 32-bit ALUs can be used as two 16-bit o

> This suggests the hardware is superscalar, with more 16-bit ALUs than 32-bit ALUs To me, it sounds like it might mean 32-bit ALUs can be used as two 16-bit ones; that's how I would approach it, unless I'm missing something? The vectorization can also happen at the superscalar level, if borrowing the instruction queue concept from out-of-order designs: buffer operations for a while until you've filled a vector unit's worth, align input data in the pipeline, execute. A smart compiler could rearrange opcodes to avoid dependency issues, and insert "flushes" or filling operations at the right time.

bullen 6y ago
I recently added half floats to my 3D MMO engine, and I was very dissappointed when I discovered that very few GPUs support it in hardware! Desktop Nvidia simply converts them to 32-bit floats up front so the performance is kept but memory is wasted, but raspberry does the 16 -> 32 bit calculations every time resulting in horrible performance. I still have to test the engine on Jetson Nano with half floats, but I'm pretty sure I will be dissapointed again, and since raspberry doesn't support it I need to backtrack the code anyway! After some further research I heard snapdragon has 16-bit support in the hardware and hopefully this is where we are heading! 32-bit is completely overkill for model data, it wastes memory and cycles! 16-bit for model data: vertex, normal, texture and index and 8-bit for bone-index and weights! Back to the 80-90s! This is the last memory & performance increase we can grab now at "5"nm without adding too much complexity! You can try the engine without half float here: http://talk.binarytask.com/task?id=5959519327505901449 http://talk.binarytask.com/task?id=5959519327505901449
- kllrnohj 6y ago
  > Desktop Nvidia simply converts them to 32-bit floats up front so the performance is kept but memory is wasted Pascal has native FP16 operations and can execute 2 FP16's at once ( https://docs.nvidia.com/cuda/pascal-tuning-guide/index.html#fp16 https://docs.nvidia.com/cuda/pascal-tuning-guide/index.html#... ) BUT, and this is where things get fucked up, Nvidia then neutered that in the GeForce lineup because market segmentation. In fact, it's slower than FP32 operations: "GTX 1080’s FP16 instruction rate is 1/128th its FP32 instruction rate" https://www.anandtech.com/show/10325/the-nvidia-geforce-gtx-1080-and-1070-founders-edition-review/5 https://www.anandtech.com/show/10325/the-nvidia-geforce-gtx-...
  - AzN1337c0d3r 6y ago
    TU102 and GA102 are back up to double-rate FP16.
    - my123 6y ago
      GA102 is 1:1 FP16/FP32 rate, but with a ton of FP32 (30 Tflops for the RTX 3080).
- mechanical_berk 6y ago
  The Raspberry Pi 4 3D hardware has pretty extensive support for 16-bit floats. eg All 32-bit FP ops support "free" conversion from/to 16-bit float on input/output, and there are a few 16-bit float ops (eg multiply) which work on 16-bit vec2s (effectively giving double the throughput if they can be used). I don't know how well the Mesa driver supports any of that stuff though. Do you know if there was something specific that was slow?
  - bullen 6y ago
    Aha, interesting, where did you read about that? I just modified my vertex and normal arrays to half floats and the performance went down 80%!!! Maybe the vc4-dri.so driver hasen't implemented it properly yet one year later? But does it still save the RAM? So the texture uvs could be accelerated? But most of the data is in vec3... (vertex and normal)
    - mechanical_berk 6y ago
      While I do not believe Broadcom has released any documentation on the 3D hardware in the Raspberry Pi 4, you can figure out a fair amount by looking at the Mesa driver. In particular see the shader processor instruction packing code (https://github.com/mesa3d/mesa/blob/master/src/broadcom/qpu/qpu_instr.h https://github.com/mesa3d/mesa/blob/master/src/broadcom/qpu/...). For vertex arrays, while in theory the driver could load vec2/vec4 F16 attributes into the vertex shader as-is, AFAICT the Mesa driver does not attempt to do this; they get converted to F32 as they are loaded from main memory. There would be downsides to getting rid of this conversion... in particular the hardware does not support F32->F16 conversions when loading attributes, so the driver would have to recompile the shader if you switched the vertex arrays to F32 (or perhaps more likely recompile the shader when it realises you are using F16 vertex arrays rather than F32). In any case, the hardware does support zero overhead F16->F32 conversions when loading attributes, so you should not see a performance drop when switching vertex arrays to F16! The performance should go up slightly as less data needs to be loaded from main memory. If you're seeing an 80% performance drop something has gone terribly wrong!
- amelius 6y ago
  Why would you even use floats when dealing with geometric data? With floats, coordinates near the origin have more precision than other coordinates, and this usually doesn't make sense because the choice of the origin is just a convention. Better to use fixed-point numbers for geometric data, so all coordinates have the same precision.
  - astrange 6y ago
    This is true, but nobody likes working in fixed-point because it's inconvenient and languages don't have good support for it. (I think this is a language problem, not an argument for floats.) You can workaround this by doing most of your math in local coordinate spaces though, so it's fine for games.
  - sjburt 6y ago
    Geometry is only a part of the problem; much more compute goes into lighting and textures. Also scenes rarely span enough orders of magnitude of physical scale that the differences in precision matter (except may when using FP16). Finally, you probably can accept reduced precision for faraway objects (usually the camera is at the origin).
  - rasz 6y ago
    Didnt Quake 1 use fixed-point 256x256 vertex meshes for all the 3d models? with morphing instead of skeletal animations on top of that resulting in funky wobbly artefacts.
    - deleted 6y ago
      [deleted]
    - dividuum 6y ago
      IIRC the vertex coordinates were single precision floats (at least that's what my 24 year old pascal code uses :-) ) that were then additionally scaled and translated (so the center point was at the floor). What made it wobbly was the animation technique used. It had several animation sequences (walking, running, dying, etc..), each with multiple frame specifying the coordinates of a fixed number of 3D vertices. The indices of those vertices corresponded to the indices in a list of 2D vertices giving coordinated into a single 2D texture. IIRC Animation between frames used linear interpolation.

bullen 6y ago

I recently added half floats to my 3D MMO engine, and I was very dissappointed when I discovered that very few GPUs support it in hardware! Desktop Nvidia simply converts them to 32-bit floats up front so the performance is kept but memory is wasted, but raspberry does the 16 -> 32 bit calculations every time resulting in horrible performance. I still have to test the engine on Jetson Nano with half floats, but I'm pretty sure I will be dissapointed again, and since raspberry doesn't support it I need to backtrack the code anyway! After some further research I heard snapdragon has 16-bit support in the hardware and hopefully this is where we are heading! 32-bit is completely overkill for model data, it wastes memory and cycles! 16-bit for model data: vertex, normal, texture and index and 8-bit for bone-index and weights! Back to the 80-90s! This is the last memory & performance increase we can grab now at "5"nm without adding too much complexity! You can try the engine without half float here: http://talk.binarytask.com/task?id=5959519327505901449 http://talk.binarytask.com/task?id=5959519327505901449

kllrnohj 6y ago
> Desktop Nvidia simply converts them to 32-bit floats up front so the performance is kept but memory is wasted Pascal has native FP16 operations and can execute 2 FP16's at once ( https://docs.nvidia.com/cuda/pascal-tuning-guide/index.html#fp16 https://docs.nvidia.com/cuda/pascal-tuning-guide/index.html#... ) BUT, and this is where things get fucked up, Nvidia then neutered that in the GeForce lineup because market segmentation. In fact, it's slower than FP32 operations: "GTX 1080’s FP16 instruction rate is 1/128th its FP32 instruction rate" https://www.anandtech.com/show/10325/the-nvidia-geforce-gtx-1080-and-1070-founders-edition-review/5 https://www.anandtech.com/show/10325/the-nvidia-geforce-gtx-...
- AzN1337c0d3r 6y ago
  TU102 and GA102 are back up to double-rate FP16.
  - my123 6y ago
    GA102 is 1:1 FP16/FP32 rate, but with a ton of FP32 (30 Tflops for the RTX 3080).
mechanical_berk 6y ago
The Raspberry Pi 4 3D hardware has pretty extensive support for 16-bit floats. eg All 32-bit FP ops support "free" conversion from/to 16-bit float on input/output, and there are a few 16-bit float ops (eg multiply) which work on 16-bit vec2s (effectively giving double the throughput if they can be used). I don't know how well the Mesa driver supports any of that stuff though. Do you know if there was something specific that was slow?
- bullen 6y ago
  Aha, interesting, where did you read about that? I just modified my vertex and normal arrays to half floats and the performance went down 80%!!! Maybe the vc4-dri.so driver hasen't implemented it properly yet one year later? But does it still save the RAM? So the texture uvs could be accelerated? But most of the data is in vec3... (vertex and normal)
  - mechanical_berk 6y ago
    While I do not believe Broadcom has released any documentation on the 3D hardware in the Raspberry Pi 4, you can figure out a fair amount by looking at the Mesa driver. In particular see the shader processor instruction packing code (https://github.com/mesa3d/mesa/blob/master/src/broadcom/qpu/qpu_instr.h https://github.com/mesa3d/mesa/blob/master/src/broadcom/qpu/...). For vertex arrays, while in theory the driver could load vec2/vec4 F16 attributes into the vertex shader as-is, AFAICT the Mesa driver does not attempt to do this; they get converted to F32 as they are loaded from main memory. There would be downsides to getting rid of this conversion... in particular the hardware does not support F32->F16 conversions when loading attributes, so the driver would have to recompile the shader if you switched the vertex arrays to F32 (or perhaps more likely recompile the shader when it realises you are using F16 vertex arrays rather than F32). In any case, the hardware does support zero overhead F16->F32 conversions when loading attributes, so you should not see a performance drop when switching vertex arrays to F16! The performance should go up slightly as less data needs to be loaded from main memory. If you're seeing an 80% performance drop something has gone terribly wrong!
amelius 6y ago
Why would you even use floats when dealing with geometric data? With floats, coordinates near the origin have more precision than other coordinates, and this usually doesn't make sense because the choice of the origin is just a convention. Better to use fixed-point numbers for geometric data, so all coordinates have the same precision.
- astrange 6y ago
  This is true, but nobody likes working in fixed-point because it's inconvenient and languages don't have good support for it. (I think this is a language problem, not an argument for floats.) You can workaround this by doing most of your math in local coordinate spaces though, so it's fine for games.
- sjburt 6y ago
  Geometry is only a part of the problem; much more compute goes into lighting and textures. Also scenes rarely span enough orders of magnitude of physical scale that the differences in precision matter (except may when using FP16). Finally, you probably can accept reduced precision for faraway objects (usually the camera is at the origin).
- rasz 6y ago
  Didnt Quake 1 use fixed-point 256x256 vertex meshes for all the 3d models? with morphing instead of skeletal animations on top of that resulting in funky wobbly artefacts.
  - deleted 6y ago
    [deleted]
  - dividuum 6y ago
    IIRC the vertex coordinates were single precision floats (at least that's what my 24 year old pascal code uses :-) ) that were then additionally scaled and translated (so the center point was at the floor). What made it wobbly was the animation technique used. It had several animation sequences (walking, running, dying, etc..), each with multiple frame specifying the coordinates of a fixed number of 3D vertices. The indices of those vertices corresponded to the indices in a list of 2D vertices giving coordinated into a single 2D texture. IIRC Animation between frames used linear interpolation.