5 ms·

That's how we felt when we were writing the software rasterizer for Larrabee! The issue is that that 1.5TFLOP is probably way more power than the M1 GPU's ~2.5T

That's how we felt when we were writing the software rasterizer for Larrabee! The issue is that that 1.5TFLOP is probably way more power than the M1 GPU's ~2.5TFLOP. The second issue is that a SW rasterizer is going to spend ~50% of its budget emulating fixed function. So, now you're way more power, for 1/4 the perf (best case). Also, you can't run any other apps, and you're probably going to have bandwidth issues to the display controller. GPUs are an optimization to try to use the excess Moore's law we have to get to the ghost of Dennard's law.

bob1029 4y ago
I think a power/latency/perf tradeoff could be agreeable for certain applications. GPUs in the cloud are not exactly cheap. Many gaming experiences do not require nanite-level graphics. Building something that can reliably output reasonable-quality 3d graphics without relying on specific GPU technologies will give you a much broader realm to operate with. I believe something along this path is the solution for streaming gaming. I perceive the failure of Stadia, et. al. as being a consequence of trying to bolt streaming onto existing GPU-based, local gaming solutions. Build something from scratch with streaming/latency as a #1 priority, and you can dramatically expand the operational radius of each datacenter (e.g. ~100km per millisecond saved).
- jvanderbot 4y ago
  Light is 300km/ms in a vacuum. Is it that much slower through switched fiber?
  - rkangel 4y ago
    Speed of light in glass is about 2/3 of the speed of light in a vacuum (refractive index of glass is around 1.5).
  - Someone 4y ago
    Signal speed in fiber is about ⅔ of that in vacuum (https://en.wikipedia.org/wiki/Optical_fiber#Refractive_index https://en.wikipedia.org/wiki/Optical_fiber#Refractive_index), but fiber won’t be straight-line between sender and receiver, light doesn’t move in a straight line inside the fiber, and the switched adds delays. https://www.pingdom.com/blog/theoretical-vs-real-world-speed-limit-of-ping/ https://www.pingdom.com/blog/theoretical-vs-real-world-speed...: “you should probably double the “ideal” response times shown above for a more realistic target to aim at“ So yes, ⅓ of light speed in vacuum seems a decent heuristic.
  - MobiusHorizons 4y ago
    Round trip latency matters here, which would get you down to 150km without any slowdown through fiber.
- thechao 4y ago
  Well; except for the fact that the 1.5TFLOP quoted in the article is because of the AMX part. The actually useful throughput of the big-core is probably more like 35GFLOP peak. This compares to the 1–2TFLOP throughput of the GPU. The CPU is easily going to be 50–100x slower than the GPU. If you're talking full-screen Angry Birds with, say, a 2x average compositing, you're going to be fine on the CPU; but, energy- and jitter- wise you'll still be happier with the GPU, overall.
- dotnet00 4y ago
  I feel like that's a somewhat out-of-touch interpretation, as Stadia failed largely because of Google's terrible reputation and the (completely valid) concerns from gamers about companies intending to turn even single player games into fragmented streaming platforms where the content is entirely dependent on the whims of the company (a fitting example being Google doing its thing and killing Stadia). They had no shortage of GPUs. NVIDIA's streaming service is doing relatively fine in comparison. They simply share a GPU between several users for anything that isn't demanding enough. They also get around some of the concerns about gaming being turned into another streaming style fragmented mess by not actually selling the games. You simply log into your account on Steam/GOG/whatever and play the games you already own as you might on a local PC. Additionally, "building something that can reliably output reasonable-quality 3d graphics without relying on specific GPU technologies" doesn't make much sense to me. If it's an accelerator designed to handle relatively modern 3d graphics, due to the programmability of a modern graphics pipeline it's effectively just a GPU. There aren't any underlying technologies that are required to be used as long as they can produce a similar output (mobile GPUs tend to have a different approach to how they implement the graphics pipeline compared to desktop GPUs for instance).
- kllrnohj 4y ago
  well 2 things: 1) 1.5 TFLOPS is already less than the GPUs in most current phones. Like you're not exactly talking a big number here. You're talking 14 year old graphics (when desktop GPUs crossed the 1TFLOP mark) 2) And, this is the bigger issue, this AMX unit only does matrix multiplies. I'd be fascinated to see someone create a 3d renderer with only matrix multiplication. I'm sure it's possible but this 1.5 tflops ain't remotely comparable to a GPU's 1.5 tflops. Like if you tried to do "traditional" pixel shader rendering code on this unit you'd instantly be cutting that 1.5 tflops to 1/16th the performance - you're now at a theoretical peak of a mere 93 gflops of GPU-comparable performance.
  - brrrrrm 4y ago
    well, AMX does asynchronous outer products from byte-addressable buffers with accumulation. it's not like tensor cores or intel's AMX where the silicon literally only does a matrix multiplication. you can easily do efficient sliding window math from the buffers (convolutions) or sparse data loads (since the instructions run asynchronously from the CPU)
hajile 4y ago
There’s nothing to preclude a CPU from adding fixed function units they can call for certain things. Larabee 2 (one of the knights series) actually included these on the chip (though they were disabled in the final compute only product). You also forgot to mention the usability bit. Branching is bad for GPUs. They either take both branches or in the optimized version, they stall the pipeline until they get a result then attempt to optimize which units need to execute on which branch. The first is simply inefficient. The last is efficient, but impracticality slow on branchy code. On the power side, bloated x86 is hardly a good showcase for what such an architecture could do. I also suspect that an automatic thread manager like most modern GPUs use could also add a lot of value.
- kllrnohj 4y ago
  > Branching is bad for GPUs. Branching is bad for wide SIMD workloads of which GPUs are the most common. But just shoving that SIMD under the control of a CPU instead won't fix your branching issues. You're still probably going to choose the path of just executing both branches & masking off the results.
- thechao 4y ago
  I worked on the LRB through LRB3. I wrote the non-polygon pipeline for omatic (lines, points, billboards); later I switched to the shader compiler (masher) under mattomatic. Larrabee 1 (all of omatic) had fixed function on the turns: texture units, and pointer chasing. The later parts did not have GPU FF; they didn’t even have some of the rasterization opcodes (faddsets & fmad233). I seem to remember that LRB could test & jmp in 4 cycles, if you were careful. Since it was 4x barrel processed, those jmps were “free”. I moved to integrated GPUs, later. x86 is bloated, but LRB was not. Also, the decoder — even a big one like x86 — isn’t a major HW problem. I’d say x86’s memory hierarchy is more of an issue.

bob1029 4y ago

I think a power/latency/perf tradeoff could be agreeable for certain applications. GPUs in the cloud are not exactly cheap. Many gaming experiences do not require nanite-level graphics. Building something that can reliably output reasonable-quality 3d graphics without relying on specific GPU technologies will give you a much broader realm to operate with. I believe something along this path is the solution for streaming gaming. I perceive the failure of Stadia, et. al. as being a consequence of trying to bolt streaming onto existing GPU-based, local gaming solutions. Build something from scratch with streaming/latency as a #1 priority, and you can dramatically expand the operational radius of each datacenter (e.g. ~100km per millisecond saved).

jvanderbot 4y ago
Light is 300km/ms in a vacuum. Is it that much slower through switched fiber?
- rkangel 4y ago
  Speed of light in glass is about 2/3 of the speed of light in a vacuum (refractive index of glass is around 1.5).
- Someone 4y ago
  Signal speed in fiber is about ⅔ of that in vacuum (https://en.wikipedia.org/wiki/Optical_fiber#Refractive_index https://en.wikipedia.org/wiki/Optical_fiber#Refractive_index), but fiber won’t be straight-line between sender and receiver, light doesn’t move in a straight line inside the fiber, and the switched adds delays. https://www.pingdom.com/blog/theoretical-vs-real-world-speed-limit-of-ping/ https://www.pingdom.com/blog/theoretical-vs-real-world-speed...: “you should probably double the “ideal” response times shown above for a more realistic target to aim at“ So yes, ⅓ of light speed in vacuum seems a decent heuristic.
- MobiusHorizons 4y ago
  Round trip latency matters here, which would get you down to 150km without any slowdown through fiber.
thechao 4y ago
Well; except for the fact that the 1.5TFLOP quoted in the article is because of the AMX part. The actually useful throughput of the big-core is probably more like 35GFLOP peak. This compares to the 1–2TFLOP throughput of the GPU. The CPU is easily going to be 50–100x slower than the GPU. If you're talking full-screen Angry Birds with, say, a 2x average compositing, you're going to be fine on the CPU; but, energy- and jitter- wise you'll still be happier with the GPU, overall.
dotnet00 4y ago
I feel like that's a somewhat out-of-touch interpretation, as Stadia failed largely because of Google's terrible reputation and the (completely valid) concerns from gamers about companies intending to turn even single player games into fragmented streaming platforms where the content is entirely dependent on the whims of the company (a fitting example being Google doing its thing and killing Stadia). They had no shortage of GPUs. NVIDIA's streaming service is doing relatively fine in comparison. They simply share a GPU between several users for anything that isn't demanding enough. They also get around some of the concerns about gaming being turned into another streaming style fragmented mess by not actually selling the games. You simply log into your account on Steam/GOG/whatever and play the games you already own as you might on a local PC. Additionally, "building something that can reliably output reasonable-quality 3d graphics without relying on specific GPU technologies" doesn't make much sense to me. If it's an accelerator designed to handle relatively modern 3d graphics, due to the programmability of a modern graphics pipeline it's effectively just a GPU. There aren't any underlying technologies that are required to be used as long as they can produce a similar output (mobile GPUs tend to have a different approach to how they implement the graphics pipeline compared to desktop GPUs for instance).
kllrnohj 4y ago
well 2 things: 1) 1.5 TFLOPS is already less than the GPUs in most current phones. Like you're not exactly talking a big number here. You're talking 14 year old graphics (when desktop GPUs crossed the 1TFLOP mark) 2) And, this is the bigger issue, this AMX unit only does matrix multiplies. I'd be fascinated to see someone create a 3d renderer with only matrix multiplication. I'm sure it's possible but this 1.5 tflops ain't remotely comparable to a GPU's 1.5 tflops. Like if you tried to do "traditional" pixel shader rendering code on this unit you'd instantly be cutting that 1.5 tflops to 1/16th the performance - you're now at a theoretical peak of a mere 93 gflops of GPU-comparable performance.
- brrrrrm 4y ago
  well, AMX does asynchronous outer products from byte-addressable buffers with accumulation. it's not like tensor cores or intel's AMX where the silicon literally only does a matrix multiplication. you can easily do efficient sliding window math from the buffers (convolutions) or sparse data loads (since the instructions run asynchronously from the CPU)

hajile 4y ago

There’s nothing to preclude a CPU from adding fixed function units they can call for certain things. Larabee 2 (one of the knights series) actually included these on the chip (though they were disabled in the final compute only product). You also forgot to mention the usability bit. Branching is bad for GPUs. They either take both branches or in the optimized version, they stall the pipeline until they get a result then attempt to optimize which units need to execute on which branch. The first is simply inefficient. The last is efficient, but impracticality slow on branchy code. On the power side, bloated x86 is hardly a good showcase for what such an architecture could do. I also suspect that an automatic thread manager like most modern GPUs use could also add a lot of value.

kllrnohj 4y ago
> Branching is bad for GPUs. Branching is bad for wide SIMD workloads of which GPUs are the most common. But just shoving that SIMD under the control of a CPU instead won't fix your branching issues. You're still probably going to choose the path of just executing both branches & masking off the results.
thechao 4y ago
I worked on the LRB through LRB3. I wrote the non-polygon pipeline for omatic (lines, points, billboards); later I switched to the shader compiler (masher) under mattomatic. Larrabee 1 (all of omatic) had fixed function on the turns: texture units, and pointer chasing. The later parts did not have GPU FF; they didn’t even have some of the rasterization opcodes (faddsets & fmad233). I seem to remember that LRB could test & jmp in 4 cycles, if you were careful. Since it was 4x barrel processed, those jmps were “free”. I moved to integrated GPUs, later. x86 is bloated, but LRB was not. Also, the decoder — even a big one like x86 — isn’t a major HW problem. I’d say x86’s memory hierarchy is more of an issue.