3 ms·

This sequence of posts is written very strangely and imprecisely? You'd almost never say "32MB of TLB". You'd say maybe, 8192 entries of TLB, each with 4kB pag

This sequence of posts is written very strangely and imprecisely? You'd almost never say "32MB of TLB". You'd say maybe, 8192 entries of TLB, each with 4kB pages, for instance. (Others here in Hacker News suggest that the M1 uses a 16kB page, so maybe 2048 entries of 16kB each?) The Twitter posts are talking about temperature at first, tile-memory. I admit that I'm ignorant to tile-memory but... The traditional answer to TLB bottlenecks is to enable large-pages or huge-page support. So I'm already feeling a bit weird that large pages / huge pages haven't been discussed in the twitter thread. (Maybe its not the answer on the GPU side of things? But at least acknowledging how we'd solve it in CPU-land would help show off how features like huge-pages could help) Like, none of the logic here makes sense to me at all. Maybe something weird is going on in the Apple / M1 world, but I have a suspicious feeling that maybe the person posting this Twitter thread is at best... misstating things in a very confusing manner and using imprecise language. At worst, they might be fundamentally incorrect on some of these discussion points?

eyelidlessness 5y ago
I don’t code for GPUs but it seemed pretty straightforward to me: 1. General purpose GPU code assumes optimization characteristics where memory constraints are less restrictive for this use case 2. Without optimizing for this platform, typical code will not exercise the full potential of the chip The thread is discussing it as a design flaw; I don’t know if it is. But if normal “recompile for ____ target” in XCode or whatever isn’t enough for common workloads to be relatively close to optimal, and you have to do and know to do special work for the memory architecture… yeah that’s probably not ideal. And the other takeaway from the thread, that they’re likely to throw more memory hardware at the problem in the future, sounds right to me.
- ngcc_hk 5y ago
  The key is apple code especially os and their tools. That is what m1 and future apple cpu and gpu are for. Hence optimisation should not be the issue. The whole thing is all about optimisation. The question is whether it affect other or 3rd party products (that cannot compete, or not shown to the public eg the private api ). And does apple care. Unless it shows is that limitation important ?
- ribit 5y ago
  The only problem: Apple GPUs are general purpose GPUs and other GPUs have bigger memory constraints than Apple G13. It is kind of funny that they complain about the small cache on M1 Ultra when 3090 Ti has 10x less cache
  - justsid 5y ago
    Not just that, but the claim that GPUs don’t mask memory latency by switching to other work is patently false as well. That’s exactly how GPUs do it. Memory bandwidth is a huge issue on GPUs and to maximize throughput, they absolutely do context switches to other work.
- dragontamer 5y ago
  > I don’t code for GPUs but it seemed pretty straightforward to me: I don't know all the words that are being used in this tweet. But the words I _DO_ understand are being used incorrectly, which is a major red-flag. 1. The temperature issues are non-sequitur, but that's where the discussion starts. There are a whole slew of power-configuration items on modern chips, none of them discussed in this set of twitter posts. 2. TLB is a CPU-issue, traditionally. I understand that Apple has an iGPU that shares the memory controller, but GPU-workloads are quite often linear and regular. It is difficult for me to imagine any step in the shader-pipeline (vertex shader, geometry shader, pixel / fragment shader) that would go through memory so randomly as to mess up the TLB. If you're saying the TLB is the problem, you need to come forth with a plausible use-case why your code is hopping around RAM so much to cause page-walks. 3. In fact, everything they say about GPUs in the thread is blatantly wrong at its face, or possibly some kind of misstatement about the peculiarity of the M1 Apple iGPU. 4. I realize that mobile GPU programmers are always talking about this "tile-based rendering" business (popular on Apple / Android phones), and I've never bothered to learn the details of it. But I can almost assure you, even in my ignorance, that this has nothing to do with TLBs at all. ----- The argument simply does not track with reality, or my understanding of GPUs in general. The vague advice I've heard for GPUs on mobile systems is to learn about the peculiarities of tile-based rendering, and to optimize for it specifically, because that's the main difference in GPU-architecture between Desktop GPUs and Phone GPUs. But nothing discussed in this set of twitter posts seems to match what I've often heard about iPhone/Android GPU programming. I've seen the experts advice on Android/Apple phone architecture / tile-based rendering. I don't know it, but... I think I can recognize it when I see them talking. This set of twitter posts is not an expert talking.
  - deleted 5y ago
    [deleted]
mlindner 5y ago
> This sequence of posts is written very strangely and imprecisely? Likely because the person doesn't have any background in understanding the subject. Twitter profile: > Jesus Christ is King! Co-host/writer for the Max Tech YouTube channel. They're a youtuber, that's the limit of their qualifications. And it's not like they're technical either. They review consumer electronics. They probably don't even know anything about CPU architecture.
- dstroot 5y ago
  Tweet storm coincided with an uploaded Video to YT at the same time about the “issue”. This is promoting their content.
- earthboundkid 5y ago
  Putting "Christ is King!" in your Twitter bio has the same problem as putting a Christian bumper sticker on your car: the person most likely to read it is the person you just flipped off.
marcan_42 5y ago
The AGX GPU MMU (variously called "GART" and "UAT", no relation to AGP GART) uses the ARM64 page table format (the GPU coprocessor literally uses it as an ARM page table among other things), so I'd expect it to support huge pages just fine since ARM64 does too. I don't know if macOS supports them, though. The page size is indeed 16K. I don't know if 4K is supported at all in the GPU. I agree though, the article reads like a rambly piece that doesn't follow and really just boils down to "my code runs slow on this machine and I'm going to blame the architecture" without going into proper details.
c7DJTLrn 5y ago
And >transaction lookaside buffer not >translation lookaside buffer I think hishnash made this discovery and this chain of tweets is echoing it.

eyelidlessness 5y ago

I don’t code for GPUs but it seemed pretty straightforward to me: 1. General purpose GPU code assumes optimization characteristics where memory constraints are less restrictive for this use case 2. Without optimizing for this platform, typical code will not exercise the full potential of the chip The thread is discussing it as a design flaw; I don’t know if it is. But if normal “recompile for ____ target” in XCode or whatever isn’t enough for common workloads to be relatively close to optimal, and you have to do and know to do special work for the memory architecture… yeah that’s probably not ideal. And the other takeaway from the thread, that they’re likely to throw more memory hardware at the problem in the future, sounds right to me.

ngcc_hk 5y ago
The key is apple code especially os and their tools. That is what m1 and future apple cpu and gpu are for. Hence optimisation should not be the issue. The whole thing is all about optimisation. The question is whether it affect other or 3rd party products (that cannot compete, or not shown to the public eg the private api ). And does apple care. Unless it shows is that limitation important ?
ribit 5y ago
The only problem: Apple GPUs are general purpose GPUs and other GPUs have bigger memory constraints than Apple G13. It is kind of funny that they complain about the small cache on M1 Ultra when 3090 Ti has 10x less cache
- justsid 5y ago
  Not just that, but the claim that GPUs don’t mask memory latency by switching to other work is patently false as well. That’s exactly how GPUs do it. Memory bandwidth is a huge issue on GPUs and to maximize throughput, they absolutely do context switches to other work.
dragontamer 5y ago
> I don’t code for GPUs but it seemed pretty straightforward to me: I don't know all the words that are being used in this tweet. But the words I _DO_ understand are being used incorrectly, which is a major red-flag. 1. The temperature issues are non-sequitur, but that's where the discussion starts. There are a whole slew of power-configuration items on modern chips, none of them discussed in this set of twitter posts. 2. TLB is a CPU-issue, traditionally. I understand that Apple has an iGPU that shares the memory controller, but GPU-workloads are quite often linear and regular. It is difficult for me to imagine any step in the shader-pipeline (vertex shader, geometry shader, pixel / fragment shader) that would go through memory so randomly as to mess up the TLB. If you're saying the TLB is the problem, you need to come forth with a plausible use-case why your code is hopping around RAM so much to cause page-walks. 3. In fact, everything they say about GPUs in the thread is blatantly wrong at its face, or possibly some kind of misstatement about the peculiarity of the M1 Apple iGPU. 4. I realize that mobile GPU programmers are always talking about this "tile-based rendering" business (popular on Apple / Android phones), and I've never bothered to learn the details of it. But I can almost assure you, even in my ignorance, that this has nothing to do with TLBs at all. ----- The argument simply does not track with reality, or my understanding of GPUs in general. The vague advice I've heard for GPUs on mobile systems is to learn about the peculiarities of tile-based rendering, and to optimize for it specifically, because that's the main difference in GPU-architecture between Desktop GPUs and Phone GPUs. But nothing discussed in this set of twitter posts seems to match what I've often heard about iPhone/Android GPU programming. I've seen the experts advice on Android/Apple phone architecture / tile-based rendering. I don't know it, but... I think I can recognize it when I see them talking. This set of twitter posts is not an expert talking.
- deleted 5y ago
  [deleted]

mlindner 5y ago

> This sequence of posts is written very strangely and imprecisely? Likely because the person doesn't have any background in understanding the subject. Twitter profile: > Jesus Christ is King! Co-host/writer for the Max Tech YouTube channel. They're a youtuber, that's the limit of their qualifications. And it's not like they're technical either. They review consumer electronics. They probably don't even know anything about CPU architecture.

dstroot 5y ago
Tweet storm coincided with an uploaded Video to YT at the same time about the “issue”. This is promoting their content.
earthboundkid 5y ago
Putting "Christ is King!" in your Twitter bio has the same problem as putting a Christian bumper sticker on your car: the person most likely to read it is the person you just flipped off.

marcan_42 5y ago

The AGX GPU MMU (variously called "GART" and "UAT", no relation to AGP GART) uses the ARM64 page table format (the GPU coprocessor literally uses it as an ARM page table among other things), so I'd expect it to support huge pages just fine since ARM64 does too. I don't know if macOS supports them, though. The page size is indeed 16K. I don't know if 4K is supported at all in the GPU. I agree though, the article reads like a rambly piece that doesn't follow and really just boils down to "my code runs slow on this machine and I'm going to blame the architecture" without going into proper details.

c7DJTLrn 5y ago

And >transaction lookaside buffer not >translation lookaside buffer I think hishnash made this discovery and this chain of tweets is echoing it.