9 ms·

Apple M1 Microarchitecture Research

237 points by my123 6y ago

deleted 6y ago
[deleted]
mhh__ 6y ago
Source for what a validation buffer is? Couldn't find a reference on google
- algorithm314 6y ago
  Probably this https://www.researchgate.net/publication/250780982_The_Validation_Buffer_Out-of-Order_Retirement_Microarchitecture https://www.researchgate.net/publication/250780982_The_Valid...
- sesuximo 6y ago
  They say the M1 may use a validation buffer instead of a re order buffer. This makes it somewhat clear if you know what a re order buffer is. Processors basically do this: read instructions, execute them out of order to fully use cpu resources, put the instructions back in order along with their results, and then write results to memory. This way the “final state” follows the order of instructions. The “put back in order” step is handled by the re order buffer. It is basically an online sort algorithm. Fully sorting instructions after execution is simple and correct, but in extreme cases may be suboptimal; perhaps it doesn’t matter if the final state is exactly in order or just close to that. I believe the article is saying the M1 may just validate states as they are written rather than fully reordering.
  - mhh__ 6y ago
    It was the distinction between validation and re-order that confused me.
  - cromwellian 6y ago
    The Sun Microsystems Rock processor had I guess, validation? Rather than re-order instructions, you just speculatively execute instructions, issue you memory ops but don't stop and wait, called "hardware scouting". If after everything resolves, you find your prediction was wrong, you rollback. So it's a kind of Transactional Memory approach, and if validation fails, you re-execute the same sequence of instructions, but the branches are different. https://en.wikipedia.org/wiki/Hardware_scout https://en.wikipedia.org/wiki/Hardware_scout
    - eru 6y ago
      Interesting. Sounds like even more opportunities for speculative instruction exploits.
  - StillBored 6y ago
    Arm isn't TSO. As long as the internal instruction stream maintains dependency information independent pieces of code could be completed and written back to memory at any time. Basically the whole, "put it back in order" bits only have to happen around memory barriers because the visibility of writes isn't guaranteed otherwise. Whether this matters given large write combining/writeback buffering that queues up early writes until the later writes have completed is one of those decade+ long arguments.
- mzs 6y ago
  There has been an update: >… The M1 seems to use something other than an entirely conventional reorder buffer, which complicates measurements a bit. So these may or may not be accurate. (This paragraph previously said "it seems to use something along the lines of a validation buffer". I think the VB hypothesis has since been disproven. Various attempts to measure ROB size have yielded values 623, 853, and 2295 (see the previous link). My uninformed hypothesis is that this may imply a kind of distributed reorder buffer, where only structures that need to know about a given operation track them, and/or some kind of out-of-order retirement.) https://github.com/dougallj/applecpu/commit/dc3c220f58f428b594887e54cf135be3bda8da23#diff-9f4857541b17e56daec5b931235513782565fbde0f869d5c7aea4a8b8de292cc https://github.com/dougallj/applecpu/commit/dc3c220f58f428b5...
black_puppydog 6y ago
Hats off to people who do this kind of work. However... *clears throat* I have real trouble using the term "research" for something like this, lumping it together with activities such as "research" in natural science, "research" in anthropology, or even journalistic "research". The difference to me is that this activity could be "done", or rather entirely avoided, if Apple didn't decide to keep it a secret. Mind you, I'm not saying Apple doesn't have good reasons to keep them secret, for some definition of "good". I just think that this person just spent hours upon hours of their life working around a quirk in our current society. I'm aware I'm splitting hairs, but "we don't know what the architecture is" to me is fundamentally different to "we don't know what dark energy is." The former is something that could "easily" be changed, the latter is not. Lumping them together just cements the idea that the systems we built and navigate, for better or worse, are to be treated like natural laws that can't be argued with. /rant
- njitbew 6y ago
  The distinction you're making is more about _fundamental_ research vs. _applied_ research, but it is research nevertheless. From the first hit I found on Google: * Fundamental researches mainly aim to answer the questions of why, what or how and they tend to contribute the pool of fundamental knowledge in the research area. * Opposite to fundamental research is applied research that aims to solve specific problems, thus findings of applied research do have immediate practical implications.
- mindvirus 6y ago
  Your comment reminds me of a scifi story (Greg Egan maybe?) where AI has taken the lead on science, and humanity's scientists' job is now just to understand the AI's discoveries.
  - atq2119 6y ago
    There's a fictitious journal article with this premise by Ted Chiang.
- monocasa 6y ago
  Just because other humans created the knowledge in question, doesn't make the search for that knowledge not research, if the knowledge isn't held by the general public in the first place, and if that knowledge requires independent gathering of data and interpretation to acquire. I agree that Apple and the quirks in our society that encourage that secretive behavior has the effect of researchers independently replicating a small subset of Apple's work in a way that they'd be redirected to something more useful in a more open society. That doesn't make the independent researcher's work not research though. Otherwise nearly the whole field of anthropology wouldn't exist or wouldn't be considered research. > Lumping them together just cements the idea that the systems we built and navigate, for better or worse, are to be treated like natural laws that can't be argued with. The systems we have built are pretty fixed in time. We can argue about how to change them going in the future, but we should also record, externally if need be like here, what has happened so that we can make better decisions going forward. Apple's choices for the M1 are arguably more set in stone than our conception of physics. We're able to retcon all of physics when we make changes to the standard model, but "what tradeoffs did Apple make in the M1" are pretty much done and set in stone (or silicon I guess).
kergonath 6y ago
That’s an impressive piece of work. I would love to know how far this is from the information in the actual internal documentation. It’s very nice to have some excitement about a mainstream CPU architecture again.
- wooger 6y ago
  Just a pain that due to Apple's policies on keeping a big margin and lack of sharing technology means that the rest of the industry won't benefit from this. Still very hard to see not getting an M1 as my next laptop, especially if they continue to do sane things line remove the shitty touchbar.
  - walterbell 6y ago
    Nuvia (former Apple M1 team members) was acquired by Qualcomm, so hopefully new laptop SoCs are on the way.
    - kergonath 6y ago
      Yeah but Qualcomm... They’re perfectly happy sitting in their arses if there’s nobody to kick them around, and they’re just a nuisance generally. I hope I’m wrong, but they won’t have direct competition from Apple because it looks unlikely that M1 devices will run Windows. They seem to be more or less happy with being 2 generations behind Apple on phone SoCs. And they basically gave up on wearables. So instead of having Wintel/Mac we’ll have Wincomm/Mac or Wintel/Wincomm/Mac; not sure it’ll be much better.
      - EricE 6y ago
        Huh? I ran Windows on my M1 MBA quite easily: https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewARM64 https://www.microsoft.com/en-us/software-download/windowsins... It was under Parallels but I’m sure over time someone will figure out how to direct boot into it if that’s really important. Performance of the M1 running Windows and games either in Crossover (commercial WINE implementation - well worth the $40 for ease of use/installation) or the Windows 10 beta actually turned out to be a bad thing. I subscribe to way too many assets in Cities:Skylines to use 16GB of RAM - I need at least 32GB so with great difficulty I returned the 16GB MBA and am eagerly awaiting the next round of AS Macs, hoping there will be at least one laptop that can take at lease 32GB of RAM. If so I will probably never update my Windows gaming machine again. Not that I could if I wanted to right now with the insane GPU shortages - but that’s another topic :p
glangdale 6y ago
It's interesting work. The results show a marked contrast with equivalent information for Intel (think uops.info, Agner Fog's work or even the Optimization Reference Manual). Most obvious is separating general purpose register stuff from SIMD. If you have a mixed workload, this means M1 is much wider in that sense (although note that the SIMD registers involved are 128b as opposed to up to 512b). Intel's execution ports tend to be special flowers - there's almost always stuff that can be done and can't be done on any given port. So, of the execution ports (0,1,5,6) each one tends to have some special flavor that the others don't share - and there are often complex "oh, you can only do 2 of those or 3 of these". There's a very small list of operations (generally logic and LEA) that can be done on all of 0,1,5,6. By contrast, most of the M1 things, you can do across every unit that is capable of doing it. Want to do 4 SIMD math ops? Go for your life! It's certainly easier to remember and understand. I imagine the counterargument on the Intel side is that it's unlikely you need that many of some given operation - so they are saving area to not make every port omni-capable. This works pretty well (right up to the point it doesn't). It will be interesting to see how the comparison between Intel and Apple plays out in the near future. M1 is a tour de force of balanced design, but I wouldn't count Intel out - especially if they can get their process mojo back (a lot of M1's current advantage is being able to invest transistors in a balanced and sensible way across the entire pipeline due to a better process).
- hajile 6y ago
  Intel's more likely argument is that they can't do that many because their decoders and ROB aren't wide enough to keep up. Given how huge their decode units are (about as big as all their integer ALUs), I don't know how x86 will actually solve this issue.
  - btown 6y ago
    Could Intel adapt a Rosetta 2-style approach? Preprocess binaries to replace ancient instructions with modern equivalents, and be able to use smaller decode units? Would the gains be worth the translation requirements?
    - 6y ago
lincpa 6y ago
My computer hardware architecture design was published on February 06, 2019. One or two years later, the Apple M1 chip adopted the "warehouse/workshop model" design and was released on November 11, 2020. Warehouse: unified memory Workshop: CPU, GPU and other cores Products (raw materials): information, data there's also a new unified memory architecture that lets the CPU, GPU, and other cores exchange information between one another, and with unified memory, the CPU and GPU can access memory simultaneously rather than copying data between one area and another. Accessing the same pool of memory without the need for copying speeds up information exchange for faster overall performance. reference: Developer Delves Into Reasons Why Apple's M1 Chip is So Fast From the introduction Apple M1 has not done global optimization of various core (workshop) scheduling. Apple M1 only optimizes the access to memory data (materials and products in the warehouse). Apple needs to further improve the programming language and compiler to support and promote my programming methodology. My architecture supports a wider range of workshop types than Apple M1, with greater efficiency, scalability and flexibility. Conclusion Apple M1 chip still needs a lot of optimization work, now its optimization level is still very simple, after all, it is only the first generation of works, released in stages. Forecast(2021-01-19): I think Intel, AMD, ARM, supercomputer, etc. will adopt the "warehouse/workshop model" In the past, the performance of the CPU played a decisive role in the performance of the computer. There were few CPU cores and the number and types of peripherals. Therefore, the CPU became the center of the computer hardware architecture. Now, with more and more CPU and GPU cores, and the number and types of peripherals, the communication, coordination, and management of cores (or components, peripherals) have become more and more important, They become a key factor in computer performance. The core views of management science and computer science are the same: Use all available resources to complete the goal with the highest efficiency. It is the best field of management science to accomplish production goals through communication, coordination, and management of various available resources. The most effective, reliable, and absolutely mainstream way is the "warehouse/workshop model". Only changing the architecture, not changing or only expanding the CPU instruction set, not only will not affect the CPU compatibility, but also bring huge optimization space. So I think Intel, AMD, ARM, supercomputing, etc. will adopt the "warehouse/workshop model", which is an inevitable trend in the development of computer hardware. Finally, "Warehouse/Workshop Model" and "Von Neumann Architecture" will become the two major architectures in the IT field. https://github.com/linpengcheng/PurefunctionPipelineDataflow#The-unification-with-Computer-Hardware-Architecture https://github.com/linpengcheng/PurefunctionPipelineDataflow...

237 points by my123 6y ago

deleted 6y ago

[deleted]

mhh__ 6y ago

Source for what a validation buffer is? Couldn't find a reference on google

algorithm314 6y ago
Probably this https://www.researchgate.net/publication/250780982_The_Validation_Buffer_Out-of-Order_Retirement_Microarchitecture https://www.researchgate.net/publication/250780982_The_Valid...
sesuximo 6y ago
They say the M1 may use a validation buffer instead of a re order buffer. This makes it somewhat clear if you know what a re order buffer is. Processors basically do this: read instructions, execute them out of order to fully use cpu resources, put the instructions back in order along with their results, and then write results to memory. This way the “final state” follows the order of instructions. The “put back in order” step is handled by the re order buffer. It is basically an online sort algorithm. Fully sorting instructions after execution is simple and correct, but in extreme cases may be suboptimal; perhaps it doesn’t matter if the final state is exactly in order or just close to that. I believe the article is saying the M1 may just validate states as they are written rather than fully reordering.
- mhh__ 6y ago
  It was the distinction between validation and re-order that confused me.
- cromwellian 6y ago
  The Sun Microsystems Rock processor had I guess, validation? Rather than re-order instructions, you just speculatively execute instructions, issue you memory ops but don't stop and wait, called "hardware scouting". If after everything resolves, you find your prediction was wrong, you rollback. So it's a kind of Transactional Memory approach, and if validation fails, you re-execute the same sequence of instructions, but the branches are different. https://en.wikipedia.org/wiki/Hardware_scout https://en.wikipedia.org/wiki/Hardware_scout
  - eru 6y ago
    Interesting. Sounds like even more opportunities for speculative instruction exploits.
- StillBored 6y ago
  Arm isn't TSO. As long as the internal instruction stream maintains dependency information independent pieces of code could be completed and written back to memory at any time. Basically the whole, "put it back in order" bits only have to happen around memory barriers because the visibility of writes isn't guaranteed otherwise. Whether this matters given large write combining/writeback buffering that queues up early writes until the later writes have completed is one of those decade+ long arguments.
mzs 6y ago
There has been an update: >… The M1 seems to use something other than an entirely conventional reorder buffer, which complicates measurements a bit. So these may or may not be accurate. (This paragraph previously said "it seems to use something along the lines of a validation buffer". I think the VB hypothesis has since been disproven. Various attempts to measure ROB size have yielded values 623, 853, and 2295 (see the previous link). My uninformed hypothesis is that this may imply a kind of distributed reorder buffer, where only structures that need to know about a given operation track them, and/or some kind of out-of-order retirement.) https://github.com/dougallj/applecpu/commit/dc3c220f58f428b594887e54cf135be3bda8da23#diff-9f4857541b17e56daec5b931235513782565fbde0f869d5c7aea4a8b8de292cc https://github.com/dougallj/applecpu/commit/dc3c220f58f428b5...

black_puppydog 6y ago

Hats off to people who do this kind of work. However... *clears throat* I have real trouble using the term "research" for something like this, lumping it together with activities such as "research" in natural science, "research" in anthropology, or even journalistic "research". The difference to me is that this activity could be "done", or rather entirely avoided, if Apple didn't decide to keep it a secret. Mind you, I'm not saying Apple doesn't have good reasons to keep them secret, for some definition of "good". I just think that this person just spent hours upon hours of their life working around a quirk in our current society. I'm aware I'm splitting hairs, but "we don't know what the architecture is" to me is fundamentally different to "we don't know what dark energy is." The former is something that could "easily" be changed, the latter is not. Lumping them together just cements the idea that the systems we built and navigate, for better or worse, are to be treated like natural laws that can't be argued with. /rant

njitbew 6y ago
The distinction you're making is more about _fundamental_ research vs. _applied_ research, but it is research nevertheless. From the first hit I found on Google: * Fundamental researches mainly aim to answer the questions of why, what or how and they tend to contribute the pool of fundamental knowledge in the research area. * Opposite to fundamental research is applied research that aims to solve specific problems, thus findings of applied research do have immediate practical implications.
mindvirus 6y ago
Your comment reminds me of a scifi story (Greg Egan maybe?) where AI has taken the lead on science, and humanity's scientists' job is now just to understand the AI's discoveries.
- atq2119 6y ago
  There's a fictitious journal article with this premise by Ted Chiang.
monocasa 6y ago
Just because other humans created the knowledge in question, doesn't make the search for that knowledge not research, if the knowledge isn't held by the general public in the first place, and if that knowledge requires independent gathering of data and interpretation to acquire. I agree that Apple and the quirks in our society that encourage that secretive behavior has the effect of researchers independently replicating a small subset of Apple's work in a way that they'd be redirected to something more useful in a more open society. That doesn't make the independent researcher's work not research though. Otherwise nearly the whole field of anthropology wouldn't exist or wouldn't be considered research. > Lumping them together just cements the idea that the systems we built and navigate, for better or worse, are to be treated like natural laws that can't be argued with. The systems we have built are pretty fixed in time. We can argue about how to change them going in the future, but we should also record, externally if need be like here, what has happened so that we can make better decisions going forward. Apple's choices for the M1 are arguably more set in stone than our conception of physics. We're able to retcon all of physics when we make changes to the standard model, but "what tradeoffs did Apple make in the M1" are pretty much done and set in stone (or silicon I guess).

kergonath 6y ago

That’s an impressive piece of work. I would love to know how far this is from the information in the actual internal documentation. It’s very nice to have some excitement about a mainstream CPU architecture again.

wooger 6y ago
Just a pain that due to Apple's policies on keeping a big margin and lack of sharing technology means that the rest of the industry won't benefit from this. Still very hard to see not getting an M1 as my next laptop, especially if they continue to do sane things line remove the shitty touchbar.
- walterbell 6y ago
  Nuvia (former Apple M1 team members) was acquired by Qualcomm, so hopefully new laptop SoCs are on the way.
  - kergonath 6y ago
    Yeah but Qualcomm... They’re perfectly happy sitting in their arses if there’s nobody to kick them around, and they’re just a nuisance generally. I hope I’m wrong, but they won’t have direct competition from Apple because it looks unlikely that M1 devices will run Windows. They seem to be more or less happy with being 2 generations behind Apple on phone SoCs. And they basically gave up on wearables. So instead of having Wintel/Mac we’ll have Wincomm/Mac or Wintel/Wincomm/Mac; not sure it’ll be much better.
    - EricE 6y ago
      Huh? I ran Windows on my M1 MBA quite easily: https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewARM64 https://www.microsoft.com/en-us/software-download/windowsins... It was under Parallels but I’m sure over time someone will figure out how to direct boot into it if that’s really important. Performance of the M1 running Windows and games either in Crossover (commercial WINE implementation - well worth the $40 for ease of use/installation) or the Windows 10 beta actually turned out to be a bad thing. I subscribe to way too many assets in Cities:Skylines to use 16GB of RAM - I need at least 32GB so with great difficulty I returned the 16GB MBA and am eagerly awaiting the next round of AS Macs, hoping there will be at least one laptop that can take at lease 32GB of RAM. If so I will probably never update my Windows gaming machine again. Not that I could if I wanted to right now with the insane GPU shortages - but that’s another topic :p

glangdale 6y ago

It's interesting work. The results show a marked contrast with equivalent information for Intel (think uops.info, Agner Fog's work or even the Optimization Reference Manual). Most obvious is separating general purpose register stuff from SIMD. If you have a mixed workload, this means M1 is much wider in that sense (although note that the SIMD registers involved are 128b as opposed to up to 512b). Intel's execution ports tend to be special flowers - there's almost always stuff that can be done and can't be done on any given port. So, of the execution ports (0,1,5,6) each one tends to have some special flavor that the others don't share - and there are often complex "oh, you can only do 2 of those or 3 of these". There's a very small list of operations (generally logic and LEA) that can be done on all of 0,1,5,6. By contrast, most of the M1 things, you can do across every unit that is capable of doing it. Want to do 4 SIMD math ops? Go for your life! It's certainly easier to remember and understand. I imagine the counterargument on the Intel side is that it's unlikely you need that many of some given operation - so they are saving area to not make every port omni-capable. This works pretty well (right up to the point it doesn't). It will be interesting to see how the comparison between Intel and Apple plays out in the near future. M1 is a tour de force of balanced design, but I wouldn't count Intel out - especially if they can get their process mojo back (a lot of M1's current advantage is being able to invest transistors in a balanced and sensible way across the entire pipeline due to a better process).

hajile 6y ago
Intel's more likely argument is that they can't do that many because their decoders and ROB aren't wide enough to keep up. Given how huge their decode units are (about as big as all their integer ALUs), I don't know how x86 will actually solve this issue.
- btown 6y ago
  Could Intel adapt a Rosetta 2-style approach? Preprocess binaries to replace ancient instructions with modern equivalents, and be able to use smaller decode units? Would the gains be worth the translation requirements?
  - 6y ago

lincpa 6y ago

My computer hardware architecture design was published on February 06, 2019. One or two years later, the Apple M1 chip adopted the "warehouse/workshop model" design and was released on November 11, 2020. Warehouse: unified memory Workshop: CPU, GPU and other cores Products (raw materials): information, data there's also a new unified memory architecture that lets the CPU, GPU, and other cores exchange information between one another, and with unified memory, the CPU and GPU can access memory simultaneously rather than copying data between one area and another. Accessing the same pool of memory without the need for copying speeds up information exchange for faster overall performance. reference: Developer Delves Into Reasons Why Apple's M1 Chip is So Fast From the introduction Apple M1 has not done global optimization of various core (workshop) scheduling. Apple M1 only optimizes the access to memory data (materials and products in the warehouse). Apple needs to further improve the programming language and compiler to support and promote my programming methodology. My architecture supports a wider range of workshop types than Apple M1, with greater efficiency, scalability and flexibility. Conclusion Apple M1 chip still needs a lot of optimization work, now its optimization level is still very simple, after all, it is only the first generation of works, released in stages. Forecast(2021-01-19): I think Intel, AMD, ARM, supercomputer, etc. will adopt the "warehouse/workshop model" In the past, the performance of the CPU played a decisive role in the performance of the computer. There were few CPU cores and the number and types of peripherals. Therefore, the CPU became the center of the computer hardware architecture. Now, with more and more CPU and GPU cores, and the number and types of peripherals, the communication, coordination, and management of cores (or components, peripherals) have become more and more important, They become a key factor in computer performance. The core views of management science and computer science are the same: Use all available resources to complete the goal with the highest efficiency. It is the best field of management science to accomplish production goals through communication, coordination, and management of various available resources. The most effective, reliable, and absolutely mainstream way is the "warehouse/workshop model". Only changing the architecture, not changing or only expanding the CPU instruction set, not only will not affect the CPU compatibility, but also bring huge optimization space. So I think Intel, AMD, ARM, supercomputing, etc. will adopt the "warehouse/workshop model", which is an inevitable trend in the development of computer hardware. Finally, "Warehouse/Workshop Model" and "Von Neumann Architecture" will become the two major architectures in the IT field. https://github.com/linpengcheng/PurefunctionPipelineDataflow#The-unification-with-Computer-Hardware-Architecture https://github.com/linpengcheng/PurefunctionPipelineDataflow...