22 ms·

Memory access on the Apple M1 processor

372 points by luigi23 6y ago

jeffbee 6y ago
Great practical information. Nice to see people who know what they are talking about putting data out there. I hope eventually these persistent HN memes about M1 memory will die: that it's "on-die" (it's not), that it's the only CPU using LPDDR4X-4267 (it's not), or that it's faster because the memory is 2mm closer to the CPU (not that either). It's faster because it has more microarchitectural resources. It can load and store more, and it can do with a single core what an Intel part needs all cores to accomplish.
- pdpi 6y ago
  This seems to be a recurring theme with the M1, and one that, in a sense, actually baffles me even more than the alternative. There is no "magic" at play here, it's just lots and lots of raw muscle. They just seem to have a freakishly successful strategy for choosing what aspects of the processor to throw that muscle at. Why is that strategy simultaneously remarkably efficient and remarkably high-performance? What enabled/led them to make those choices where others haven't?
  - jeffbee 6y ago
    I don't have any inside-Apple perspective, but my guess is having a tight feedback cycle between the profiles of their own software and the abilities of their own hardware has helped them greatly. The reason I think so is when I was at Google is was 7 years between when we told Intel what could be helpful, and when they shipped hardware with the feature. Also, when AMD first shipped the EPYC "Naples" it was crippled by some key uarch weaknesses that anyone could have pointed out if they had been able to simulate realistic large programs, instead of tiny and irrelevant SPEC benchmarks. If Apple is able to simulate or measure their own key workloads and get the improvements in silicon in a year or two they have a gigantic advantage over anyone else.
    - dcolkitt 6y ago
      Interesting point. This would suggest pretty sizable synergies from the oft-rumored Microsoft acquisition of Intel.
      - f6v 6y ago
        > Microsoft acquisition of Intel Could that possibly be approved by governments?
        gigatexal 6y ago
        Nope. Not a lawyer but I doubt it at all.
      - ChuckNorris89 6y ago
        Microsoft doesn't need to acquire Intel, they need to do what Apple did and acquire a stellar ARM design house that will build a chip with x86 translation, tailored to accelerate the typical workloads on Windows machines and sell those chips to the likes of Dell and Lenovo and tell developers "ARM Windows is the future, x86 Windows will be sunset in 5 years and no longer supported by us, start porting your apps ASAP and in the mean time, try our X86 emulator on our ARM silicon, it works great." Microsoft proved with the XBOX and Surface series they can make good hardware if they want, now they need to move to chip design.
        ralfd 6y ago
        Apple has at most 10% of the computer market and is just one player among many. I am skeptical Microsoft with their 90% dominance would or should be allowed this much power over the industry.
        sjwright 6y ago
        The traditional personal computer market isn't nearly as important as it used to be. However you slice the pie, there's no way you can define the pieces to be 10% Apple and 90% Microsoft with a straight face.
  - gameswithgo 6y ago
    I see two main things behind it: 1. they are the only ones who have 5nm chips because they paid a lot to TSMC for that right 2. they gave up on expandable memory, which lets them solder it right next to the cpu, which likely makes it easier to ship with really high clocks. and/or they just spent the money it takes to get binned lpddr4 at that speed. So a good cpu design, just like AMD and Intel have, but one generation ahead on node size, and fast ram. Its not special low latency ram or anything, just clocked higher than maybe any other production machine, though enthusiasts sometimes clock theirs higher on desktops!
    - mtgx 6y ago
      In other words: money. Throwing money at the (right) problems made them better than others. "But doesn't Intel have a lot of money, too?" Sure, but Intel has also been running around like a headless chicken this past decade (pretty much literally, since Otellini left) combined with them getting very complacent because they had "no real competition."
    - epistasis 6y ago
      > So a good cpu design, just like AMD and Intel have The design seems to be very different, in that it's far far wider, and supposedly has a much better branch predictor. > fast ram Is that a property of the RAM clock, or a function of a better memory controller? The RAM certainly doesn't appear to have any better latency.
      - gameswithgo 6y ago
        Right, latency isn't (much) affected by a higher clock rate. Getting ram to run fast requires both good ram chips and good controller/motherboard. and yes, obviously apples bespoke ARM cpu is quite a bit different than Zen3 Ryzens x86 cpu, but I'm not sure it is net-better. When Zen4 hits at 5nm I expect it will perform on par or better than the M1, but we won't know till it happens!
  - acdha 6y ago
    > What enabled/led them to make those choices where others haven't? Others have to some extent — AMD is certainly not out of the game — so I'd treat this more as the question of how they've been able to go more aggressively down that path. One of the really obvious answers is that they control the whole stack — not just the hardware and OS but also the compilers and high-level frameworks used in many demanding contexts. If you're Intel or Qualcomm, you have a wider range of things to support _and_ less revenue per device to support it, and you are likely to have to coordinate improvements with other companies who may have different priorities. Apple can profile things which their users do and direct attention to the right team. A company like Intel might profile something and see that they can make some changes to the CPU but the biggest gains would require work by a system vendor, a compiler improvement, Windows/Linux kernel change, etc. — they contribute a large amount of code to many open source projects but even that takes time to ship and be used.
    - astrange 6y ago
      Intel does lots of contributions across the OS (Linux and glibc) to compilers including their own (gcc, icc, ispc, etc). Their problems aren't their ability, it's that Intel is poorly managed and internal groups are constantly fighting with each other. Also, compiler support for CPUs is very overrated. Heavy compiler investment was attempted with Itanium and debunked; giant OoO CPUs like Intel's or M1 barely care about code quality, and the compilers have very little tuning for individual models.
      - acdha 6y ago
        > Intel does lots of contributions across the OS (Linux and glibc) to compilers including their own (gcc, icc, ispc, etc). Their problems aren't their ability, it's that Intel is poorly managed and internal groups are constantly fighting with each other. I wasn't just talking about Intel but the concept of separate CPU and compiler vendors in general. Intel contributes a ton of open source but even if they were perfectly organized it takes time for everything to happen on different schedules before it's generally available: get patches into something like Linux or gcc, wait possibly years for Red Hat to ship a release using the new version, etc. Certain users — e.g. game or scientific developers — might jump on a new compiler or feature faster, of course, but that's far from a given and it means they're not going to get the across-the-board excellent scores that Apple is showing. > Also, compiler support for CPUs is very overrated. Heavy compiler investment was attempted with Itanium and debunked; giant OoO CPUs like Intel's or M1 barely care about code quality, and the compilers have very little tuning for individual models. This isn't entirely wrong but it's definitely not complete. Itanium failed because brilliant compilers didn't exist and it was barely faster even with hand-tuned code, especially when you adjusted for cost, but that doesn't mean that it doesn't matter at all. I've definitely seen significant improvements caused by CPU family-specific tuning and, more importantly, when new features are added (e.g. SIMD, dedicated crypto instructions, etc.) a compiler or library which knows how to use those can see huge improvements on specific benchmarks. That was more what I had in mind since those are a great example of where Apple's integration shines: when they have a task like “Make H.265 video cheap on a phone” or “Use ML to analyze a video stream” they can profile the whole stack, decide where it makes sense to add hardware acceleration, and then update their choice of the compiler toolchain and higher-level libraries (e.g. Accelerate.framework) and ship the entire thing at the time of their choosing whereas AMD/Intel/Qualcomm and maybe nVidia have to get Microsoft/Linux and maybe someone like Adobe on board to get the same thing done. That isn't a certain win — Apple can't work on everything at once and they certainly make mistakes — but it's hard to match unless they do screw up.
  - deleted 6y ago
    [deleted]
  - mhh__ 6y ago
    I think it's worth saying that because AMD have only just really hit their stride, Intel were under almost zero pressure to improve which has really hurt them especially with the process. X86 is definitely a coefficient overhead, but if Intel put their designs on 5nm they'd look pretty good too - Jim Keller (when he was still there) hinted their offerings for a year or so in the future are significantly bigger to the point of him judging it to be worth mentioning so I wouldn't write them off.
    - megap1x3ls 6y ago
      Do you know what 5nm is?
  - baybal2 6y ago
    > There is no "magic" at play here, it's just lots and lots of raw muscle. They just seem to have a freakishly successful strategy for choosing what aspects of the processor to throw that muscle at. There is no freakishly successful strategy at play there as well. It's just all previous attempts at "fast ARM" chip were rather half hearted "add a pipeline step there, add extra register there, increase datapath width there," and not to squeeze it to the limit.
  - TYPE_FASTER 6y ago
    Apple has been iterating on their proprietary mobile ARM-based processors since 2010, and has gotten really good at it. I would imagine that producing billions of consumer devices with these chips has helped give them a lot of experience in shortened time frame. I also wonder if having the hardware and software both worked on in-house is an advantage. I mean, if you're developing power management software for a mobile OS, and you're using a 3rd-party vendor, then you read the documentation, and work with the vendor if you have questions. If it's all internal, you call them, and could make suggestions on future processor design too based on OS usage statistics and metrics.
    - simonh 6y ago
      In fact there is clear evidence of this with M1. It has optimised instructions to speed up retain/release of NSObject subclasses, which is a frequent operation on almost all Objective-C and Swift classes. They also designed the M1 to support a memory management profile used by x86 (and not ARM) to accelerate Rosetta translated binaries. I'm sure there are more.
  - gavin_gee 6y ago
    didnt they also make some interesting hires a few years ago like Anand from Anandtech and some other silicon vets that likely helped them design the M1 approach?
  - barkingcat 6y ago
    The answer is that they have raw hard numbers from the hundres of millions of iPads/iPhones sold each year, and can use the metrics from those devices to optimize the next generation of devices. These improvements didn't come from nowhere. It came from iterations of iOS hardware.
  - coldtea 6y ago
    >Why is that strategy simultaneously remarkably efficient and remarkably high-performance? What enabled/led them to make those choices where others haven't? The things people give them complains about: (a) keeping a walled garden, (b) moving fast and taking the platform to new directions all at once (c) controlling the whole stack Which means they're not beholden to compatibility with third party frameworks and big players, or with their own past, and thus can rely on their APIs, third party devs etc, to cater to their changes to the architecture. And they're not chained to the whims of the CPU vendor (as the OS vendor) or the OS vendor (as the CPU vendor) either, as they serve the role of both. And of course they benchmarked and profiled the hell out of actual systems.
    - fartcannon 6y ago
      They could still do all this shit without the walled garden. To me, it suggests they aren't willing to compete. They're anti-competitive.
      - marrvelous 6y ago
        With the walled garden, Apple can set enforceable timelines for the software ecosystem to adopt to architectural changes. Remember the transition to arm64? Apple forced everything on the App Store to ship universal binaries. Without the App Store walled garden, software isn’t required to keep up to date with architectural changes. Instead, keeping current is only a requirement to being featured on the App Store (which would just be a single way to install software, not the only method).
        danaris 6y ago
        Well, and on the Mac, it's not the only method. The walled garden here has big open gates. That said, all software on the Mac, post-Catalina, has to be 64-bit, whether it's distributed through the Mac App Store or not, because the 32-bit system libraries are no longer included at all.
        coldtea 6y ago
  - adam_arthur 6y ago
    Certainly Apple's processors are far ahead, but they're a full process generation (5nm) ahead of their competitors. They paid their way to that exclusive right through TSMC. I'm sure they'll still come out ahead in benchmarks, but the numbers will be much closer once AMD moves to 5nm. You absolutely cannot fairly compare chips from different fab generations. I don't see many comments hammering this point home enough... it's not like the performance gap is through engineering efforts that are leagues ahead. Certainly some can be attributed to that, and Apple has the resources to poach any talent necessary.
    - wmf 6y ago
      From a customer's perspective it's not my problem. Everyone had the opportunity to bid on that fab capacity and they decided not to.
      - adam_arthur 6y ago
        Yeah, totally agreed. But if you read these comments, they seem to be in total amazement about the performance gap and not acknowledging how much of an advantage being a fab generation ahead is. Customers don't care, but discussion of the merits of the chip should be more nuanced about this. It also implies that the gap won't exist for very long, as AMD will move onto 5nm soon
        tandr 6y ago
        > It also implies that the gap won't exist for very long, as AMD will move onto 5nm soon ... yes, if there is any capacity left. Capacity for the new process is a limited resource after all.
        jack_pp 6y ago
        People keep pointing this out but has Intel had such significant performance improvements since sandy bridge? With x86 it seems that lately you would be foolish to upgrade less than once every 3-4 years because the difference is just not that significant
  - jandrese 6y ago
    It seems like Apple listened when people talked about how all modern processors bottleneck on memory access and decided to focus heavily on getting those numbers better. Of course this leads to the question that if everyone in the industry knew this was the issue why weren't Intel and AMD pushing harder on it? They already both moved the memory controller onboard so they had the opportunity to aggressively optimize it like Apple has done, but instead we have year after year where the memory lags behind the processor in speed improvements, to the point where it is ridiculous how many clock cycles a main memory access takes on a modern x86 chip.
    - proverbialbunny 6y ago
      My guess is it had to do with limitations tied to the x86_64 instruction set. It doesn't matter how much modifications you do, if you don't start with a good foundation, you're going to be limited to that foundation.
      - nkurz 6y ago
        I think the current consensus among experts is that the instruction set is not the limiting factor. Modern x64 microprocessors have a separate front-end that handles instruction decoding. These instructions are decoded to internal proprietary "micro-ops". The internal buffers and actual execution units see only these µops. One can measure where the bottlenecks are, and it's rare to find that the front-end is the bottleneck. While it's arguably true that x64 is a poorly designed "foundation", it's unlikely to be causing any performance difference here.
        wtallis 6y ago
        > One can measure where the bottlenecks are, and it's rare to find that the front-end is the bottleneck. Part of this is due to the fact that x86 processor designers won't include more execution units than they can feed from their instruction decoders. Apple's processors are much wider than x86 on both the decode and execution resources, and it's pretty clear that the M1 would not perform as well if its decoders were as narrow as current x86 cores.
  - SurfingInVR 6y ago
    Something I've seen no one else mentioning: Apple's low-spec tier is $1000, not $70.
    - square_usual 6y ago
      It's $699, for a complete device, not a part of one.
      - perryizgr8 6y ago
        What's the screen resolution on that $699 "complete device"?
        FreshFries 6y ago
        5120 by 2880
        square_usual 6y ago
        Comparable to a $70 processor, at least.
  - dv_dt 6y ago
    No fighting the sales department on where to put the market segmentation bottlenecks?
  - smoldesu 6y ago
    The big "enabler" was their mass-purchase of 5nm lithography across the board. Even still though, 4ghz*8c isn't anything new, and isn't really that remarkable besides the low TDP (which is incidentally dwarfed by the display, which draws up to 5x more power than the CPU does). I think the big issue is that Apple has painted themselves into a corner here: ARM won't play nice with the larger CPUs they want to make, and the pressure for them to provide a competent graphics solution on custom silicon is mounting. They spent a lot of time this generation marketing their "energy efficiency" and battery life, but many consumers/professionals (myself included) don't really care about either of these things.
    - krunkcoin 6y ago
      "the display, which draws up to 5x more power than the CPU does" - wat? Apple-supplied monitoring tools report that M1 under full load (all CPU and GPU cores) can draw over 30 watts. Laptop displays don't consume 150 watts. If anything, the displays in either of the M1 portables likely consume about 5 times less than 30W, even at full brightness. "ARM won't play nice with the larger CPUs they want to make" - wat? Apple holds an architectural license. This means they paid a lot upfront a long time ago and therefore have a more or less perpetual right to design their own Arm cores without input from Arm. "a competent graphics solution" - Also wat? M1 has an excellent GPU. It doesn't compete with discrete GPUs that use 300 watts, but that's fine: M1 is the chip for entry level Macs, designed for the smallest and lightest segments of their notebook line. And in that product segment, it has been every bit as much a revelation as the CPU. It's very fast, and uses little power given the performance. What exactly do you think is going to happen when they scale that basic GPU design up? Despite your dismissiveness, in modern silicon architecture energy efficiency is incredibly important: for any given power budget, the more efficient you are the more performance you can deliver. The performance Apple gets out of about 10W on M1 suggests they'll have few problems building a larger GPU to compete with Nvidia and AMD discrete GPUs.
- ed25519FUUU 6y ago
  In other words, it’s better architecture. If anything this makes it seem more impressive to me.
  - amelius 6y ago
    No, it's the same architecture but with different parameters. It's like the difference between the situation where every car uses 4 cylinders, and then Apple comes along and makes a car with 5 cylinders.
    - kllrnohj 6y ago
      Your analogy was so close! It's Apple comes along and makes an 8 cylinder engine. Since, you know, the other CPUs are 4-wide decode and Apple's M1 is 8-wide decode :)
      - ben-schaaf 6y ago
        Zen 3 is effectively 8 wide with their micro-op cache. Intel similarly has been 6 wide for ages.
- gameswithgo 6y ago
  What other laptop ships with LPDDR4X clocked at 4267? I agree though that being closer to the cpu isn't having any appreciable effect on latency, but being soldered close to the cpu probably does make it easier for them to hit that high clock rate.
  - wmf 6y ago
    Tiger Lake laptops such as the XPS 13.
  - jeffbee 6y ago
    As WMF mentions, Tiger Lake laptops like my Razer Book have the same memory. It is not appreciably closer to the CPU in the Apple design. In Intel's Tiger Lake reference designs the memory is also in two chips that are mounted right next to the CPU.
    - danaris 6y ago
      And (genuine question) how do the Tiger Lake laptops compare with the M1 MacBooks thus far?
      - skavi 6y ago
        AnandTech has decent benchmarks for both Tiger Lake [0] and M1 [1]. [0]: https://www.anandtech.com/show/16084/intel-tiger-lake-review-deep-dive-core-11th-gen https://www.anandtech.com/show/16084/intel-tiger-lake-review... [1]: https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste...
      - jeffbee 6y ago
        The outcome seems to depend greatly on the physical design of the laptops. The elsewhere-mentioned Dell XPS 13 has a particularly poor cooling design, which is why I chose the Razer Book instead. Despite being marketed in a very silly way to gamers only, it seems to have competent mechanical design.
- titzer 6y ago
  > it can do with a single core what an Intel part needs all cores to accomplish. Care to explain what you mean specifically by this?
  - saagarjha 6y ago
    The M1 has extremely high single-core performance.
    - temac 6y ago
      It is not 4 times faster than an Intel core, though...
      - saagarjha 6y ago
        It is in memory performance, which is what I assumed was being measured here.
        kllrnohj 6y ago
        How are you defining memory performance and where are your supporting comparisons? This article only discusses the M1's behavior, and makes no comparisons to any other CPU.
        FabHK 6y ago
        FWIW, I ran it on a MacBook Pro (13-inch, 2019, Four Thunderbolt 3 ports), 2.4 GHz Quad-Core Intel Core i5, 8 GB 2133 MHz LPDDR3: two : 49.6 ns (x 5.5) two+ : 64.8 ns (x 5.2) three: 72.8 ns (x 5.6) EDIT to add: above was just `cc`. Below is with `cc -O3 -Wall`, as in Lemire's article: two : 62.8 ns (x 7.1) two+ : 69.2 ns (x 5.5) three: 95.3 ns (x 7.3)
        namibj 6y ago
        You _need_ to use -mnative because it otherwise retains backwards compatibility to older x86.
- ksec 6y ago
  >HN memes about M1 memory will die It is not only HN. It is practically the whole Internet. Go around the Top 20 hardware and Apple website forum and you see the same thing, also vastly amplify by a few KOL on twitter. I dont remember I have ever seen anything quite like it in tech circle. People were happily running around spreading misinformation.
  - jeffbee 6y ago
    Yeah, I know. There was some kid on Twitter who was trying to tell me that it was the solder in an x86 machine (he actually said "a Microsoft computer") that made them slower. Apple, without the solder was much faster. According to this person's bio they had an undergraduate education in computer science ¯\_(ツ)_/¯
  - Bootvis 6y ago
    What is a KOL?
    - tyingq 6y ago
      "Key Opinion Leader". I think it's the new word for "Influencer".
      - walterbell 6y ago
        Who introduced the term KOL and bestows the K title?
      - ksec 6y ago
        I am pretty sure KOL predates Influencer in modern internet usage. Before that they were simply known as Internet Celebrities. May be it is rarely used now. So apology for not explaining the acronyms.
        secondcoming 6y ago
        First I've heard of it!
- sliken 6y ago
  The most impressive thing I've seen is that when accessed in a TLB friendly fashion that the latency is around 30ns. Anandtech has a graph showing this, specifically the R per RV prange graph. I've verified this personally with a small microbenchmark I wrote. I've not seen anything else close to this memory latency.
  - tandr 6y ago
    Sorry, what would AMD's or Intel's "latest and greatest" numbers for the same be?
    - sliken 6y ago
      Here's the M1: https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... Scroll down to the latency vs size map and look at the R per RV prange. That gets you 30ns or so. Similar for AMD's latest/greatest the Ryzen 9 5950X: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-dive-review-5950x-5900x-5800x-and-5700x-tested/5 https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di... The same R per RV prange is in the 60ns range.
      - epistasis 6y ago
        Could this be coming from the page size being 4x as large for Apple Silicon versus x86? I don't fully understand the benchmark, but it appears to be accessing a variety of pages from the same first level TLB lookup? It's been a long time since I dealt with this stuff (wanted to get 1GB huge pages in Linux for some huge huge hash tables), so maybe I'm misunderstanding.
        sliken 6y ago
        Cachelines, page sizes, and size of the TLB all play a role. But with tinkering you can see those effects yourself and I played with 1, 2, 4, 8, 16, and 32 "pages" which I assumed were 4KB each and didn't see much difference. Measured latencies do increase slowly, but you expect that as the TLB becomes progressively more of a bottleneck. If you use a 1GB array and see full random with much higher latency than a sliding window then you can be pretty sure that the page size is much less than 1GB. Getting the cacheline off by a factor of 2 does make a small difference since you get occasional cache hits instead of zero, but as long as the array tested is several times larger than cache the impact is small. But all in all the M1 has excellent memory bandwidth, excellent latency, and shows significantly better throughput on random workloads as you use more cores. Normal PC desktops have 2 memory channels (even the higher end i7/i9/ryzen7/ryzen9), only the $$$$ workstation chips like threadripper and some of the $$$$ Intel's have more. The little ole M1 in a mac mini, starting at $700 has at least 8 memory channels. So basically the M1 delivers on all fronts, larger and lower latency caches, wide issue, large reorder buffers, excellent IPC, and excellent power efficiency.
- megablast 6y ago
  You’ll get people guessing, since Apple itself puts out so little information.
- PragmaticPulp 6y ago
  I don't understand this competition to attribute the M1's speed to one specific change, while downplaying all of the others. M1 is fast because they optimized everything across the board. The speed is the cumulative result of many optimizations, from the on-die memory to the memory clock speed to the architecture.
  - adam_arthur 6y ago
    It's fast because they optimized everything across the board, and also paid for exclusive access to TSMC 5nm process.
- foldr 6y ago
  >or that it's faster because the memory is 2mm closer to the CPU (not that either) Not to disagree with your overall point, but 2mm is a long way when dealing with high frequency signals. You can't just eyeball this and infer that it makes no difference to performance or power consumption.
  - jeffbee 6y ago
    If it works, it works. There will be no observable performance difference for DDR4 SDRAM implementations with the same timing parameters, regardless of the trace length. There are systems out there with 15cm of traces between the memory controller pins and the DRAM chips. The only thing you can say against them is they might consume more power driving that trace. But you wouldn't say they are meaningfully slower.
    - foldr 6y ago
      You can't just eyeball the PCB layout for a GHz frequency circuit and say "yeah that would definitely work just the same if you moved that component 2mm in this direction". It's certainly possible to use longer trace lengths, but that may come with tradeoffs. >The only thing you can say against them is they might consume more power driving that trace Power consumption is really important in a laptop, and Apple clearly care deeply about minimising it. For all we know for sure, moving the memory closer to the CPU may have been part of what's enabled Apple to run higher frequency memory with acceptable (to them) power draw.
- titzer 6y ago
  Frankly, I find Lemire does oversimplified, poor-quality control, back-of-the-envelope microbenchmarking all the time that provides little to no insight other than establishing a general trend. It's sophomoric and a poor demonstration about how to well-controlled benchmarking that might yield useful, repeatable, and transferrable results.
  - alecco 6y ago
    Can you give an example? I've seen Lemire correct his posts on many occasions and the source code is published. I don't know many blogs doing anything remotely like that.
    - titzer 6y ago
      Sure. He often benchmark some small C++ code on his "laptop" CPU (which one exactly? microarchs matter!) and then committing classic microbenchmarking pitfalls such as: - benchmarking something small enough to inspect machine code, but not inspecting machine code - not plotting distribution, average, variance etc - no attention paid to CPU frequency governor settings - measuring too short a run - measuring too small a dataset that it fits entirely in L1
- RachelF 6y ago
  >that it's "on-die" (it's not) It appears to be mounted on the same chip package. Why did Apple do this if not for speed?
  - wmf 6y ago
    On-package memory is not faster. I suspect it is more power efficient though.
    - rkangel 6y ago
      It makes it easier to get to a particular clock speed. The geometry, interconnect lengths etc are all tightly controlled, the noise is less because you're not on the main PCB and you have interconnect options that aren't whatever your PCB process is (e.g. commonly gold wires).
- deleted 6y ago
  [deleted]
s800 6y ago
What's the precision of these ns level measurements?
- mhh__ 6y ago
  The answer to that is usually very context dependant, and on what you're measuring. As long as you use a histogram first and don't blindly calculate (say) the mean it should he obvious. Two examples( that are slightly bigger than this but the same principles apply): If you benchmark a std::vector at insertion, you'll see a flat graph with n tall spikes at ratios of it's reallocation amount apart, and it scales very very well. The measurements are clean. If, however, you do the same for a linked list you get a linearly increasing graph but it's absolutely all over the place because it doesn't play nice with the memory hierarchy. The std_dev of a given value of n might be a hundred times worse than the vector.
- CyberRabbi 6y ago
  Clock_gettime(CLOCK_REALTIME) on macos provides nanosecond-level precision.
  - geocar 6y ago
    I seem to recall OSX didn't used to have clock_gettime, so it's news to me that it even exists -- I might have been away from OSX too long. Is there any performance difference between that and mach_absolute_time() ?
    - saagarjha 6y ago
      It's new in macOS Sierra. I believe mach_absolute_time is slightly faster but not by much–both just read the commpage these days to save on a syscall.
    - lilyball 6y ago
      It was added some years ago, and I believe mach_absolute_time is actually now implemented in terms of (the implementation of) clock_gettime. The documentation on mach_absolute_time now even says you should use clock_gettime_nsec_np(CLOCK_UPTIME_RAW) instead. macOS also has clock constants for a monotonic clock that increases while sleeping (unlike CLOCK_UPTIME_RAW and mach_absolute_time).
kergonath 6y ago
It’s a good introduction, but it’s a bit disappointing that it ends that way. I’d love to read more about what’s behind the figure and more technical info about how it might work.
- alblue 6y ago
  This isn’t specific to the M1 but I tap about cache lines in my last QCon presentation (where I also suggested that a 128b cache line wasn’t far away): https://www.infoq.com/presentations/microarchitecture-modern-cpu/ https://www.infoq.com/presentations/microarchitecture-modern... However the speed benefits come from a much larger L1 cache and the fact that the ram is in the same chip which will reduce latency that is the benefit for most of it. The program (instruction) cache is also a lot bigger and has the advantage that as a fixed size isa can be much wider in execution than in x86 but that’s unlikely to be of benefit here, other than perhaps slightly in terms of queuing up multiple outstanding loads.
  - kristianp 6y ago
    This post says that the m1 has a 128 byte cache line size. So that time has arrived! https://news.ycombinator.com/item?id=25660769 https://news.ycombinator.com/item?id=25660769
    - alblue 6y ago
      Yup, my presentation was back in March 2020 and the M1 came out later in the year — sooner than I was expecting, TBH; I thought that it was a couple of years out when I said it :-)
  - kergonath 6y ago
    Thanks!
lrossi 6y ago
Shouldn’t you choose the random numbers such that array[idx1] ^ array[idx1 + 1] are guaranteed to fall in the same cache line? Assuming that it has that. Right now some accesses cross the end of the cache line.
- CyberRabbi 6y ago
  Technically you are correct but it’s expected to cross a cache line 1/16 times (or however many ints there are in a cache line). There is an implicit assumption that that is relatively infrequent enough that it shouldn’t increase the average time too much, but that assumption should be tested.
nabla9 6y ago
What is the cache line size and page table size in M1? sysconf(_SC_PAGESIZE); /* posix */ Can you get direct processor information like LEVEL1_ICACHE_ASSOC and LEVEL1_ICACHE_LINESIZE from the M1??
- momothereal 6y ago
  `getconf PAGESIZE` returns 16384 on the base M1 MacBook Air. The L1 cache values aren't there. The macOS `getconf` doesn't support -a (listing all variables), so they may just be under a different name. edit: see replies for `sysctl -a` output
  - lilyball 6y ago
    Is it possibly exposed via sysctl, which does support a flag to list all variables?
    - messe 6y ago
      From sysctl -a on my M1: hw.cachelinesize: 128 hw.l1icachesize: 131072 hw.l1dcachesize: 65536 hw.l2cachesize: 4194304 EDIT: also, when run under Rosetta hw.cachelinesize is halved: hw.cachelinesize: 64 hw.l1icachesize: 131072 hw.l1dcachesize: 65536 hw.l2cachesize: 4194304
      - Nokinside 6y ago
        M1 cache lines are double of what is commonly used by Intel, AMD and other ARM microarchtectures use. That's significant difference.
        deleted 6y ago
        [deleted]
      - JonathonW 6y ago
        Compared to the i9-9880H in my 16" MacBook Pro: hw.cachelinesize: 64 hw.l1icachesize: 32768 hw.l1dcachesize: 32768 hw.l2cachesize: 262144 hw.l3cachesize: 16777216 The M1 doubles the line size, doubles the L1 data cache (i.e. same number of lines), quadruples the L1 instruction cache (i.e. double the lines), and has a 16x larger L2 cache, but no L3 cache.
waterside81 6y ago
For people who know more about this stuff than me: are these sorts optimizations only possible because Apple controls the whole stack and can make the hardware & OS/software perfectly match up with one another or is this something that Intel can do but doesn't for some reasons (tradeoffs)?
- deleted 6y ago
  [deleted]
- wmf 6y ago
  No, there's no cross-stack optimization here. The M1 gives very high performance for all code.
  - qeternity 6y ago
    I think this gets lost in the fray between the "omg this is magic" and then the Apple haters. The M1 is a very good chip. Apple has hired an amazing team and resourced them well. But from a pure hardware perspective, the M1 is quite evolutionary. However the whole Apple Silicon experience is revolutionary and magical due to the tight software pairing. Both teams deserve huge praise for the tight coordination and unreal execution.
    - acdha 6y ago
      I think this is part of the reason where there are so many people trying to find reasons to downplay it: humans love the idea of “one weird trick” which makes a huge difference and we sometimes find those in tech but rarely for mature fields like CPU design. For many people, this is unsatisfying like asking an athlete their secret, and getting a response like “eat well, train a lot, don't give up” with nary a shortcut in sight.
- AnthonyMouse 6y ago
  > are these sorts optimizations only possible because Apple controls the whole stack and can make the hardware & OS/software perfectly match up with one another or is this something that Intel can do but doesn't for some reasons (tradeoffs)? Interestingly it's the other way around. Apple is using TSMC's 5nm process (they don't have their own fabs), which is better than Intel's in-house fabs, so it's Intel's vertical integration which is hurting them compared to the non-vertically integrated Apple. Also, the answer to "is this only possible because of vertical integration" is always no. Intel and Microsoft regularly coordinate to make hardware and software work together. Intel is one of the largest contributors to the Linux kernel, even though they don't "own" it. Two companies coordinating with one another can do anything they could do as an individual company. Sometimes the efficiency of this is lower because there are communication barriers and isn't a single chain of command. But sometimes it's higher because you don't have internal politics screwing everything up when the designers would be happy with outsourcing to TSMC because they have a competitive advantage, but the common CEO knows that would enrich a competitor and trash their internal investment in their own fabs, and forces the decision that leads to less competitive products.
djacobs7 6y ago
Is the article saying that the M1 is slower than we would have expected in this case? My understanding, based on the article, is that a normal processor, we would have expected arr[idx] + arr[idx+1] and arr[idx] to take the same amount of time. But the M1 is so parallelized that it goes to grab both arr[idx] and arr[idx+1] separately. So we have to wait for both of those two return. Meanwhile, on a less parallelized processor, we would have done arr[idx] first and waited for it to return, and the processor would realize that it already had arr[idx+1] without having to do the second fetch. Am I understanding this right?
- phkahler 6y ago
  >> My understanding, based on the article, is that a normal processor, we would have expected arr[idx] + arr[idx+1] and arr[idx] to take the same amount of time. That depends. If the two accesses are on the same cache line, then yes. But since idx is random that will not happen sometimes. He never says how big array[] is in elements or what size each element is. I thought DRAM also had the ability to stream out consecutive addresses. If so then it looks like Apple could be missing out here. Then again, if his array fits in cache he's just measuring instruction counts. His random indexes need to cover that whole range too. There's not enough info to figure out what's going on.
  - SekstiNi 6y ago
    > There's not enough info to figure out what's going on. If you only look at the article this is true. However, the source code is freely available: https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2021/01/06 https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/...
    - deleted 6y ago
      [deleted]
    - africanboy 6y ago
      I ran the benchmark on my system It's a 6 years old system, fastest times are in the 25ns range - 2-wise+ is 5% slower than 2-wise - 3-wise is 46% slower than 2-wise - 3-wise is 39% slower than 2-wise+ on the M1 - 2-wise+ is 40% slower than 2-wise - 3-wise is 46% slower than 2-wise - 3-wise is 4% slower than 2-wise+
jayd16 6y ago
>our naive random-memory model Doesn't everyone use the (I believe) still valid concepts of latency and bandwidth?
- wyldfire 6y ago
  How do latency and bandwidth relate to the cost model for the code in the benchmark? When creating the model discussed in the post, we're using it to try to make a static prediction about how the code will execute. Note that the goal of the post is not to merely measure the memory access performance, it's to understand the specific microarchitecture and how it might deliver the benefits that we see in benchmarks.
- whoisburbansky 6y ago
  The concepts are still broadly valid, the naivety being referred to is the assumption that two non adjacent memory reads will be twice as slow as one memory read or two adjacent reads.
- sroussey 6y ago
  Depends on context. For example, what is the bandwidth and latency when you ask for the value at the same memory address in an infinite loop? And how does that compare to the latency and bandwidth of a memory module you buy on NewEgg?
  - fluffy87 6y ago
    L1 BW. When people use BW in their performance models, they don’t use only 1 bandwidth, but whatever combination of bandwidth makes sense for the _memory access pattern_. So if you are always accessing the same word, the first acces runs at DRAM BW, and subsequent ones at L1 BW, and any meaningful performance model will take that into account.
foota 6y ago
Is this per core or shared between cores?
- hundchenkatze 6y ago
  Per core I think, emphasis is mine. > It looks like a single core has about 28 levels of memory parallelism, and possibly more.
  - foota 6y ago
    I was wondering if this might be a shared resource though, since it doesn't seem they tested with multiple threads.
wrsh07 6y ago
Ok, summary: This article lays out three scenarios: 1) accessing two random elements 2) accessing 3 random elements 3) accessing two pairs of adjacent elements (same as (1) but also the elements after each random element) It then does some trivial math to use the loaded data. A naive model might only consider memory accesses and might assume accessing an adjacent element is free. On the Mac m1 core, this is not the case. While the naive model might expect cases 1 & 3 to cost the same and case 2 to cost 50% more, instead cases 2 & 3 are nearly the same (3 slightly faster) and case 2 is about 50% more expensive than 1.
- jayd16 6y ago
  I don't really understand the comparison because it seems like scenario 3 (2+) is doing more XORs and twice the accesses to array over the same amount of iterations. We have to assume these are byte arrays, yes? Or at least some size that's smaller than the cache line. You would still pay for the extra unaligned fetches. I don't think this is a valid scenario at all, M1 or not. Anyone want to run these tests on an Intel machine and let us know if the authors "naive model" test hold there?
  - africanboy 6y ago
    I did it on an old i7 laptop https://news.ycombinator.com/item?id=25661055 https://news.ycombinator.com/item?id=25661055
  - wrsh07 6y ago
    The point of the naive model is that you assume memory accesses dominate That is, the math part is so trivial compared to the memory access that you could do a bunch of math and you would still only notice a change in the number of memory accesses. Also it looks like the response to yours links their test and the naive model predicts correctly
    - jayd16 6y ago
      I think 5% is a non-trivial difference but alright, its a much bigger difference on the M1. I guess I still don't understand whats going on here. Scenario 1 has two spatially close reads followed by two dependent random access reads. Scenario 3 (2+) has two spatially close reads, and two pairs of dependent random access reads of two spatially close locations. Why does it follow that this is caused by a change in memory access concurrency? The two required round trips should dominate both on the M1 and an Intel but for some reason the M1 performs worse than that. Why? I can't help but feel the first snippet triggers some SIMD path while the 3rd snippet fails to.
syntaxing 6y ago
I’m super curious if it’s true that my 8GB M1 will die quickly because of the aggressive swaps. I guess time will tell.
- acdha 6y ago
  FWIW, I have a 2010 MBA which was _heavily_ used for years as a primary development system. The SSD only started to show signs of degraded performance last year and that wasn't massive. I would be quite surprised if the technology has become worse.
deleted 6y ago
[deleted]
willvarfar 6y ago
A lot of commenters here are saying that Apples advantage is that it can profile the real workloads and optimise for that. Well that’s true and could very well be an advantage. An advantage in that they did it, not in that only they have access to it. Intel and AMD can trivially profile real world workloads too. Did they? I don’t know what Apple did, but the impression I get is that intel certainly hasn’t.
- gameswithgo 6y ago
  of course every cpu designer is using real world workloads to guide design
helloycombinat 6y ago
I don't have a M1 chip but it'd be interesting to see what happens if we make sure if the random value is never n*CACHE_SIZE-1
fctorial 6y ago
I can't find any info about the memory bus of apple m1. Is it 8 channels 16 bit each? That's drastically different from AMDs 2 channels 64 bit each. It looks like apple m1 is much less eager when caching memory rows. Maybe because it doesn't have l3 cache. Edit: This test utilizes the 8x16bit memory bus of apple m1 fully. It's mostly just fetching random locations from memory, which can all be parallelized by the cpu pipeline. It explains why the results are exactly 4x slower on my ryzen 3 with 2 memory channels. So the summary is that m1 is optimized for dynamic languages that tend to do a DDOS attack on RAM with a lot of random memory access, but it might take a performance hit with compiled languages and traditional HPC techniques that tend to process data in sequence like ECS.
- skohan 6y ago
  That's an interesting observation - as someone who's built a few ECS implementations, one of the things I've always taken for granted is that things like cache line size are more or less set in stone, given the ubiquity of x86, so it's interesting to consider that the rise of ARM might create additional complexities there. I'm a bit of two minds about this: on the one hand, for a long time I've wanted a language for writing allocators which is more explicit about memory, and offers good abstractions for low-level memory operations (maybe Zig is going in this direction). In some sense, it feels like the move towards programmers thinking less about memory management has been a bit of a dead-end, and what we really want is better tools for memory management. Fragmentation in terms of how processors handle memory goes against this goal in some ways. On the other hand, it's a bit of a "holy grail" to imagine a hardware stack which obviates the need for memory optimization, and really does treat loading from and storing to memory anywhere on the heap as the same. But I imagine that the interesting things which the M1 is doing with memory are helping a lot with the worst case performance, and maybe even average case performance, but they're probably not doing much for the best case.
- Dylan16807 6y ago
  That would make sense for LPDDR4 but it apparently claims to have a 128 byte cache line size and I'm not sure how to square that with 16 bit channel width.
gokulkrishh09 6y ago
Interesting article.
lincpa 6y ago
Apple M1 chip adopts Warehouse/Workshop Model - Warehouse: unified memory - Workshop: CPU, GPU and other cores - Product( material): information,data there’s also a new unified memory architecture that lets the CPU, GPU, and other cores exchange information between one another, and with unified memory, the CPU and GPU can access memory simultaneously rather than copying data between one area and another. Accessing the same pool of memory without the need for copying speeds up information exchange for faster overall performance. reference: 1. Developer Delves Into Reasons Why Apple’s M1 Chip is So Fast. https://www.macrumors.com/2020/11/30/m1-chip-speed-explanation-developer/ https://www.macrumors.com/2020/11/30/m1-chip-speed-explanati... 2. The Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Warehouse/Workshop Model https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow

372 points by luigi23 6y ago

Great practical information. Nice to see people who know what they are talking about putting data out there. I hope eventually these persistent HN memes about M1 memory will die: that it's "on-die" (it's not), that it's the only CPU using LPDDR4X-4267 (it's not), or that it's faster because the memory is 2mm closer to the CPU (not that either). It's faster because it has more microarchitectural resources. It can load and store more, and it can do with a single core what an Intel part needs all cores to accomplish.

pdpi 6y ago
This seems to be a recurring theme with the M1, and one that, in a sense, actually baffles me even more than the alternative. There is no "magic" at play here, it's just lots and lots of raw muscle. They just seem to have a freakishly successful strategy for choosing what aspects of the processor to throw that muscle at. Why is that strategy simultaneously remarkably efficient and remarkably high-performance? What enabled/led them to make those choices where others haven't?
- jeffbee 6y ago
  I don't have any inside-Apple perspective, but my guess is having a tight feedback cycle between the profiles of their own software and the abilities of their own hardware has helped them greatly. The reason I think so is when I was at Google is was 7 years between when we told Intel what could be helpful, and when they shipped hardware with the feature. Also, when AMD first shipped the EPYC "Naples" it was crippled by some key uarch weaknesses that anyone could have pointed out if they had been able to simulate realistic large programs, instead of tiny and irrelevant SPEC benchmarks. If Apple is able to simulate or measure their own key workloads and get the improvements in silicon in a year or two they have a gigantic advantage over anyone else.
  - dcolkitt 6y ago
    Interesting point. This would suggest pretty sizable synergies from the oft-rumored Microsoft acquisition of Intel.
    - f6v 6y ago
      > Microsoft acquisition of Intel Could that possibly be approved by governments?
      - gigatexal 6y ago
        Nope. Not a lawyer but I doubt it at all.
    - ChuckNorris89 6y ago
      Microsoft doesn't need to acquire Intel, they need to do what Apple did and acquire a stellar ARM design house that will build a chip with x86 translation, tailored to accelerate the typical workloads on Windows machines and sell those chips to the likes of Dell and Lenovo and tell developers "ARM Windows is the future, x86 Windows will be sunset in 5 years and no longer supported by us, start porting your apps ASAP and in the mean time, try our X86 emulator on our ARM silicon, it works great." Microsoft proved with the XBOX and Surface series they can make good hardware if they want, now they need to move to chip design.
      - ralfd 6y ago
        Apple has at most 10% of the computer market and is just one player among many. I am skeptical Microsoft with their 90% dominance would or should be allowed this much power over the industry.
        sjwright 6y ago
        The traditional personal computer market isn't nearly as important as it used to be. However you slice the pie, there's no way you can define the pieces to be 10% Apple and 90% Microsoft with a straight face.
- gameswithgo 6y ago
  I see two main things behind it: 1. they are the only ones who have 5nm chips because they paid a lot to TSMC for that right 2. they gave up on expandable memory, which lets them solder it right next to the cpu, which likely makes it easier to ship with really high clocks. and/or they just spent the money it takes to get binned lpddr4 at that speed. So a good cpu design, just like AMD and Intel have, but one generation ahead on node size, and fast ram. Its not special low latency ram or anything, just clocked higher than maybe any other production machine, though enthusiasts sometimes clock theirs higher on desktops!
  - mtgx 6y ago
    In other words: money. Throwing money at the (right) problems made them better than others. "But doesn't Intel have a lot of money, too?" Sure, but Intel has also been running around like a headless chicken this past decade (pretty much literally, since Otellini left) combined with them getting very complacent because they had "no real competition."
  - epistasis 6y ago
    > So a good cpu design, just like AMD and Intel have The design seems to be very different, in that it's far far wider, and supposedly has a much better branch predictor. > fast ram Is that a property of the RAM clock, or a function of a better memory controller? The RAM certainly doesn't appear to have any better latency.
    - gameswithgo 6y ago
      Right, latency isn't (much) affected by a higher clock rate. Getting ram to run fast requires both good ram chips and good controller/motherboard. and yes, obviously apples bespoke ARM cpu is quite a bit different than Zen3 Ryzens x86 cpu, but I'm not sure it is net-better. When Zen4 hits at 5nm I expect it will perform on par or better than the M1, but we won't know till it happens!
- acdha 6y ago
  > What enabled/led them to make those choices where others haven't? Others have to some extent — AMD is certainly not out of the game — so I'd treat this more as the question of how they've been able to go more aggressively down that path. One of the really obvious answers is that they control the whole stack — not just the hardware and OS but also the compilers and high-level frameworks used in many demanding contexts. If you're Intel or Qualcomm, you have a wider range of things to support _and_ less revenue per device to support it, and you are likely to have to coordinate improvements with other companies who may have different priorities. Apple can profile things which their users do and direct attention to the right team. A company like Intel might profile something and see that they can make some changes to the CPU but the biggest gains would require work by a system vendor, a compiler improvement, Windows/Linux kernel change, etc. — they contribute a large amount of code to many open source projects but even that takes time to ship and be used.
  - astrange 6y ago
    Intel does lots of contributions across the OS (Linux and glibc) to compilers including their own (gcc, icc, ispc, etc). Their problems aren't their ability, it's that Intel is poorly managed and internal groups are constantly fighting with each other. Also, compiler support for CPUs is very overrated. Heavy compiler investment was attempted with Itanium and debunked; giant OoO CPUs like Intel's or M1 barely care about code quality, and the compilers have very little tuning for individual models.
    - acdha 6y ago
      > Intel does lots of contributions across the OS (Linux and glibc) to compilers including their own (gcc, icc, ispc, etc). Their problems aren't their ability, it's that Intel is poorly managed and internal groups are constantly fighting with each other. I wasn't just talking about Intel but the concept of separate CPU and compiler vendors in general. Intel contributes a ton of open source but even if they were perfectly organized it takes time for everything to happen on different schedules before it's generally available: get patches into something like Linux or gcc, wait possibly years for Red Hat to ship a release using the new version, etc. Certain users — e.g. game or scientific developers — might jump on a new compiler or feature faster, of course, but that's far from a given and it means they're not going to get the across-the-board excellent scores that Apple is showing. > Also, compiler support for CPUs is very overrated. Heavy compiler investment was attempted with Itanium and debunked; giant OoO CPUs like Intel's or M1 barely care about code quality, and the compilers have very little tuning for individual models. This isn't entirely wrong but it's definitely not complete. Itanium failed because brilliant compilers didn't exist and it was barely faster even with hand-tuned code, especially when you adjusted for cost, but that doesn't mean that it doesn't matter at all. I've definitely seen significant improvements caused by CPU family-specific tuning and, more importantly, when new features are added (e.g. SIMD, dedicated crypto instructions, etc.) a compiler or library which knows how to use those can see huge improvements on specific benchmarks. That was more what I had in mind since those are a great example of where Apple's integration shines: when they have a task like “Make H.265 video cheap on a phone” or “Use ML to analyze a video stream” they can profile the whole stack, decide where it makes sense to add hardware acceleration, and then update their choice of the compiler toolchain and higher-level libraries (e.g. Accelerate.framework) and ship the entire thing at the time of their choosing whereas AMD/Intel/Qualcomm and maybe nVidia have to get Microsoft/Linux and maybe someone like Adobe on board to get the same thing done. That isn't a certain win — Apple can't work on everything at once and they certainly make mistakes — but it's hard to match unless they do screw up.
- deleted 6y ago
  [deleted]
- mhh__ 6y ago
  I think it's worth saying that because AMD have only just really hit their stride, Intel were under almost zero pressure to improve which has really hurt them especially with the process. X86 is definitely a coefficient overhead, but if Intel put their designs on 5nm they'd look pretty good too - Jim Keller (when he was still there) hinted their offerings for a year or so in the future are significantly bigger to the point of him judging it to be worth mentioning so I wouldn't write them off.
  - megap1x3ls 6y ago
    Do you know what 5nm is?
- baybal2 6y ago
  > There is no "magic" at play here, it's just lots and lots of raw muscle. They just seem to have a freakishly successful strategy for choosing what aspects of the processor to throw that muscle at. There is no freakishly successful strategy at play there as well. It's just all previous attempts at "fast ARM" chip were rather half hearted "add a pipeline step there, add extra register there, increase datapath width there," and not to squeeze it to the limit.
- TYPE_FASTER 6y ago
  Apple has been iterating on their proprietary mobile ARM-based processors since 2010, and has gotten really good at it. I would imagine that producing billions of consumer devices with these chips has helped give them a lot of experience in shortened time frame. I also wonder if having the hardware and software both worked on in-house is an advantage. I mean, if you're developing power management software for a mobile OS, and you're using a 3rd-party vendor, then you read the documentation, and work with the vendor if you have questions. If it's all internal, you call them, and could make suggestions on future processor design too based on OS usage statistics and metrics.
  - simonh 6y ago
    In fact there is clear evidence of this with M1. It has optimised instructions to speed up retain/release of NSObject subclasses, which is a frequent operation on almost all Objective-C and Swift classes. They also designed the M1 to support a memory management profile used by x86 (and not ARM) to accelerate Rosetta translated binaries. I'm sure there are more.
- gavin_gee 6y ago
  didnt they also make some interesting hires a few years ago like Anand from Anandtech and some other silicon vets that likely helped them design the M1 approach?
- barkingcat 6y ago
  The answer is that they have raw hard numbers from the hundres of millions of iPads/iPhones sold each year, and can use the metrics from those devices to optimize the next generation of devices. These improvements didn't come from nowhere. It came from iterations of iOS hardware.
- coldtea 6y ago
  >Why is that strategy simultaneously remarkably efficient and remarkably high-performance? What enabled/led them to make those choices where others haven't? The things people give them complains about: (a) keeping a walled garden, (b) moving fast and taking the platform to new directions all at once (c) controlling the whole stack Which means they're not beholden to compatibility with third party frameworks and big players, or with their own past, and thus can rely on their APIs, third party devs etc, to cater to their changes to the architecture. And they're not chained to the whims of the CPU vendor (as the OS vendor) or the OS vendor (as the CPU vendor) either, as they serve the role of both. And of course they benchmarked and profiled the hell out of actual systems.
  - fartcannon 6y ago
    They could still do all this shit without the walled garden. To me, it suggests they aren't willing to compete. They're anti-competitive.
    - marrvelous 6y ago
      With the walled garden, Apple can set enforceable timelines for the software ecosystem to adopt to architectural changes. Remember the transition to arm64? Apple forced everything on the App Store to ship universal binaries. Without the App Store walled garden, software isn’t required to keep up to date with architectural changes. Instead, keeping current is only a requirement to being featured on the App Store (which would just be a single way to install software, not the only method).
      - danaris 6y ago
        Well, and on the Mac, it's not the only method. The walled garden here has big open gates. That said, all software on the Mac, post-Catalina, has to be 64-bit, whether it's distributed through the Mac App Store or not, because the 32-bit system libraries are no longer included at all.
        coldtea 6y ago
- adam_arthur 6y ago
  Certainly Apple's processors are far ahead, but they're a full process generation (5nm) ahead of their competitors. They paid their way to that exclusive right through TSMC. I'm sure they'll still come out ahead in benchmarks, but the numbers will be much closer once AMD moves to 5nm. You absolutely cannot fairly compare chips from different fab generations. I don't see many comments hammering this point home enough... it's not like the performance gap is through engineering efforts that are leagues ahead. Certainly some can be attributed to that, and Apple has the resources to poach any talent necessary.
  - wmf 6y ago
    From a customer's perspective it's not my problem. Everyone had the opportunity to bid on that fab capacity and they decided not to.
    - adam_arthur 6y ago
      Yeah, totally agreed. But if you read these comments, they seem to be in total amazement about the performance gap and not acknowledging how much of an advantage being a fab generation ahead is. Customers don't care, but discussion of the merits of the chip should be more nuanced about this. It also implies that the gap won't exist for very long, as AMD will move onto 5nm soon
      - tandr 6y ago
        > It also implies that the gap won't exist for very long, as AMD will move onto 5nm soon ... yes, if there is any capacity left. Capacity for the new process is a limited resource after all.
      - jack_pp 6y ago
        People keep pointing this out but has Intel had such significant performance improvements since sandy bridge? With x86 it seems that lately you would be foolish to upgrade less than once every 3-4 years because the difference is just not that significant
- jandrese 6y ago
  It seems like Apple listened when people talked about how all modern processors bottleneck on memory access and decided to focus heavily on getting those numbers better. Of course this leads to the question that if everyone in the industry knew this was the issue why weren't Intel and AMD pushing harder on it? They already both moved the memory controller onboard so they had the opportunity to aggressively optimize it like Apple has done, but instead we have year after year where the memory lags behind the processor in speed improvements, to the point where it is ridiculous how many clock cycles a main memory access takes on a modern x86 chip.
  - proverbialbunny 6y ago
    My guess is it had to do with limitations tied to the x86_64 instruction set. It doesn't matter how much modifications you do, if you don't start with a good foundation, you're going to be limited to that foundation.
    - nkurz 6y ago
      I think the current consensus among experts is that the instruction set is not the limiting factor. Modern x64 microprocessors have a separate front-end that handles instruction decoding. These instructions are decoded to internal proprietary "micro-ops". The internal buffers and actual execution units see only these µops. One can measure where the bottlenecks are, and it's rare to find that the front-end is the bottleneck. While it's arguably true that x64 is a poorly designed "foundation", it's unlikely to be causing any performance difference here.
      - wtallis 6y ago
        > One can measure where the bottlenecks are, and it's rare to find that the front-end is the bottleneck. Part of this is due to the fact that x86 processor designers won't include more execution units than they can feed from their instruction decoders. Apple's processors are much wider than x86 on both the decode and execution resources, and it's pretty clear that the M1 would not perform as well if its decoders were as narrow as current x86 cores.
- SurfingInVR 6y ago
  Something I've seen no one else mentioning: Apple's low-spec tier is $1000, not $70.
  - square_usual 6y ago
    It's $699, for a complete device, not a part of one.
    - perryizgr8 6y ago
      What's the screen resolution on that $699 "complete device"?
      - FreshFries 6y ago
        5120 by 2880
      - square_usual 6y ago
        Comparable to a $70 processor, at least.
- dv_dt 6y ago
  No fighting the sales department on where to put the market segmentation bottlenecks?
- smoldesu 6y ago
  The big "enabler" was their mass-purchase of 5nm lithography across the board. Even still though, 4ghz*8c isn't anything new, and isn't really that remarkable besides the low TDP (which is incidentally dwarfed by the display, which draws up to 5x more power than the CPU does). I think the big issue is that Apple has painted themselves into a corner here: ARM won't play nice with the larger CPUs they want to make, and the pressure for them to provide a competent graphics solution on custom silicon is mounting. They spent a lot of time this generation marketing their "energy efficiency" and battery life, but many consumers/professionals (myself included) don't really care about either of these things.
  - krunkcoin 6y ago
    "the display, which draws up to 5x more power than the CPU does" - wat? Apple-supplied monitoring tools report that M1 under full load (all CPU and GPU cores) can draw over 30 watts. Laptop displays don't consume 150 watts. If anything, the displays in either of the M1 portables likely consume about 5 times less than 30W, even at full brightness. "ARM won't play nice with the larger CPUs they want to make" - wat? Apple holds an architectural license. This means they paid a lot upfront a long time ago and therefore have a more or less perpetual right to design their own Arm cores without input from Arm. "a competent graphics solution" - Also wat? M1 has an excellent GPU. It doesn't compete with discrete GPUs that use 300 watts, but that's fine: M1 is the chip for entry level Macs, designed for the smallest and lightest segments of their notebook line. And in that product segment, it has been every bit as much a revelation as the CPU. It's very fast, and uses little power given the performance. What exactly do you think is going to happen when they scale that basic GPU design up? Despite your dismissiveness, in modern silicon architecture energy efficiency is incredibly important: for any given power budget, the more efficient you are the more performance you can deliver. The performance Apple gets out of about 10W on M1 suggests they'll have few problems building a larger GPU to compete with Nvidia and AMD discrete GPUs.
ed25519FUUU 6y ago
In other words, it’s better architecture. If anything this makes it seem more impressive to me.
- amelius 6y ago
  No, it's the same architecture but with different parameters. It's like the difference between the situation where every car uses 4 cylinders, and then Apple comes along and makes a car with 5 cylinders.
  - kllrnohj 6y ago
    Your analogy was so close! It's Apple comes along and makes an 8 cylinder engine. Since, you know, the other CPUs are 4-wide decode and Apple's M1 is 8-wide decode :)
    - ben-schaaf 6y ago
      Zen 3 is effectively 8 wide with their micro-op cache. Intel similarly has been 6 wide for ages.
gameswithgo 6y ago
What other laptop ships with LPDDR4X clocked at 4267? I agree though that being closer to the cpu isn't having any appreciable effect on latency, but being soldered close to the cpu probably does make it easier for them to hit that high clock rate.
- wmf 6y ago
  Tiger Lake laptops such as the XPS 13.
- jeffbee 6y ago
  As WMF mentions, Tiger Lake laptops like my Razer Book have the same memory. It is not appreciably closer to the CPU in the Apple design. In Intel's Tiger Lake reference designs the memory is also in two chips that are mounted right next to the CPU.
  - danaris 6y ago
    And (genuine question) how do the Tiger Lake laptops compare with the M1 MacBooks thus far?
    - skavi 6y ago
      AnandTech has decent benchmarks for both Tiger Lake [0] and M1 [1]. [0]: https://www.anandtech.com/show/16084/intel-tiger-lake-review-deep-dive-core-11th-gen https://www.anandtech.com/show/16084/intel-tiger-lake-review... [1]: https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste...
    - jeffbee 6y ago
      The outcome seems to depend greatly on the physical design of the laptops. The elsewhere-mentioned Dell XPS 13 has a particularly poor cooling design, which is why I chose the Razer Book instead. Despite being marketed in a very silly way to gamers only, it seems to have competent mechanical design.
titzer 6y ago
> it can do with a single core what an Intel part needs all cores to accomplish. Care to explain what you mean specifically by this?
- saagarjha 6y ago
  The M1 has extremely high single-core performance.
  - temac 6y ago
    It is not 4 times faster than an Intel core, though...
    - saagarjha 6y ago
      It is in memory performance, which is what I assumed was being measured here.
      - kllrnohj 6y ago
        How are you defining memory performance and where are your supporting comparisons? This article only discusses the M1's behavior, and makes no comparisons to any other CPU.
        FabHK 6y ago
        FWIW, I ran it on a MacBook Pro (13-inch, 2019, Four Thunderbolt 3 ports), 2.4 GHz Quad-Core Intel Core i5, 8 GB 2133 MHz LPDDR3: two : 49.6 ns (x 5.5) two+ : 64.8 ns (x 5.2) three: 72.8 ns (x 5.6) EDIT to add: above was just `cc`. Below is with `cc -O3 -Wall`, as in Lemire's article: two : 62.8 ns (x 7.1) two+ : 69.2 ns (x 5.5) three: 95.3 ns (x 7.3)
        namibj 6y ago
        You _need_ to use -mnative because it otherwise retains backwards compatibility to older x86.
ksec 6y ago
>HN memes about M1 memory will die It is not only HN. It is practically the whole Internet. Go around the Top 20 hardware and Apple website forum and you see the same thing, also vastly amplify by a few KOL on twitter. I dont remember I have ever seen anything quite like it in tech circle. People were happily running around spreading misinformation.
- jeffbee 6y ago
  Yeah, I know. There was some kid on Twitter who was trying to tell me that it was the solder in an x86 machine (he actually said "a Microsoft computer") that made them slower. Apple, without the solder was much faster. According to this person's bio they had an undergraduate education in computer science ¯\_(ツ)_/¯
- Bootvis 6y ago
  What is a KOL?
  - tyingq 6y ago
    "Key Opinion Leader". I think it's the new word for "Influencer".
    - walterbell 6y ago
      Who introduced the term KOL and bestows the K title?
    - ksec 6y ago
      I am pretty sure KOL predates Influencer in modern internet usage. Before that they were simply known as Internet Celebrities. May be it is rarely used now. So apology for not explaining the acronyms.
      - secondcoming 6y ago
        First I've heard of it!
sliken 6y ago
The most impressive thing I've seen is that when accessed in a TLB friendly fashion that the latency is around 30ns. Anandtech has a graph showing this, specifically the R per RV prange graph. I've verified this personally with a small microbenchmark I wrote. I've not seen anything else close to this memory latency.
- tandr 6y ago
  Sorry, what would AMD's or Intel's "latest and greatest" numbers for the same be?
  - sliken 6y ago
    Here's the M1: https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... Scroll down to the latency vs size map and look at the R per RV prange. That gets you 30ns or so. Similar for AMD's latest/greatest the Ryzen 9 5950X: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-dive-review-5950x-5900x-5800x-and-5700x-tested/5 https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di... The same R per RV prange is in the 60ns range.
    - epistasis 6y ago
      Could this be coming from the page size being 4x as large for Apple Silicon versus x86? I don't fully understand the benchmark, but it appears to be accessing a variety of pages from the same first level TLB lookup? It's been a long time since I dealt with this stuff (wanted to get 1GB huge pages in Linux for some huge huge hash tables), so maybe I'm misunderstanding.
      - sliken 6y ago
        Cachelines, page sizes, and size of the TLB all play a role. But with tinkering you can see those effects yourself and I played with 1, 2, 4, 8, 16, and 32 "pages" which I assumed were 4KB each and didn't see much difference. Measured latencies do increase slowly, but you expect that as the TLB becomes progressively more of a bottleneck. If you use a 1GB array and see full random with much higher latency than a sliding window then you can be pretty sure that the page size is much less than 1GB. Getting the cacheline off by a factor of 2 does make a small difference since you get occasional cache hits instead of zero, but as long as the array tested is several times larger than cache the impact is small. But all in all the M1 has excellent memory bandwidth, excellent latency, and shows significantly better throughput on random workloads as you use more cores. Normal PC desktops have 2 memory channels (even the higher end i7/i9/ryzen7/ryzen9), only the $$$$ workstation chips like threadripper and some of the $$$$ Intel's have more. The little ole M1 in a mac mini, starting at $700 has at least 8 memory channels. So basically the M1 delivers on all fronts, larger and lower latency caches, wide issue, large reorder buffers, excellent IPC, and excellent power efficiency.
megablast 6y ago
You’ll get people guessing, since Apple itself puts out so little information.
PragmaticPulp 6y ago
I don't understand this competition to attribute the M1's speed to one specific change, while downplaying all of the others. M1 is fast because they optimized everything across the board. The speed is the cumulative result of many optimizations, from the on-die memory to the memory clock speed to the architecture.
- adam_arthur 6y ago
  It's fast because they optimized everything across the board, and also paid for exclusive access to TSMC 5nm process.
foldr 6y ago
>or that it's faster because the memory is 2mm closer to the CPU (not that either) Not to disagree with your overall point, but 2mm is a long way when dealing with high frequency signals. You can't just eyeball this and infer that it makes no difference to performance or power consumption.
- jeffbee 6y ago
  If it works, it works. There will be no observable performance difference for DDR4 SDRAM implementations with the same timing parameters, regardless of the trace length. There are systems out there with 15cm of traces between the memory controller pins and the DRAM chips. The only thing you can say against them is they might consume more power driving that trace. But you wouldn't say they are meaningfully slower.
  - foldr 6y ago
    You can't just eyeball the PCB layout for a GHz frequency circuit and say "yeah that would definitely work just the same if you moved that component 2mm in this direction". It's certainly possible to use longer trace lengths, but that may come with tradeoffs. >The only thing you can say against them is they might consume more power driving that trace Power consumption is really important in a laptop, and Apple clearly care deeply about minimising it. For all we know for sure, moving the memory closer to the CPU may have been part of what's enabled Apple to run higher frequency memory with acceptable (to them) power draw.
titzer 6y ago
Frankly, I find Lemire does oversimplified, poor-quality control, back-of-the-envelope microbenchmarking all the time that provides little to no insight other than establishing a general trend. It's sophomoric and a poor demonstration about how to well-controlled benchmarking that might yield useful, repeatable, and transferrable results.
- alecco 6y ago
  Can you give an example? I've seen Lemire correct his posts on many occasions and the source code is published. I don't know many blogs doing anything remotely like that.
  - titzer 6y ago
    Sure. He often benchmark some small C++ code on his "laptop" CPU (which one exactly? microarchs matter!) and then committing classic microbenchmarking pitfalls such as: - benchmarking something small enough to inspect machine code, but not inspecting machine code - not plotting distribution, average, variance etc - no attention paid to CPU frequency governor settings - measuring too short a run - measuring too small a dataset that it fits entirely in L1
RachelF 6y ago
>that it's "on-die" (it's not) It appears to be mounted on the same chip package. Why did Apple do this if not for speed?
- wmf 6y ago
  On-package memory is not faster. I suspect it is more power efficient though.
  - rkangel 6y ago
    It makes it easier to get to a particular clock speed. The geometry, interconnect lengths etc are all tightly controlled, the noise is less because you're not on the main PCB and you have interconnect options that aren't whatever your PCB process is (e.g. commonly gold wires).
deleted 6y ago
[deleted]

s800 6y ago

What's the precision of these ns level measurements?

mhh__ 6y ago
The answer to that is usually very context dependant, and on what you're measuring. As long as you use a histogram first and don't blindly calculate (say) the mean it should he obvious. Two examples( that are slightly bigger than this but the same principles apply): If you benchmark a std::vector at insertion, you'll see a flat graph with n tall spikes at ratios of it's reallocation amount apart, and it scales very very well. The measurements are clean. If, however, you do the same for a linked list you get a linearly increasing graph but it's absolutely all over the place because it doesn't play nice with the memory hierarchy. The std_dev of a given value of n might be a hundred times worse than the vector.
CyberRabbi 6y ago
Clock_gettime(CLOCK_REALTIME) on macos provides nanosecond-level precision.
- geocar 6y ago
  I seem to recall OSX didn't used to have clock_gettime, so it's news to me that it even exists -- I might have been away from OSX too long. Is there any performance difference between that and mach_absolute_time() ?
  - saagarjha 6y ago
    It's new in macOS Sierra. I believe mach_absolute_time is slightly faster but not by much–both just read the commpage these days to save on a syscall.
  - lilyball 6y ago
    It was added some years ago, and I believe mach_absolute_time is actually now implemented in terms of (the implementation of) clock_gettime. The documentation on mach_absolute_time now even says you should use clock_gettime_nsec_np(CLOCK_UPTIME_RAW) instead. macOS also has clock constants for a monotonic clock that increases while sleeping (unlike CLOCK_UPTIME_RAW and mach_absolute_time).

kergonath 6y ago

It’s a good introduction, but it’s a bit disappointing that it ends that way. I’d love to read more about what’s behind the figure and more technical info about how it might work.

alblue 6y ago
This isn’t specific to the M1 but I tap about cache lines in my last QCon presentation (where I also suggested that a 128b cache line wasn’t far away): https://www.infoq.com/presentations/microarchitecture-modern-cpu/ https://www.infoq.com/presentations/microarchitecture-modern... However the speed benefits come from a much larger L1 cache and the fact that the ram is in the same chip which will reduce latency that is the benefit for most of it. The program (instruction) cache is also a lot bigger and has the advantage that as a fixed size isa can be much wider in execution than in x86 but that’s unlikely to be of benefit here, other than perhaps slightly in terms of queuing up multiple outstanding loads.
- kristianp 6y ago
  This post says that the m1 has a 128 byte cache line size. So that time has arrived! https://news.ycombinator.com/item?id=25660769 https://news.ycombinator.com/item?id=25660769
  - alblue 6y ago
    Yup, my presentation was back in March 2020 and the M1 came out later in the year — sooner than I was expecting, TBH; I thought that it was a couple of years out when I said it :-)
- kergonath 6y ago
  Thanks!

lrossi 6y ago

Shouldn’t you choose the random numbers such that array[idx1] ^ array[idx1 + 1] are guaranteed to fall in the same cache line? Assuming that it has that. Right now some accesses cross the end of the cache line.

CyberRabbi 6y ago
Technically you are correct but it’s expected to cross a cache line 1/16 times (or however many ints there are in a cache line). There is an implicit assumption that that is relatively infrequent enough that it shouldn’t increase the average time too much, but that assumption should be tested.

nabla9 6y ago

What is the cache line size and page table size in M1? sysconf(_SC_PAGESIZE); /* posix */ Can you get direct processor information like LEVEL1_ICACHE_ASSOC and LEVEL1_ICACHE_LINESIZE from the M1??

momothereal 6y ago
`getconf PAGESIZE` returns 16384 on the base M1 MacBook Air. The L1 cache values aren't there. The macOS `getconf` doesn't support -a (listing all variables), so they may just be under a different name. edit: see replies for `sysctl -a` output
- lilyball 6y ago
  Is it possibly exposed via sysctl, which does support a flag to list all variables?
  - messe 6y ago
    From sysctl -a on my M1: hw.cachelinesize: 128 hw.l1icachesize: 131072 hw.l1dcachesize: 65536 hw.l2cachesize: 4194304 EDIT: also, when run under Rosetta hw.cachelinesize is halved: hw.cachelinesize: 64 hw.l1icachesize: 131072 hw.l1dcachesize: 65536 hw.l2cachesize: 4194304
    - Nokinside 6y ago
      M1 cache lines are double of what is commonly used by Intel, AMD and other ARM microarchtectures use. That's significant difference.
      - deleted 6y ago
        [deleted]
    - JonathonW 6y ago
      Compared to the i9-9880H in my 16" MacBook Pro: hw.cachelinesize: 64 hw.l1icachesize: 32768 hw.l1dcachesize: 32768 hw.l2cachesize: 262144 hw.l3cachesize: 16777216 The M1 doubles the line size, doubles the L1 data cache (i.e. same number of lines), quadruples the L1 instruction cache (i.e. double the lines), and has a 16x larger L2 cache, but no L3 cache.

waterside81 6y ago

For people who know more about this stuff than me: are these sorts optimizations only possible because Apple controls the whole stack and can make the hardware & OS/software perfectly match up with one another or is this something that Intel can do but doesn't for some reasons (tradeoffs)?

deleted 6y ago
[deleted]
wmf 6y ago
No, there's no cross-stack optimization here. The M1 gives very high performance for all code.
- qeternity 6y ago
  I think this gets lost in the fray between the "omg this is magic" and then the Apple haters. The M1 is a very good chip. Apple has hired an amazing team and resourced them well. But from a pure hardware perspective, the M1 is quite evolutionary. However the whole Apple Silicon experience is revolutionary and magical due to the tight software pairing. Both teams deserve huge praise for the tight coordination and unreal execution.
  - acdha 6y ago
    I think this is part of the reason where there are so many people trying to find reasons to downplay it: humans love the idea of “one weird trick” which makes a huge difference and we sometimes find those in tech but rarely for mature fields like CPU design. For many people, this is unsatisfying like asking an athlete their secret, and getting a response like “eat well, train a lot, don't give up” with nary a shortcut in sight.
AnthonyMouse 6y ago
> are these sorts optimizations only possible because Apple controls the whole stack and can make the hardware & OS/software perfectly match up with one another or is this something that Intel can do but doesn't for some reasons (tradeoffs)? Interestingly it's the other way around. Apple is using TSMC's 5nm process (they don't have their own fabs), which is better than Intel's in-house fabs, so it's Intel's vertical integration which is hurting them compared to the non-vertically integrated Apple. Also, the answer to "is this only possible because of vertical integration" is always no. Intel and Microsoft regularly coordinate to make hardware and software work together. Intel is one of the largest contributors to the Linux kernel, even though they don't "own" it. Two companies coordinating with one another can do anything they could do as an individual company. Sometimes the efficiency of this is lower because there are communication barriers and isn't a single chain of command. But sometimes it's higher because you don't have internal politics screwing everything up when the designers would be happy with outsourcing to TSMC because they have a competitive advantage, but the common CEO knows that would enrich a competitor and trash their internal investment in their own fabs, and forces the decision that leads to less competitive products.

djacobs7 6y ago

Is the article saying that the M1 is slower than we would have expected in this case? My understanding, based on the article, is that a normal processor, we would have expected arr[idx] + arr[idx+1] and arr[idx] to take the same amount of time. But the M1 is so parallelized that it goes to grab both arr[idx] and arr[idx+1] separately. So we have to wait for both of those two return. Meanwhile, on a less parallelized processor, we would have done arr[idx] first and waited for it to return, and the processor would realize that it already had arr[idx+1] without having to do the second fetch. Am I understanding this right?

phkahler 6y ago
>> My understanding, based on the article, is that a normal processor, we would have expected arr[idx] + arr[idx+1] and arr[idx] to take the same amount of time. That depends. If the two accesses are on the same cache line, then yes. But since idx is random that will not happen sometimes. He never says how big array[] is in elements or what size each element is. I thought DRAM also had the ability to stream out consecutive addresses. If so then it looks like Apple could be missing out here. Then again, if his array fits in cache he's just measuring instruction counts. His random indexes need to cover that whole range too. There's not enough info to figure out what's going on.
- SekstiNi 6y ago
  > There's not enough info to figure out what's going on. If you only look at the article this is true. However, the source code is freely available: https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2021/01/06 https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/...
  - deleted 6y ago
    [deleted]
  - africanboy 6y ago
    I ran the benchmark on my system It's a 6 years old system, fastest times are in the 25ns range - 2-wise+ is 5% slower than 2-wise - 3-wise is 46% slower than 2-wise - 3-wise is 39% slower than 2-wise+ on the M1 - 2-wise+ is 40% slower than 2-wise - 3-wise is 46% slower than 2-wise - 3-wise is 4% slower than 2-wise+

jayd16 6y ago

>our naive random-memory model Doesn't everyone use the (I believe) still valid concepts of latency and bandwidth?

wyldfire 6y ago
How do latency and bandwidth relate to the cost model for the code in the benchmark? When creating the model discussed in the post, we're using it to try to make a static prediction about how the code will execute. Note that the goal of the post is not to merely measure the memory access performance, it's to understand the specific microarchitecture and how it might deliver the benefits that we see in benchmarks.
whoisburbansky 6y ago
The concepts are still broadly valid, the naivety being referred to is the assumption that two non adjacent memory reads will be twice as slow as one memory read or two adjacent reads.
sroussey 6y ago
Depends on context. For example, what is the bandwidth and latency when you ask for the value at the same memory address in an infinite loop? And how does that compare to the latency and bandwidth of a memory module you buy on NewEgg?
- fluffy87 6y ago
  L1 BW. When people use BW in their performance models, they don’t use only 1 bandwidth, but whatever combination of bandwidth makes sense for the _memory access pattern_. So if you are always accessing the same word, the first acces runs at DRAM BW, and subsequent ones at L1 BW, and any meaningful performance model will take that into account.

foota 6y ago

Is this per core or shared between cores?

hundchenkatze 6y ago
Per core I think, emphasis is mine. > It looks like a single core has about 28 levels of memory parallelism, and possibly more.
- foota 6y ago
  I was wondering if this might be a shared resource though, since it doesn't seem they tested with multiple threads.

wrsh07 6y ago

Ok, summary: This article lays out three scenarios: 1) accessing two random elements 2) accessing 3 random elements 3) accessing two pairs of adjacent elements (same as (1) but also the elements after each random element) It then does some trivial math to use the loaded data. A naive model might only consider memory accesses and might assume accessing an adjacent element is free. On the Mac m1 core, this is not the case. While the naive model might expect cases 1 & 3 to cost the same and case 2 to cost 50% more, instead cases 2 & 3 are nearly the same (3 slightly faster) and case 2 is about 50% more expensive than 1.

jayd16 6y ago
I don't really understand the comparison because it seems like scenario 3 (2+) is doing more XORs and twice the accesses to array over the same amount of iterations. We have to assume these are byte arrays, yes? Or at least some size that's smaller than the cache line. You would still pay for the extra unaligned fetches. I don't think this is a valid scenario at all, M1 or not. Anyone want to run these tests on an Intel machine and let us know if the authors "naive model" test hold there?
- africanboy 6y ago
  I did it on an old i7 laptop https://news.ycombinator.com/item?id=25661055 https://news.ycombinator.com/item?id=25661055
- wrsh07 6y ago
  The point of the naive model is that you assume memory accesses dominate That is, the math part is so trivial compared to the memory access that you could do a bunch of math and you would still only notice a change in the number of memory accesses. Also it looks like the response to yours links their test and the naive model predicts correctly
  - jayd16 6y ago
    I think 5% is a non-trivial difference but alright, its a much bigger difference on the M1. I guess I still don't understand whats going on here. Scenario 1 has two spatially close reads followed by two dependent random access reads. Scenario 3 (2+) has two spatially close reads, and two pairs of dependent random access reads of two spatially close locations. Why does it follow that this is caused by a change in memory access concurrency? The two required round trips should dominate both on the M1 and an Intel but for some reason the M1 performs worse than that. Why? I can't help but feel the first snippet triggers some SIMD path while the 3rd snippet fails to.

syntaxing 6y ago

I’m super curious if it’s true that my 8GB M1 will die quickly because of the aggressive swaps. I guess time will tell.

acdha 6y ago
FWIW, I have a 2010 MBA which was _heavily_ used for years as a primary development system. The SSD only started to show signs of degraded performance last year and that wasn't massive. I would be quite surprised if the technology has become worse.

deleted 6y ago

[deleted]

willvarfar 6y ago

A lot of commenters here are saying that Apples advantage is that it can profile the real workloads and optimise for that. Well that’s true and could very well be an advantage. An advantage in that they did it, not in that only they have access to it. Intel and AMD can trivially profile real world workloads too. Did they? I don’t know what Apple did, but the impression I get is that intel certainly hasn’t.

gameswithgo 6y ago
of course every cpu designer is using real world workloads to guide design

helloycombinat 6y ago

I don't have a M1 chip but it'd be interesting to see what happens if we make sure if the random value is never n*CACHE_SIZE-1

fctorial 6y ago

I can't find any info about the memory bus of apple m1. Is it 8 channels 16 bit each? That's drastically different from AMDs 2 channels 64 bit each. It looks like apple m1 is much less eager when caching memory rows. Maybe because it doesn't have l3 cache. Edit: This test utilizes the 8x16bit memory bus of apple m1 fully. It's mostly just fetching random locations from memory, which can all be parallelized by the cpu pipeline. It explains why the results are exactly 4x slower on my ryzen 3 with 2 memory channels. So the summary is that m1 is optimized for dynamic languages that tend to do a DDOS attack on RAM with a lot of random memory access, but it might take a performance hit with compiled languages and traditional HPC techniques that tend to process data in sequence like ECS.

skohan 6y ago
That's an interesting observation - as someone who's built a few ECS implementations, one of the things I've always taken for granted is that things like cache line size are more or less set in stone, given the ubiquity of x86, so it's interesting to consider that the rise of ARM might create additional complexities there. I'm a bit of two minds about this: on the one hand, for a long time I've wanted a language for writing allocators which is more explicit about memory, and offers good abstractions for low-level memory operations (maybe Zig is going in this direction). In some sense, it feels like the move towards programmers thinking less about memory management has been a bit of a dead-end, and what we really want is better tools for memory management. Fragmentation in terms of how processors handle memory goes against this goal in some ways. On the other hand, it's a bit of a "holy grail" to imagine a hardware stack which obviates the need for memory optimization, and really does treat loading from and storing to memory anywhere on the heap as the same. But I imagine that the interesting things which the M1 is doing with memory are helping a lot with the worst case performance, and maybe even average case performance, but they're probably not doing much for the best case.
Dylan16807 6y ago
That would make sense for LPDDR4 but it apparently claims to have a 128 byte cache line size and I'm not sure how to square that with 16 bit channel width.

gokulkrishh09 6y ago

Interesting article.

lincpa 6y ago

Apple M1 chip adopts Warehouse/Workshop Model - Warehouse: unified memory - Workshop: CPU, GPU and other cores - Product( material): information,data there’s also a new unified memory architecture that lets the CPU, GPU, and other cores exchange information between one another, and with unified memory, the CPU and GPU can access memory simultaneously rather than copying data between one area and another. Accessing the same pool of memory without the need for copying speeds up information exchange for faster overall performance. reference: 1. Developer Delves Into Reasons Why Apple’s M1 Chip is So Fast. https://www.macrumors.com/2020/11/30/m1-chip-speed-explanation-developer/ https://www.macrumors.com/2020/11/30/m1-chip-speed-explanati... 2. The Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Warehouse/Workshop Model https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow