41 ms·
This is fascinating: > Retain and release are tiny actions that almost all software, on all Apple platforms, does all the time. ….. The Apple Silicon system ar
by brundolf 6y ago
This is fascinating:
> Retain and release are tiny actions that almost all software, on all Apple platforms, does all the time. ….. The Apple Silicon system architecture is designed to make these operations as fast as possible. It’s not so much that Intel’s x86 architecture is a bad fit for Apple’s software frameworks, as that Apple Silicon is designed to be a bespoke fit for it …. retaining and releasing NSObjects is so common on MacOS (and iOS), that making it 5 times faster on Apple Silicon than on Intel has profound implications on everything from performance to battery life.
> Broadly speaking, this is a significant reason why M1 Macs are more efficient with less RAM than Intel Macs. This, in a nutshell, helps explain why iPhones run rings around even flagship Android phones, even though iPhones have significantly less RAM. iOS software uses reference counting for memory management, running on silicon optimized to make reference counting as efficient as possible; Android software uses garbage collection for memory management, a technique that requires more RAM to achieve equivalent performance.
- lisper 6y agoRemember Lisp machines? The M1 is a Swift machine.
- brundolf 6y agoI'm wondering if the "optimized for reference-counting" thing applies to other languages too. i.e. if I write a piece of software in Rust, and I make use of Rc<>, will Macs be extra tolerant of that overhead? In theory it seems like the answer should be yes
- lilyball 6y agoI sure hope so. In macOS 10.15, the fast path for a retain on a (non-tagged-pointer) Obj-C object does a C11 relaxed atomic load followed by a C11 relaxed compare-and-exchange. This seems pretty standard for retain-release and I'd expect Rust's Rc<> to be doing something similar. It's possible Apple added some other black magic to the runtime in 10.16 (and they haven't released the 10.16 objc sources yet) but it's hard to imagine what they'd do that makes more sense than just optimizing for relaxed atomic operations.
- steveklabnik 6y ago(Rc does not use atomics, Arc does and from a peek at the source it does indeed use Relaxed)
- wahern 6y agoI didn't understand why the implementation wouldn't just do an atomic increment, but I guess Obj-C semantics provide too much magic to permit such a simple approach. The actual code, in addition to [presumably] not being inlined, does not seem easy to optimize at the hardware level: https://github.com/apple/swift-corelibs-foundation/blob/main/CoreFoundation/Base.subproj/CFRuntime.c#L1365 https://github.com/apple/swift-corelibs-foundation/blob/main... The native Swift retain (swift_retain above) seems to be somewhere inside this mess: https://github.com/apple/swift/blob/main/stdlib/public/runtime/HeapObject.cpp https://github.com/apple/swift/blob/main/stdlib/public/runti...
- lilyball 6y agoWhat you’ve linked is CoreFoundation’s retain, Objective-C’s can be found in https://opensource.apple.com/source/objc4/objc4-787.1/runtime/objc-object.h.auto.html https://opensource.apple.com/source/objc4/objc4-787.1/runtim... (look for objc_object::rootRetain). The short answer for why it can’t just be an increment is because the reference count is stored in a subset of the bits of the isa pointer, and when the reference count grows too large it has to overflow into a separate sidetable. So it does separate load and CAS operations in order to implement this overflow behavior.
- saagarjha 6y agoI’d think the bigger win would have to be in the release part of the code, which actually cares about contention.
- plorkyeran 6y agoYes, it applies to everything which uses atomics and is not something special in the runtime. It's also worth noting that this is an optimization that iphones have had for the last few years ever since those switched to arm64.
- saagarjha 6y agoThe fast atomics are relatively new.
- favorited 6y agoNo, because Rc<> isn't atomic. Arc<>, however, would get the benefit. The reason retain/release in ObjC/Swift are so much faster here is because they are atomic operations.
- amelius 6y agoSo, useful for structural sharing between threads.
- klelatti 6y agoBut how? It has an Arm CPU - how does it differ from any other machine with a 64 bit Arm CPU?
- brundolf 6y agoThe M1 is much more than a CPU, and a CPU is much more than an instruction set
- klelatti 6y agoI know it's a fast Arm CPU - I've read the Anandtech analysis etc - and that there is lots of extra hardware on the SoC. But the specific point was why is it a Swift machine. What makes it particularly suited to running Swift?
- brundolf 6y agoThe exact reason is out of my depth, but the original quote makes it clear that there is something. Memory bandwidth would be one possibility.
- dathinab 6y agoMy guess: 1. "weak" memory ordering (atomic Aquire/Release loads/stores) 2. low memory latencies between cache and system memory (so dirty pages in caches are faster updated etc.) 3. potential a coherency impl. optimized for this kind of atomic access (purely speculative: e.g. maybe sometimes updating less then a page in a cache when detecting certain atomic operations which changed a page or maybe wrt. the window marked for exclusive access in context of ll/sc-operations and similar) Given that it's common for a object to stay in the same thread I'm not sure how much 2. matters for this point (but it does matters for general perf.). But I guess there is a lot in 3. where especially with low latency ram you might be able to improve performance for this cases.
- gwking 6y ago
- mpweiher 6y agoIf anything, an Objective-C machine: they apparently have a special branch predictor for objc_msgSend()! But they learned the lesson from SOAR (Smalltalk On A RISC) and did not follow the example of LISP machines, Smalltalk machines, Java machines, Rekursiv, etc. and build specialized hardware and instructions. The benefits of the specialization are much, much less than expected, and the costs of being custom very high. Instead, they gave the machine large caches and tweaked a few general purpose features so they work particularly well for their preferred workloads. I wonder if they made trap on overflow after arithmetic fast.
- Karupan 6y ago> running on silicon optimized to make reference counting as efficient as possible I'm curious to understand this. Is this because of a specific instruction set support, or just the overall unified memory architecture?
- ubercow13 6y agoHow does this work? Isn't reference counting a lot of +1 and -1?
- gpderetta 6y agoOn intel atomically increnting a counter is always sequentially consistent. ARM might get away with weaker barriers.
- estebank 6y agoIt is a lot of atomic +1 and -1, meaning possible thread contention, meaning that no matter how many cores your hardware has you have a worst case scenario where all your atomic reference counted objects have to be serialized, slowing everything down. I do not know how ObjectiveC/Swift deals with this normally, but making that operation as fast as possible on the hardware can have huge implications in real life, as evidenced by the new Macs.
- dathinab 6y agoIt's a lot +-1 on atomic variables guarded using atomic memory operations (mainly with the Aquire/Release ordering) on memory which might be shared between threads. So low latency of the cache to system RAM can help here, at least for cases where the Rc is shared between threads. But also if the thread is not shared between threads but the thread is moved to a different CPU. Still it's probably not the main reason. Given how atomic (might) be implemented on ARM and that the cach and memory is on the same chip my main guess is that they did some optimizations in the coherency protocol/implementation (which keeps the memory between caches and the system memory/RAM coherent). I believe there is a bit of potential to optimize for RC, i.e. to make that usage pattern of atomics fast. Lastly they probably take special care that the atomic related instructions used by Rc are implemented as efficient as possible (mostly fetch_add/fetch_sub).
- dirtypersian 6y agoJust to be clear, the RAM/memory and cache are not on the same chip/die/silicon. They are part of the same packaging though. > which keeps the memory between caches and the system memory/RAM coherent Isn't this already true of every multi-core chip ever designed; the whole point of coherency is to keep the RAM/memory coherent between all the cores and their caches.
- darren_ 6y agoThis quote doesn’t really cover why M1 macs are more efficient with less ram than intel macs? You’ve got a memory budget, it’s likely broadly the same on both platforms, the speed at which your retains/releases happen isn’t going to be the issue. it’s not like intel macs use GC where m1 uses RC. (It explains why iOS does better with less ram than android, but the quote is specifically claiming this as a reason for 8GB ram to be acceptable)
- nojito 6y agoThe quote right after explains your concerns. >The memory bandwidth on the new Macs is impressive. Benchmarks peg it at around 60GB/sec–about 3x faster than a 16” MBP. Since the M1 CPU only has 16GB of RAM, it can replace the entire contents of RAM 4 times every second. Think about that…
- nottorp 6y agoSounds like some form of GDDR instead of plain DDR. Not only faster, but I bet simultaneously accessible from both the CPU and GPU.
- MBCook 6y ago> Not only faster, but I bet simultaneously accessible from both the CPU and GPU. It is. We know they're using a unified memory architecture, they pointed it out in the presentation.
- nottorp 6y agoUnified doesn't necessarily mean dual ported.
- dfox 6y agoDue to how DRAM works the array itself cannot have more than one port and almost certainly even the DRAM chip as a whole is still some variation of single ported SDRAM (long time ago there were various pseudo-dual-ported DRAM chips, but these were only really useful for framebuffer-like applications). But given that there are multiple levels of cache in the SoC it is somewhat moot point.
- jml7c5 6y agoI think the author doesn't understand what Gruber wrote here. Android uses more memory because most Android software is written to use more memory (relying on garbage collection). It has nothing to do with the chips. If you ran Android on an M1, it wouldn't magically need less RAM. And Photoshop compiled for x86 is going to use about the same amount of memory as Photoshop compiled for Apple silicon. Sure, if you rewrote Photoshop to use garbage collection everywhere then memory consumption would increase, but that has nothing to do with the chip.
- Ar-Curunir 6y agoIf your hardware enables more regular/efficient garbage collection, then it absolutely can lower memory consumption. Given that the M1 chip was designed to better support reference counting, it makes sense that doing the same for HC could lead to a benefit
- socialdemocrat 6y agoMaybe I misread, but I understood that more as Apple using ARC and that gives them a memory advantage. M1 is simply making that more efficient by doing retain-release faster. But I agree that should not change total memory usage. But I think in general you could say that Apple has focused more on optimizing their OS for memory usage than the competition may have done. Android uses Java which eats memory like crazy and I suspect C# is not that much better being a managed and garbage collected language. Not sure how much .NET stuff is used on Windows, but I suspect a lot. macOS in contrast is really dominated by Objective-C and Swift which does not use these memory hungry garbage collection schemes, nor require JIT compilation which also eats memory.
- reader_mode 6y ago> I suspect C# is not that much better being a managed and garbage collected language C# is better than JVM in that it has custom value types. Say you want to allocate an array of points in Java you basically have to allocate array[pointer] all pointing to tiny 8 byte objects (for eg. 32 bit float x and y coords) + the overhead of object header. If you use C# and structs it just allocates a flat array of floats with zero overhead. Not only do you pointlessly use memory, you have indirection lookup costs, potential cache misses, more objects for GC to traverse, etc. etc. JVM really sucks at this kind of stuff and so much of GUI programming is passing around small structs like that for rendering. FWIW I think they are working on some proposal to add value types to JVM but that probably won't reach Android ever.
- fomine3 6y agoI can't understand why less RAM is enough specially in Apple Silicon rather than Intel. Is the argument proved? RAM capacity is just RAM capacity. Possibly Swift-made apps uses less RAM compared to other apps, but microarchitecture shouldn't be matter.
- socialdemocrat 6y agoI don't quite see that either. But I suspect that it is just macOS itself which uses less memory than Windows in general. But the much faster SSD to RAM transfer for the M1 means that shuffling stuff in and out of RAM is much faster meaning RAM matters less.
- fomine3 6y agoI think OS' usage isn't main matter for RAM. Browser, Electron apps, professional apps like Photoshop, IntelliJ, and VM should be. I expected SSD is way fast, but benchmark says its SSD is about below 3000MB/s RW, not very fast but usual Gen3 x4 speed.
- lhl 6y agoOn my almost-stock (primarily used for iOS deployment) MBP, Catalina uses 3GB RAM (active+wired) memory on boot. It's much more than my Linux laptop (~400MB). I haven't booted Win10 recently but I'd assume it'd be close to macOS. The transfer speeds of the M1 SSDs have been benched at 2.7GB/s - about the same speed as mid-range NVMe SSDs (my ADATA SX8200 Pro and Sabrent Rocket are both faster and go for about $120/TB).
- wsc981 6y agoI’m guessing swapping happens quicker, perhaps due to the unified memory architecture. With quicker swapping you’d be less likely to notice a delay. That said I’d still be very hesitant buying a 8 GB M1 Mac. When my iMac only had 8 GB it was a real pain to use. Increasing my iMac’s memory to 24 GB made it usable.
- fomine3 6y ago
- rowls66 6y agoThe bit about reference counting being the reason that Macs and iOS devices get better performance with less ram makes no sense. As a memory management strategy, reference counting will always use more ram because a reference count must be stored with every object in the system. Storing all of those reference counts requires memory. A reference counting strategy would be more efficient in processor utilization compared to garbage collection as it does not need to perform processor intensive sweeps through memory identifying unreferenced objects. So reference counting trades memory for processor cycles. It is not true that garbage collection requires more ram to achieve equivalent performance. It is in fact the opposite. For programs with identical object allocations, a GC based system would require less memory, but would burn more CPU cycles.
- jandrese 6y agoI think it's more a design pattern you see more with GC collected languages where people instantiate objects with pretty much every action and then let the the GC handle the mess afterward. Every function call involves first creating a parameters object, populating it, then forgetting about it immediately afterward. I've seen this with java where the memory usage graph looks like a sawtooth, with 100s of MB being allocated and then freed up a couple of seconds later.
- jhgb 6y agoIsn't it the case with Java that it will do this because you do have the memory to spend on it? Generally this "handling the mess afterward" involves some kind of nursery or early generation these days, but their size may be use-case-dependent. If tuned for a 8/16 GB environment, presumably the "sawtooth" wouldn't need to be as tall.
- notriddle 6y agoYes, but the reason it does it is to make it fast. The shorter your sawtooth spokes, the slower your app.
- 6y ago
- lend000 6y agoAt the hardware level, does this mean they have a much faster TLB than competing CPU's, perhaps optimized to patterns in which NSObjects are allocated? Speaking of which, does Apple use a custom malloc or one of the popular implementations like C malloc, tcmalloc, jemalloc, etc.?
- saagarjha 6y agoApple uses its own libmalloc: https://opensource.apple.com/tarballs/libmalloc/ https://opensource.apple.com/tarballs/libmalloc/
- vagrantJin 6y ago> This, in a nutshell, helps explain why iPhones run rings around even flagship Android phones, For the price, it better run circles and squares. It should cook my dinner too.
- JimDabell 6y agoA new 2020 iPhone SE costs US$399 and outperforms flagship Android phones that cost much more: https://www.androidcentral.com/cheapest-iphone-has-more-powerful-processor-most-expensive-android-phone https://www.androidcentral.com/cheapest-iphone-has-more-powe... https://www.androidauthority.com/iphone-se-vs-most-powerful-android-galaxy-s20-ultra-1120575/ https://www.androidauthority.com/iphone-se-vs-most-powerful-...
- devxpy 6y agoI, for one would love to see an M1 chip cripple itself under the memory load of 4 intellij windows and chrome tabs, among other applications.
- Cthulhu_ 6y agoThat's my take as well; I have a fairly modern Macbook, it's just fine, it's just that the software I run on it is far from ideal. Intellij has a ton of features but it's pretty heavyweight because of it; I'd like a properly built native IDE with user experience speed at the forefront. That's what I loved about Sublime Text. But I also like an IDE for all the help it gives me, things that the alternative editors don't do yet. I've used VS Code for a while as well, it's faster than Atom at least but it's still web technology which feels like a TON of overhead that no amount of hardware can compensate for. I've heard of Nova, but apparently I installed it and forgot to actually work with the trial so I have no clue how well it works. I also doubt it works for my particular needs, which intellij doesn't have a problem with (old codebase with >10K LOC files of php 5.2 code and tons of bad JS, new codebase with Go and Typescript / React).
- caogecym 6y agoIf you want faster Intellij experience, you can try disable built-in plug-ins and features you don’t need, power saving mode is one quick way to disable heavyweight features.
- devxpy 6y agoThe problem with intellij is the JVM - memory usage is just off the charts. And then combine it with the gradle daemon, and it gets very hairy.
- girvo 6y agoTaking that as true for a moment, I wonder what other programming languages get a benefit from Apple's silicon then? PHP et al. use reference counting too, do they get a free win, or is there something particular about Obj-C and Swift?
- saagarjha 6y agoNot the specific thing mentioned in the article. There is other hardware that optimizes Objective-C specifically, but it’s a branch hint.
- andrekandre 6y agoany links or other info about that? sounds realy cool
- saagarjha 6y agohttps://twitter.com/jckarter/status/1326285905607323650 https://twitter.com/jckarter/status/1326285905607323650
- andrekandre 6y agothats realy impressive... and really vindicates the whole "if your serious about software, make your own hardware too"
- vbezhenar 6y agoI don't think this applies to good software. Nobody will retain/release something in a tight loop. And typical retain/releases don't consume much time. Of course it improves metrics like any other micro-optimization, so it's good to have it, but that's about it.
- saagarjha 6y agoRetains and releases often happen across function call boundaries.
- IshKebab 6y agoI don't think this really makes sense. How many of the benchmarks that people have been running are written in Objective-C? They're mostly hardcore graphics and maths workloads that won't be retaining and releasing many NSObjects.
- ajuc 6y agoAndroid phones are build on managed code, but PC computers are built on C/C++ mostly (almost all productivity apps, browsers, games, the operating system itself). And the only GC code most people run is garbage collected on apple too - it's Java Script on the web. I'm not familiar with MacOs, are the apps there mostly managed code? Even if they were and even if refcounting on Mac is that much faster than refounting on PC - refcounted code would still lose to manual memory management on average.
- mpweiher 6y ago> "M1 and memory efficiency" Hi folks! It looks like my blog post[1] was the primary source for this (it's referenced both by this post and by the Gruber post), and to be clear, I did not claim that this helps ARM Macs use less RAM than Intel Macs. I think John misunderstood that part and now it has blown up a bit... I did claim that this helps Macs and iPhones use less RAM than most non-Apple systems, as part of Apple's general obsessiveness about memory consumption (really, really obsessive!). This part of the puzzle is how to get greater convenience for heap allocation. Most of the industry has settled on tracing GCs, and they do really well in microbenchmarks. However, they need a lot of extra RAM to be competitive on a system level (see references in the blog post). OTOH, RC trends to be more frugal and predictable, but its Achilles heel, in addition to cyclic references, has always been the high cost of, well, managing all those counts all the time, particularly in a multithreaded environment where you have to do this atomically. Turns out, Apple has made uncontented atomic access about as fast as a non-atomic memory access on M1. This doesn't use less RAM, it decreases the performance cost of using the more frugal RC. As far as I can tell, the "magic" of the whole package comes down to a lot of these little interconnecting pieces, your classic engineering tradeoffs, which have non-obvious consequences over there and then let you do this other thing over here, that compensates for the problem you caused in this other place, but got a lot out etc. Overall, I'd say a focus on memory and power. So they didn't add special hardware for NSObject, but they did add special hardware that also tremendously helps NSObjet reference counting. And apparently they also added a special branch predictor for objc_msgSend(). 8-). Hey, 16 billion transistors, what's a branch predictor or two among friends.. ¯\_(ツ)_/¯ [1] https://blog.metaobject.com/2020/11/m1-memory-and-performance.html https://blog.metaobject.com/2020/11/m1-memory-and-performanc...
- llampx 6y agoThanks for the clarification, it seems this post has gotten lost in the comments. You could try making a new post on your blog and add it to HN as a new post.
- pierrebai 6y agoI agreed. It think it's typical of cargo-culting: explanations don't need to make sense, it's all about the breathless enthusiasm. Look, want to know how M1 achieve its result? Easy. Apple is first with a 5nm chips. Look in the past: every CPU maker gains both speed and power efficiency when going down a manufacturing node. Intel CPU were still using a 14nm node (although they called 12+++) while Apple M1 is now at 5nm. According to this [1] chart, that's a transistor density at least 4x. Not saying Apple has no CPU design chops, They've been at it for their phones for quite a while. But people are just ignoring the elephant in the room: Apple gives TSMC a pile of cash to be exclusive for mass production on their latest 5nm tech. [1] https://www.techcenturion.com/7nm-10nm-14nm-fabrication#nbspnbspnbspnbsp7nm_vs_10nm_vs_12nm_vs_14nm_Transistor_Densities