29 ms·
Tales of the M1 GPU
- samgranieri 4y agoThis is seriously impressive!! Hats off to everyone involved
- 2OEH8eoCRo0 4y agoApple should upstream their drivers. Who is Asahi Lina? Is that an actual person?
- dboreham 4y ago> Apple should upstream their drivers. To what upstream project?
- 2Gkashmiri 4y agoi can only assume the poster meant apple should add linux drivers of M1/m2 to mainline linux kernel
- squarefoot 4y agoI'm not sure Apple would want users to run anything but Apple operating systems on their hardware, and the other way around (fighting against hackintoshes back in the day). I hope to be proven wrong though, as their hardware is really interesting.
- rjsw 4y agoI thought someone had posted here that Apple ran Linux on their hardware for manufacturing tests.
- gjsman-1000 4y agoHector Martin claims so which (frankly) suggests he has some insiders. However, just because Apple "runs linux" doesn't mean they "run linux" the way you are thinking. It's very easy for corporations to write slapdash, horrific, unmaintainable kernel forks that run on a specific piece of hardware. That's just fine when you are testing hardware before handing it to your OS team, but absolutely unacceptable for upstreaming. For examples of this, take a look at old Android devices (and their ancient kernels), or the original Correlium port of Linux to Apple Silicon (which happened almost half a year before the Asahi Linux beta - but the code was sheer unmaintainable crap). Upstream it? Heck no - it would be rejected entirely and need almost a total rewrite from scratch. Just because you can write a functional driver doesn't mean it is anywhere close to a good, maintainable driver. So, in a nutshell... yes, Apple does use Linux for early manufacturing tests. But it would almost certainly not be in a state where we could benefit much from it, and certain features would likely not be implemented. It's not anywhere near as simple as "Apple has done the work already - just upstream it please!"
- monocasa 4y ago> Hector Martin claims so which (frankly) suggests he has some insiders. Eh, I've heard the same thing form people I trust, and I'm not the Apple news version of deep throat or something. It's simply not a very well kept secret. Although part of me wonders if the code will flow the other way. Now that marcan has put in the elbow grease to upstream concepts like 16KB pages, the non standard ordering for regular MMIO and PCI on Apple Silicon, etc, will Apple embrace those in their custom distro to avoid having so much un-upstreamed code? We'll probably never find out, but it's fun to think about.
- hot_gril 4y agoThey've never taken issue with other OSes running on their hardware. They made Boot Camp for Intel Mac users to run Windows and wrote Windows drivers to use Apple-specific hardware like the iSight cameras. They even showcased this feature in the TV ads. Linux also worked but wasn't explicitly supported. The rule is only, no macOS on non-Apple hardware.
- izacus 4y ago> They've never taken issue with other OSes running on their hardware. What on earth are you talking about, iPads, iPhones and all other new devices from Apple outright lockout any kind of 3rd party OS from booting on them. MacBooks are the only ones grandfathered into old behaviour of actually allowing 3rd party OSes (although they've also lost the ability to boot Windows in latest generation as well).
- sbuk 4y agoIt's clear the context of the whole thread is Apple's OS X/macOS based lineage of machines. Also, they haven't lost the ability to boot Windows, seemingly due to licensing issues with Qualcomm, Microsoft hasn't offered a Windows port yet. Of course, one couldn't boot directly into Windows from the previous generation of PowerPC based Macs either...
- izacus 4y ago"They've never taken issue with other OSes running on their hardware." is a false statement even within context of this thread. They've thanken issue with other OSes on their hardware.
- sbuk 4y agoIs it any different to the false statement that is "(although they've also lost the ability to boot Windows in latest generation as well)"? The last line specifically refers to macOS. No need to be deliberately obtuse.
- kitsunesoba 4y ago> (fighting against hackintoshes back in the day) Apple doesn't really care about individual hackintoshers. Some of their devs have griped about the uselessness of stack traces and logs coming in from hackintoshed machines (bogus errors produced by slight hardware mismatches, drivers developed by amateur community members filling logs with garbage, etc) but they've never gone after anybody who was hackintoshing for personal use. There's even been fairly big YouTubers who've done it without issue, and back when Macs were Intel only and had abysmal thermals/performance a surprising number of Mac/iOS devs were using hackintoshes as their primary dev machines that they submitted to the App Store with. Where they draw the line is selling hackintoshed machines or any of the tools to facilitate the process. Eliminate financial gain from the equation and you'll probably be fine.
- antihero 4y agoThe Linux kernel
- madeofpalk 4y agoIs Apple/macOS downstream of the Linux kernel?
- hot_gril 4y agoNo it's not
- tiagod 4y ago>Asahi Lina, our GPU kernel sourceress. Lina joined the team to reverse engineer the M1 GPU kernel interface, and found herself writing the world’s first Rust Linux GPU kernel driver. When she’s not working on the Asahi DRM kernel driver, she sometimes hacks on open source VTuber tooling and infrastructure.
- deleted 4y ago[deleted]
- worldsavior 4y agoAsahi Lina is a maintainer in Asahi Linux project. She is now much known because of the achivement she earned, programming the Asahi Linux GPU driver for MacOS.
- madeofpalk 4y agoIs Asahi Lina a pseudonym? Is Asahi Linux named after her? Or is it all one big coincidence?
- nicoburns 4y ago> Is Asahi Lina a pseudonym? This one.
- masklinn 4y ago> Is Asahi Lina a pseudonym? Yes > Is Asahi Linux named after her? Other way around, it’s a pseudonym created to work on the project, IIRC Lina got involved early-mid 2022 while Hector Martin started the project in late 2020 / early 2021.
- mlindner 4y agoWhy does she have so much clout when the latter has been working on this for much longer? I feel like there's some fanboyism happening here. This person basically just joined the project.
- masklinn 4y agoThey "just joined the project" and reverse engineered and developed the kernel drivers from relatively little prior work, in just a few months. From afar they also seem to have a pretty ebullient and infectious personality and to stream a lot (we're talking 10h streams every few days: https://www.youtube.com/@AsahiLina/streams https://www.youtube.com/@AsahiLina/streams) which is impressive work and dedication.
- Rufbdbskrufb473 4y ago
- sliken 4y agoTheir = apple? Their = Asahi Lina's? Asahi Linux has been upstreaming, but of course it's ongoing. The GPU driver in particular depends on some rust inside the kernel bits which aren't in the mainline kernel, yet. The 6.1 kernel has some Rust bits, 6.2 will have more, but I don't believe that will be enough for the GPU driver ... yet.
- wincy 4y agoI think it’s a vTuber anime persona of a very talented programmer or something?
- dkdbejwi383 4y agoAnd, sorry for continuing this thread, what is a "vTuber anime persona"?
- bogeholm 4y agovTuber = virtual YouTuber (YouTube as genericized term for “video platform”, as in Xerox for copying) Anime = Japanese cartoon style
- wincy 4y agoIt’s like a full time filter where the only online identity someone presents is as an anime girl. I think it was popularized by a company called HoloLive that has Japanese girls do this as a full time job? It’s a VERY niche culture thing. Check out the Asahi Lina channel to see what I mean. https://m.youtube.com/watch?v=LonzMviFCNs https://m.youtube.com/watch?v=LonzMviFCNs
- wilsonnb3 4y agoSurprisingly it is not that niche anymore - for example, the hololive subreddit has 800,000 subscribers which is about half of the hockey subreddit.
- masklinn 4y agoDo all subscribers to team subreddits follow the main sub as well though? Because I assume fans follow "their" teams' subreddit, and there's several in the 100k range.
- wilsonnb3 4y agoI think most fans subscribe to the "main" sub (r/hocket, nba, baseball, etc) for general league news and discussion in addition to team subs. Hololive functions in a similar way - the most popular members have their own subreddits in addition to the main hololive sub.
- nicoburns 4y ago> Apple should upstream their drivers. Apple don't have linux drivers. It would be great if they wrote some, but it's never going to happen. > Who is Asahi Lina? Is that an actual person? The virtual persona of an actual person who has chosen to remain anonymous (hence the name which would be a crazy coincidence otherwise).
- gjsman-1000 4y ago> Who is Asahi Lina? Is that an actual person? Man... if I was a conspiracy theorist who believed Apple was genuinely evil, what if Asahi Lina is an Apple employee? ;)
- embedded3 4y agoApple has pushed OS updates to help the Asahi team, though they did not publicize this, of course.
- unicornporn 4y agoSource for this claim?
- gregsadetsky 4y agohttps://news.ycombinator.com/item?id=29591578 https://news.ycombinator.com/item?id=29591578
- plonk 4y agoYou got downvoted unfairly, asking for sources is totally normal.
- sbuk 4y agoThis has been long established. marcan_42 has repeatedly posted about this here, on twitter and reddit to name a few locations.
- rvz 4y ago
- ccouzens 4y agoAssuming it is the case I don't think it's polite to share this information. I don't know their motivation for creating a separate public image, but I think we should respect their decision to do so by not connecting them.
- CameronNemo 4y agoThe Linux kernel does not accept anonymous contributions due to legal reasons. https://www.kernel.org/doc/html/latest/process/1.Intro.html https://www.kernel.org/doc/html/latest/process/1.Intro.html > It is imperative that all code contributed to the kernel be legitimately free software. > For that reason, code from anonymous (or pseudonymous) contributors will not be accepted. > All contributors are required to “sign off” on their code, stating that the code can be distributed with the kernel under the GPL. > Code which has not been licensed as free software by its owner, or which risks creating copyright-related problems for the kernel (such as code which derives from reverse-engineering efforts lacking proper safeguards) cannot be contributed.
- saagarjha 4y agoI expect Asahi Lina to be more than happy to assign copyright to an appropriate entity to contribute it upstream.
- bitwize 4y ago
- tgcordell 4y agoI understood deadnaming to generally pertain to gender identity. I don't think it's far-fetched to initially consider Asahi Lina's name as a Pseudonym/Alias/Pen-Name, as many creatives (authors, artists, musicians) have been doing for hundreds of years. If it is a gender identity decision, I still don't view it as malicious for the OP to ask. The context just isn't there in the blog post to make that clear.
- mlindner 4y agoLet's not head down this direction of madness please. I've followed Japanese vtubers for some time and that is CERTAINLY not the case. Vtubers are just aliases for the real person. And each person picks and chooses how much they blend their real lives into that alias. There are even some vtubers that will have a camera facing on themselves while they stream as a vtuber (for example stream their body, but not their face) or will alternate streams between a vtuber persona and a real live camera or vtubers who stream as a vtuber but the real person behind the vtuber is an open secret (i.e. artists who engage in vtubing but sell artwork at comic conventions attending as a real person). There's a huge range and spectrum of ways people choose to do vtubing. (Note: A lot of the latter cases are more possible in Japan because of the general social/legal concept there that taking pictures of people without their permission is at least extremely rude and sometimes also illegal if you don't blur their face when publishing it. This is helped by the fact that it's a legal requirement that all devices capable of taking photographs must make a photographing noise when doing so. For example on iPhone in Japan it is impossible to silence the shutter sound effect without modifying the device hardware.)
- astrange 4y ago> This is helped by the fact that it's a legal requirement that all devices capable of taking photographs must make a photographing noise when doing so. That's not a legal requirement, it's a carrier request. > For example on iPhone in Japan it is impossible to silence the shutter sound effect without modifying the device hardware. That now turns off once you take them out of the country though.
- yjftsjthsd-h 4y ago> Apple should upstream their drivers. Apple's drivers are upstreamed, in Darwin. I'm not aware of any reason to believe that Apple has any Linux drivers that they could upstream.
- psychphysic 4y agoWhat an interesting tale. But I feel like an anime character described it to me. I'm totally fine with it (I'm grateful the story is being told at all), but it is surreal tone for technical writing.
- qrio2 4y agopretty much the appeal(?) of asahi lina. it's been a weird ride to follow for sure.
- throwntoday 4y agoHave to say as much as I want to watch their streams, I can't get passed the annoying voice.
- psychphysic 4y agoI had a quick look and the streams it made the hair stand up on the back of my neck. Not in a good way. The write up wasn't so bad but the entire package is too much for me.
- ddoolin 4y agoI went and checked out the YouTube video attached to one of the tweets. Just...wow. I see what you mean.
- mlindner 4y agoI've said the same but I got flagged on hacker news for saying it. I'm not quite sure why it's an objectionable thing to say.
- renox 4y agoBah I've watched some of Jon Blow's vidéos on Jay, even though he's much more pleasant to watch on video, it's still annoying: a normal blog would be so much more 'efficient'.
- 4y ago
- nicoburns 4y agoThe m1n1 hypervisor specialised for debugging is a pretty genius idea. Is anyone aware of anyone else taking a similar approach? Seems like it would be a pretty generally applicable technique and would make OS/hardware driver development a lot more approachable.
- gjsman-1000 4y agoNot that I know of. m1n1 originated from software that (IIRC) was used initially for reverse-engineering the Wii.
- hedgehog 4y agoSome developers used user mode Linux for driver development, and I think some development has happened on the NetBSD rump kernel more recently. I find the work that goes into building this kind of tooling all pretty impressive.
- bri3d 4y agoEven before true CPU-supported "hypervisors," there was shim software like SoftICE that worked similarly to m1n1 in that you would run an OS underneath and then use a supervisor tool to trace and debug the OS under inspection. More recently, it's fairly common to use a hypervisor or simulator for kernel debugging in device driver development on Windows via Hyper-V. A lot of Linux driver development is done using qemu as well, although this is usually more targeted and isn't quite the same "put a thin shim over the OS running on the hardware" approach. The flexibility and I/O tracing framework in m1n1 are pretty uniquely powerful, though, since it was built for reverse engineering specifically.
- averne_ 4y agoThe nouveau project used a kernel module to intercept mmio accesses: https://nouveau.freedesktop.org/MmioTrace.html https://nouveau.freedesktop.org/MmioTrace.html. Generally speaking hooking onto driver code is one of the preferred ways of doing dynamic reverse engineering. For userspace components, you can build an LD_PRELOAD stub that logs ioctls, and so on.
- ngcc_hk 4y ago
- tiahura 4y agoIs anyone porting the GPU driver to Windows?
- masklinn 4y agoPorting what?
- deleted 4y ago[deleted]
- s3p 4y agoThis made me laugh. Had the same question lol.
- abdusco 4y agoGPU driver. So that one can install Windows on Apple Silicon and get accelerated graphics.
- gjsman-1000 4y agoIt would take more than a GPU Driver, or actually, drivers at all. Windows has some behavior that is hard-coded and cannot be changed except by Microsoft. This behavior can fairly easily be bent on Linux, but Windows drivers can't bend the rules for some of these issues. The biggest issue that immediately comes to mind is that the M1 MMU (Memory Management Unit) only can handle 16K pages. Linux can easily be built for 16K, and most programs work on 16K, but Windows currently only supports 4K on ARM. No driver can fix that - only Microsoft. Of course, if Microsoft joins the Apple Silicon train (which Apple has said they are more than welcome to do), then they probably will have official Apple documentation and not need the Asahi Linux findings.
- fulafel 4y agoIn https://asahilinux.org/2022/03/asahi-linux-alpha-release/ https://asahilinux.org/2022/03/asahi-linux-alpha-release/ it says that 4k pages work on the HW but 16k has tome advantages. Dropping 4k page size supporrt would imply a departure from arm architecture compatibility compatibility I think, it might have also nontechnical problems (ARM license).
- dimator 4y agowatching a virtual persona stream their development of their M1 GPU drivers is one of the most cyberpunk things I've ever seen! it's easy to forget that this world is looking closer and closer to those dreamed up by Gibson, Stephenson, etc. what a time to be alive.
- Mistletoe 4y agoI like your optimism, but it seems more like a Phillip K. Dick novel to me. >In 2021, society is driven by a virtual Internet, which has created a degenerate effect called "nerve attenuation syndrome" or NAS. Megacorporations control much of the world, intensifying the class hostility already created by NAS. from Johnny Mnemonic What can we do to make it more utopian?
- masklinn 4y ago> I like your optimism, but it seems more like a Phillip K. Dick novel to me. Is it? It's basically forums & avatars brought in the medium of audio and video communication.
- extrememacaroni 4y agoThat's probably what everyone would say if they didn't read books. But some read books so... OMG CYBERPUNK DYSTOPIA PHILIP K DICK
- mattnewton 4y agoSmall nit but I was confused, I think Johnny Mnemonic is Gibbson? And I had to look up NAS, I think that part of the movie not the book. I think we have another couple decades before androids and mood organs of Phillip K dick at least but I could be wrong. But parts of Gibbsonian cyberpunk is already here!
- Smoosh 4y ago> already here Just unevenly distributed.
- deleted 4y ago[deleted]
- adultSwim 4y agoKeep it up! I'm sticking with Thinkpads until Linux support
- GekkePrutser 4y agoAlso they have matte screens and real keyboards. The 2015MBP one was the last one that was passable for me, what came after is horrible. Even the new MBP that has real ports again is still not as good as the 2015 in terms of keyboard.
- gsora 4y agoThinkpad keyboards are great (I own a couple T400’s and used to daily drive a X61s), but the latest MacBook Pros have real, actually good keyboards afaik too.
- GekkePrutser 4y agoI've tried them but they're nowhere near Lenovo quality sadly. Unfortunately Lenovo is on a downward trend with ever less travel with each generation so I guess they'll be on par in a few years :(
- girvo 4y agoThat is indeed the trend, for basically all manufacturers. For my part, it's not too much of a worry -- I barely use my laptops keyboard directly, thats what my custom mechanical keeb is for, after all. The laptop being thin, light, and powerful is important to me (otherwise I'll just use a desktop). I despise my work laptop (a Metabox/rebranded Clevo 15" thing), as it is huge, thick, and has a terrible keyboard when I'm forced to use it. These low-travel ones may not be perfect, but they're decent, and that's enough for me personally for the times I do need to use it.
- sbuk 4y agoI can’t agree. I have a recent X series, and at this point, it’s approaching butterfly-keyboard levels of unreliability. I’ve always found the travel to be on the spongy side, but this iteration is terrible. Also, my trackpad literally has to warm up before it’s usable. I’ve tried with the track point, but it just doesn’t work for me.
- kubb 4y agoI can hardly think of a better recommendation of Rust for kernel development than this.
- helf 4y agoThe fact so much hardware these days is running a full real-time OS all the time annoys me. I know it is normal and understandable but everything is such a black box and it has already caused headaches (looking at you, Intel).
- dist1ll 4y agoThere's this great USENIX talk by Timothy Roscoe [1], which is part of the Enzian Team at ETH Zürich. It's about the dominant unholistic approach to modern operating system design, which is reflected in the vast number of independent, proprietary, under-documented RTOSes running in tandem on a single system, and eventually leading to uninspiring and lackluster OS research (e.g. Linux monoculture). I'm guessing that hardware and software industries just don't have well-aligned interests, which unfortunately leaks into OS R&D. [1] https://youtu.be/36myc8wQhLo https://youtu.be/36myc8wQhLo
- lbotos 4y agoAlso one by Bryan Cantrill recently around the stack of black boxes: https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-not-to-open-it-the-need-for-holistic-systems/ https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-...
- est31 4y agoI think making it harder to build an OS by increasing its scope is not going to help people to build Linux alternatives. As for the components, at least their interfaces are standardized. You can remove memory sticks by manufacturer A and replace them with memory sticks from manufacturer B without problem. Same goes for SATA SSDs or mice or keyboards. Note that I'm all in favour of creating OSS firmware for devices, that's amazing. But one should not destroy the fundamental boundary between the OS and the firmware that runs the hardware.
- dist1ll 4y agoBuilding an OS is hard. There's no way around its complexity. But closing your eyes and pretending everything is a file is a security disaster waiting to happen (actually, happening every day). And furthermore, OS research is not only about building Linux alternatives. There are a lot of operating systems that have a much narrower focus than full-blown multi-tenant GPOS. So building holistic systems with a narrower focus is a much more achievable goal. > As for the components, at least their interfaces are standardized That's not true once you step into SoC land. Components are running walled-garden firmware and binary blobs that are undocumented. There's just no incentive to provide a developer platform if no one gives a shit about holistic OSes in the first place.
- deleted 4y ago[deleted]
- hias 4y ago
- calrizien 4y agoGreat work! Seems like Apple could make this a whole lot easier by giving even the slightest support...
- kajaktum 4y agoHalf the comments here are talking about the vtuber herself. Who cares. It's been talked before. Just imagine if half the thread is discussing what gender she is. What I am interested in is the claims here https://asahilinux.org/2022/11/tales-of-the-m1-gpu/#rust-is-magical https://asahilinux.org/2022/11/tales-of-the-m1-gpu/#rust-is-.... (what is it called if it comes with a proof?). The resident C/C++ experts here would have you believe that the same is possible in C/C++. Is that true?
- dagmx 4y agoIn C? No, not unless you write your own scaffolding to do it. In C++? Maybe, but you’d need to make sure you stay on top of using thread safe structures and smart pointers. What Rust does is flip this. The default is the safe path. So instead of risking forgetting smart pointers and thread safe containers, the compiler keeps you honest. So you’re not spending time chasing oddities because you missed a variable initialisation, or you’re hitting a race condition or some kind of use after free. While there’s a lot of people who say that this slows you down and a good programmer doesn’t need it, my experience is even the best programmers forget and (at least for me), I spend more time trying to reason about C++ code than rust, because I can trust my rust code more. Put another way, Rust helps with reducing how much of the codebase I need to consider at any given time to just the most local scope. I work in many heavy graphics C and C++ libraries , and have never had that level of comfort or mental locality.
- tomjakubowski 4y agoI agree that Rust is the better language because it gives you the safe tools by default. Smart pointers are no panacea for memory safety in C++ though: even if you use them fastidiously, avoiding raw pointer access, iterator invalidation or OOB access will come for you. The minute you allocate and have to resize, you're exposed.
- dagmx 4y agoYeah that’s definitely true. Anytime I use any C++ api that I did not write or analyze at my best is a shotgun waiting to go off. Hell, even then, that assumes I know enough to catch everything. I’ve recently been doing security analysis of some very popular repos and C++ is terrifying at times. Rust isn’t perfect but it gives me so much more trust in everything I do.
- neonsunset 4y agoThis looks to be very impressive and interesting but the writing style makes it an incredibly aggravating slog to get through.
- rafale 4y agoLoved reading this. About the triangle/cube screenshot, they were taken on Linux on a physical Mac OS computer? How were you able to deploy your driver, does the M1 GPU have a basic text/console mode allowing you to start and work with Linux? Awesome job.
- puyoxyz 4y agoDisplaying to the screen and stuff was already working, you can already use Asahi Linux and have a GUI and everything, it’s just that it’s all rendered by the CPU right now
- gfody 4y agois there an easy way to unvoicemod the streams?
- pulvinar 4y agoTurn on closed-captioning, and mute the sound.
- fareesh 4y agoI've never played games on my M1 Macbook - what are some popular reasonably graphics intensive games that it would support? Could it run Dota2 for example?
- drooopy 4y agoResident Evil Village recently came out and it performs surprisingly well even on the low end MacBook Air M1 with only 7 GPU cores. What's even more impressive is that the game is playable (low gfx settings, 30fps) when running that machine on low power mode.
- tingol 4y agoDota 2 runs pretty great.
- rpastuszak 4y agoDisco Elysium, Hades and CIV VI run really well on my MBA m1 (using a 4K display). These games are not as resource heavy as Dota2 AFAIK but I’m comparing them to my maxed out 16inch MBP from 2020 which acted more like a cursed semi sentient toaster than a hi spec laptop.
- punnerud 4y agoNot one comment here about the “GPU drivers in Python”. I like the idea of iteration speed, over pure speed. And the coprocessor called “ASC” also have similarities with Python, where the GPU is doing the heavy lifting, but the ASC (like Python) interact using shared memory. The same Python is doing with a lot of its libraries (written in C/C++)
- saagarjha 4y ago> And the coprocessor called “ASC” also have similarities with Python It's a processor, not a programming language :) The team has essentially strapped the API into something that you can poke with Python instead of with a native driver.
- grappler 4y agoI'm loving seeing the shift from embedding and linking to tweets and twitter accounts to embedding and linking to fediverse posts and profiles.
- kaiby 4y ago> It feels like Rust’s design guides you towards good abstractions and software designs. > The compiler is very picky, but once code compiles it gives you the confidence that it will work reliably. > Sometimes I had trouble making the compiler happy with the design I was trying to use, and then I realized the design had fundamental issues! I experience a similar sentiment all the time when writing Rust code (which for now is admittedly just toy projects). So far it's felt like the compiler gives you just enough freedom to write programs in a "correct" way. I don't really do unsafe/lower-level coding, so I can't speak to much there however.