8 ms·
Triforce – a beamformer for Apple Silicon laptops
- mmaunder 2y agoIs this akin to a phased array RF antenna (like the Starlink dish) but for audio?
- nikisweeting 2y agoYes, except the output is something that has to sound "subjectively good" after all the DSP, vs rf beamforming where you have a very easy metric (dropped packets) that you can optimize the beamforming direction with.
- mmaunder 2y agoThanks!
- amelius 2y agoIt would be great if this was implemented in a way that also other manufacturers can easily start building mic arrays such that it would make them immediately useful.
- TheDong 2y agoI would be surprised if Apple didn't have patents on their mic array, meaning that another manufacturer would ideally prefer if their setup is different and incompatible to reduce the chance of accidental patent infringement. I'd search to see, but reading patents is an info-hazard which increases your chance of infringing, so I've quit reading them entirely.
- __MatrixMan__ 2y agoGolly isn't IP great
- amelius 2y agoMaybe they can still install the array, and we can simply "apt-get install illegal-package". But all joking aside, there is a tremendous amount of literature on the mathematics of beamforming. I'd be surprised if any of it is patented in a way that isn't circumventable.
- xhkkffbf 2y agoYes, I'm sure they have some patents because that's what big companies do/have to do. But the basic idea has been around for a long time, not just in audio but also in microwave space/domain. So I'm sure there's plenty of prior art.
- jeroenhd 2y agoMaybe they're doing something new, but beamforming microphone arrays can be found in just about any brand of laptop if you go high end enough. I do think most such devices will present themselves as less capable than they actually are (I.E. just a stereo input) for maximum OS compatibility, but the technique isn't Apple exclusive as far as I know.
- internetter 2y ago> beamforming microphone arrays can be found in just about any brand of laptop if you go high end enough. Are you sure? I’ve never heard a laptop microphone better than the MacBook. Maybe they do beamform and there’s other issues, but
- wbl 2y agoThere is a customer who has deployed beamforming microphones for decades. They do however have a somewhat different goal and medium.
- pvg 2y agoGithub repo https://github.com/chadmed/triforce https://github.com/chadmed/triforce
- dang 2y agoThanks! We've changed the URL above to that from https://crates.io/crates/triforce-lv2 https://crates.io/crates/triforce-lv2.
- boesboes 2y ago> Much like with the speakers, Apple are trying way too hard to be fancy here, and implement an adaptive beamformer in userspace to try and isolate the desired signal from background noise. Might be fancy, but it does make for surpisingly good audio from a laptops.
- deleted 2y ago[deleted]
- p_l 2y agoHonestly, with speakers it was mainly a patent avoidance thing (patent on essentially the same thing but done with dedicated hardware, doing it with software on "application processor" bypassed the patent claims) A lot of similar stuff is done in firmware on x86 laptops, to the point that both AMD and Intel now share considerable portion of the stack, with both using Xtensa cores for DSP, with Sound Open Firmware as SDK. When I use built-in microphone array on my laptop, it's parsed through the DSPs "transparently" to end user. But technically you can load your own firmware there.
- foobiekr 2y agoIf it was just patent avoidance why aren’t there any non-apple laptops either their sound quality? Both the microphones and the speakers are some of the best audio I’ve ever encountered.
- jeroenhd 2y agoAren't there? I haven't had any trouble with background noise in calls from my ThinkPad, which also does some microphone array trickery as far as I can tell. Unfortunately the drivers for Linux are nowhere near as good so the extra processing the Intel driver does isn't useful for my day to day experience, but I've never had any quality issues. Apple does have some excellent audio engineers for the speakers, although these days the difference isn't as stark as it was five or ten years ago. Of course you need to get a good Windows laptop to get any such quality, and many people and companies seem to only bother spending money on premium laptops if they're made by Apple.
- swyx 2y agook noob here - what can i use this thing for? a better desktop-only voice app? is there a reason apple hasn't exposed a higher level api for this given the hardware (mic array) looks like it's already sufficient in macs?
- kfarr 2y agoLooks like the ability to use MacBook mic when not using Macos
- everfrustrated 2y agoApple did it as a software function so it's not in hardware, hence this implementation for people wanting to run (presumably) Asahi Linux.
- Aissen 2y agoIt's a component of Asahi Linux. It's integrated and enabled by default if you have the right laptop.
- rogerbinns 2y agoThis is how Apple addressed audio hardware and do something similar for speakers. Instead of trying to make speakers that have the desired frequency response or microphones that produce the desired signal, they let the analog hardware do whatever it does. Then in software they use digital signal processing. For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response, and for the microphones they do this work to extract the desired signal. If Linux addressed the speakers as is, you would get unpleasant sound, and if it read the microphones as is, it would get a lot of noise. That is why Asahi had to add digital signal processing to the audio input and output, to get the "correct" audio. It does mean the processing is specific to the analogue audio hardware in each of the different Mac models. The processing could be done in additional hardware, but why bother when you have a very good CPU that can do the work.
- codedokode 2y ago> For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response As I understand, this is not a magic pill: it probably won't help to pull out frequencies which are suppressed by 30-40 dB and I assume that if the frequency response graph is too wavy (lot of narrow peaks and dips), it won't help either. Also, you need to have calibration files to use this method, right?
- everfrustrated 2y agoI can't speak for this implementation, but on MacOS, the beamforming is amazing. When used in a noise office or cafe environment it eliminates background noise to an extent I can always tell if a colleague is using it or their worse headphone mic.
- deleted 2y ago[deleted]
- bustling-noose 2y agoI was sitting at a Starbucks next to a VERY noisy street on a google meet call on an M1 Air with usb-c AirPods (the cheap 19$ one) and I asked the person on the other end if they can hear me at all. To my surprise they couldn’t hear any noise just my voice. No idea which part in the whole setup achieved this but I feel like stuff like AI and all have some applications that can blow you away. Not putting the damn thing in everything!
- gerad 2y agoThat could definitely be Google Meet. I think it does some pretty fancy AI background noise reduction.
- lucb1e 2y agoDue to some unfortunate circumstances I had a customer call on Google Meet once while walking across Paris. I ware barely understandable while holding the earphones' mic in front of my mouth... Good hardware definitely beats software trying to make something out of nothing: can't make directionality out of 1 mic, so Google Meet couldn't filter out background noise in that situation. Though it didn't help that these USB-C DACs seem to all be terrible (I tried several with the best findable reviews) compared to any old headphone jack where the device's internal DAC just worked
- pja 2y agoIn that situation you’re better off using Meet audio only & holding the phone as if doing a video call - the background noise cancellation modern phones do is very impressive but it only really works if you use the phone as a phone & hold it up to your ear / mouth.
- derhuerst 2y agoblog post with the background story why this was created: https://asahilinux.org/2025/03/progress-report-6-14/#is-this-thing-on https://asahilinux.org/2025/03/progress-report-6-14/#is-this...
- sillysaurusx 2y agoThank you. I was about to ask exactly that.
- llm_nerd 2y agoFor the software to perform beamforming it must be provided the discrete microphone inputs, as opposed to being provided some sort of pre-mixed feed. As such, why is Apple "trying way too hard to be fancy here" if you can just use one of those mics? Or is the alternative that they do the "beamforming" in hardware regardless of the OS?
- p_l 2y agoAvoiding extra coprocessor and/or avoiding patent dispute like they did with speakers (which differ from a H-K patent by not having a discrete chip implementing it)
- bri3d 2y ago> if you can just use one of those mics? They're extremely omnidirectional and very sensitive. With a single mic with no beamforming you get basically all of the sounds from every part of the room, including and especially horribly loud sounds from (eg.) the keyboard and mouse. Apple selected their microphones based on the constraints their system had (beam formed array) rather than the "usual" laptop microphone which is physically not very sensitive and highly directional towards the front of the laptop, and in turn, those microphones are not particularly useful without beam forming. Other laptops with beamformed arrays simply don't expose the raw mics to userland, by doing the beamforming in firmware, but this of course comes with its own set of issues.
- com2kid 2y ago> Other laptops with beamformed arrays simply don't expose the raw mics to userland Not always true, back in the Windows XP days (!!!) some laptops would expose the array to software and let the user configure where the mics record from. It is unfortunate that user control has been polished out of modern systems in exchange for "it just kind of works".
- 011110111110111 2y ago[flagged]
- pzo 2y ago> the microphone array found in the following Apple Silicon laptops: > MacBook Pro 13" (M1/M2) > MacBook Air 13" (M1/M2) > MacBook Pro 14" (M1 Pro/Max, M2 Pro/Max) > MacBook Pro 16" (M1 Pro/Max, M2 Pro/Max) > MacBook Air 15" (M2) Does it mean M2/M3 don't have similar array of microphones or rather not tested? I'm even curious if this is only supported on Linux or MacOS as well - not sure if apple provides dedicated microphone stream for each mic?
- skygazer 2y agoasahi Linux doesn’t yet support m3 and m4 processors.
- monocasa 2y agoThey list M2 devices. M3 is just not supported by Asahi Linux, so not being listed is just orthogonal to if M3 has any mics like this. MacOS has its own software deep within the system for handling this; it's only exposed as a normal microphone to application software.
- entropicdrifter 2y agoIt's made just for Asahi Linux. MacOS does some very similar beamforming math behind the scenes, so it just presents you with a single unified mic.
- _zoltan_ 2y agoI think the title should say "for asahi linux", else it's misleading. of course Apple has this implemented.
- rob74 2y agoWell, if it's available on crates.io, I guess nobody will think that it's from Apple. Also, it could conceivably be used in other software besides Asahi too...
- deleted 2y ago[deleted]
- pvg 2y agoIncomplete or 'not 100% obvious' is not really 'misleading'. Titles don't say everything about a story or we wouldn't need stories. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20%22work%20a%20little%22&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- lucb1e 2y agoDoes it not also run on macOS? It could be useful if you want to tweak usage of the mic array yourself, rather than relying on proprietary magic that gives you the output it deems best
- com2kid 2y agoOver 20 years ago I had a Toshiba Tablet PC convertible that had a beam forming array of microphones and it came with software that let you point where you wanted to record from. The use case was for lectures, you could tell the laptop to just record from behind it, pointing the beam in the direction of the professor. Amazing idea and something I haven't seen since.
- bayindirh 2y agoIn the golden age of mini camcorders, some Sony Handycams had "zoom" microphones which used beam forming to limit gathered sound to roughly the area equal to the what your sensor sees. Another great idea. Oh. They still make similar stuff: https://electronics.sony.com/imaging/imaging-accessories/all-accessories/p/ecmgz1m https://electronics.sony.com/imaging/imaging-accessories/all...
- atorodius 2y agoI feel like my iPhone does it. But not sure. Sound definitely changes when you zoom while recording
- bayindirh 2y agoMine is too old to test the claim, but knowing that it has at least three microphones on board, It'd be absurd if Apple didn't implement it.
- entropicdrifter 2y agoIt's pretty computationally cheap, too, as long as you've got the math right and an easy way to choose where to aim the beam
- internetter 2y agoThey do. They rarely mention it but they do: https://devstreaming-cdn.apple.com/videos/wwdc/2019/249a0jw909n3uq0/249/249_introducing_multicamera_capture_for_ios.pdf?dl=1 https://devstreaming-cdn.apple.com/videos/wwdc/2019/249a0jw9...
- raphlinus 2y agoRegarding the SIMD optimizations, the authors may want to look into faer. I haven't had a great experience with its underlying library pulp, as I'm trying to things that go beyond its linear algebra roots, but if the goal is primarily to accelerate linear algebra operations, I think it will go well. I've got a blog post and associated podcast on Rust SIMD in the pipeline, we'll touch on this. [1]: https://docs.rs/faer/latest/faer/ https://docs.rs/faer/latest/faer/
- rob74 2y agoWow, this really puts into perspective how much work has to be put into even the most insignificant details of getting Linux to run on (Apple Silicon) Macs. I say "insignificant" with all due respect because, well, the built-in microphone sees very little use (except if you have forgotten your headset). Or, to quote the progress report (https://asahilinux.org/2025/03/progress-report-6-14/#is-this-thing-on https://asahilinux.org/2025/03/progress-report-6-14/#is-this...): "This is Apple though. Nothing is ever simple."
- beng-nl 2y agoI always prefer headset too, but I did find it striking how good the audio quality of the built in mic was compared to headset when I tried it once..
- brundolf 2y agoThe built-in microphone is actually excellent, I often use it even when I have my AirPods Pro in because the sound quality is so much better If you've got headphones with a wraparound microphone on its own arm then it could be better, but everyday headphones are limited by the position of the microphone
- eptcyka 2y agoI hope you do not take notes or brush dust off the macbook whilst in a video call.
- gabrielhidasy 2y agoWhy, are they not able to reject these types of noise? My X1 doesn't even register typing in a video call
- pantalaimon 2y agoEven on Linux?
- jrockway 2y ago
- Aurornis 2y ago> Much like with the speakers, Apple are trying way too hard to be fancy here, and implement an adaptive beamformer in userspace to try and isolate the desired signal from background noise. That’s a rather unfair characterization. I’ve found the array to work very well in practice. It’s hardly trying to hard.
- Terretta 2y agoPerhaps the author meant "Apple is succeeding too hard". :-)
- deleted 2y ago[deleted]
- deathanatos 2y agoThey are atrocious, IME. I continually get near muted. I.e., if I record the signal, my voice is there, but extremely faint. Unusable for VC audio, and I've moved completely to a headset mic because of it.
- marmarama 2y agoYou can get surprisingly good results from cheap laptop hardware (as well as fancier hardware like an MBP) using software DSP techniques. One of the things I'm pleased about is that quite a bit of Asahi's audio work is just as applicable to generic laptops as it is to Macs. I already use the Bankstown bass harmonics synthesis plugin developed for Asahi and a convolution EQ on a cheap HP laptop, with startlingly impressive results, using the Pipewire plugin chain autoload feature also developed for Asahi. I suspect there are quite a few use cases for this beamformer outside of the Asahi ecosystem as well.
- JohnBooty 2y agoMuch like with the speakers, Apple are trying way too hard to be fancy here Could the author of this package comment on this statement? I'd be really interested in their opinion of their speaker implementation. What's overly complicated there? The hardware? The software? As a MBP user and hobbyist audio guy I've been really impressed with the implementation of those speakers, particularly on the larger MBP models. But I'm just a hobbyist and don't have any knowledge of them other than the driver arrangement (tweeter + dual opposed woofers). It certainly seems like they're pulling the same tricks used by "good" bluetooth speaker designers in order to wring acceptable perf and bass extension from teeny tiny speakers (adaptive EQ etc)
- crazygringo 2y agoI'm confused too. These days, "spatial audio" on speakers (different from on headphones) and beamforming mics is starting to feel standard, at least on premium hardware. Dumb, noisy, cramped, unbalanced audio just doesn't cut it anymore.
- numpad0 2y agoif you think fake 5.1ch sounds better, not like better for enjoying action movies, you've never had exposure to a >$99 pair of bookshelf speakers with a non-USB powered class D amp. change my mind.
- crazygringo 2y agoHuh? Who's talking about bookshelf speakers? This is about laptop speakers that just pass audio directly through, vs. laptop speakers that process the audio including spatially. Yes, it sounds dramatically better. And it's not just about "fake 5.1" but even just mono or stereo. External speakers are a totally different conversation.
- deleted 2y ago[deleted]
- 2y ago
- apitman 2y ago> This is an attempt at a beamformer armed only with first year undergrad level engineering maths and some vague idea of the principles gleaned from various webpages and PDFs Not certain if OP is saying they are currently an undergrad, but impressive if so
- qoez 2y agoYou should be able to send data and record it in a way that measures local geometry like hands since there's a microphone array like this, interesting.
- GeekyBear 2y agoThere is a more general discussion on the latest Asahi Linux progress report. > Unfortunately, PDM mics are very omnidirectional and very sensitive. We cannot get by without some kind of beamforming. https://asahilinux.org/2025/03/progress-report-6-14/ https://asahilinux.org/2025/03/progress-report-6-14/ Also, it turned out that some previous work done for the speaker output was reused here for mic input. > Thanks to the groundwork laid in PipeWire and WirePlumber for speaker support, wiring up a DSP chain including Triforce for the microphones was really simple. We just had to update the config files, and let WirePlumber figure out the rest!
- Tade0 2y agoMy (never finished) master's thesis was about something similar - taking advantage of the fact that (almost) all smartphones have at least two microphones I wanted to locate and separate a speaker in 3D. A few takeaways: -The sampling rate is slightly off between devices - approximately ±1 sample per second - not a lot, but you need to take that into account. -Spectral characteristics in consumer microphones are all over the place - two phones of the same model, right out of the box, will have not only measurable, but also audible differences. -Sound bounces off of everything, particularly concrete walls. -A car is the closest thing to an anechoic chamber you can readily access. -The Fourier transform of a Gaussian is a Gaussian, which is very helpful when you need to estimate the frequency of a harmonic signal (like speech) with a wavelength shorter than half your window, but just barely.
- gpm 2y ago> - A car is the closest thing to an anechoic chamber you can readily access. I recall a youtuber solving the anechoic chamber problem by finding a big empty field - nothing to reflect off of except the ground - and maybe putting some foam below the experiment. It doesn't kill environmental noise, of course, but it apparently did a very good job of killing reflections from his own instruments.
- Tade0 2y agoIn my case wind noise disturbed the signal too much. Normally there's additional processing which deals with it, but I was working with (next to) raw data.
- ipunchghosts 2y ago>The Fourier transform of a Gaussian is a Gaussian, which is very helpful when you need to estimate the frequency of a harmonic signal (like speech) with a wavelength shorter than half your window, but just barely. I get the gaussian link. But, can you explain your point with more detail?
- Tade0 2y ago
- neuroelectron 2y agoI'm really enjoying this trend of minimal dependencies, but I'm not taking off my tinfoil hat yet.
- m463 2y agoI wonder if there's a way to do this in reverse for people who use the speakerphone or play a video in a restaurant.
- jedisct1 2y agoHow to download and install the app? There are no instructions.
- gregjw 2y agoI have no idea what any of this means
- sand500 2y agoI find this kind of thing a good case for LLMs as they can dumb down the technical jargon: From Gemini: ``` Imagine you're trying to record someone talking in a noisy room using your MacBook's built-in microphones. This software acts like a super-smart filter: * It knows where the microphones are: Apple laptops have multiple tiny microphones. * It listens to all of them at once: It takes the input from all the microphones. * It figures out where the person talking is: It analyzes the sound to find the direction of the voice. * It focuses on that voice: It boosts the sound coming from that direction. * It quiets down the other noises: It reduces the sound from other directions, like background chatter. So, instead of getting a muddy recording with lots of noise, you get a clearer recording of the person you want to hear. Basically, it makes your MacBook's microphones sound much better in noisy environments. And it's designed to work within audio programs that use a specific plugin format called LV2. ```
- CaffeineLD50 2y ago/_\ /_\/_\ Triforcin'
- 7e 2y agoIt’s funny how the author has the chutzpah to simultaneously insult Apple and while also failing to replicate what they have done.
- stevenhuang 2y agoYou misread. It's more like grudging admiration that Apple took the proprietary software beamforming route. It's a remark to its technical function but acknowledges that because of its closed implementation, the microphones just cannot be used outside of the macOS ecosystem without additional effort (like in this repo). Which is, as I'm sure you agree, is unfortunate and at least deserving of some (minor) reprisal.
- parasubvert 2y agoThis is one of the cooler features of Apple Vision Pro, it does such good beamforming for the wearer's mouth that someone could be screaming next to you or blasting music, and other parties on Zoom or FaceTime will not hear them.