3 ms·

ok noob here - what can i use this thing for? a better desktop-only voice app? is there a reason apple hasn't exposed a higher level api for this given the har

by swyx 2y ago

ok noob here - what can i use this thing for? a better desktop-only voice app? is there a reason apple hasn't exposed a higher level api for this given the hardware (mic array) looks like it's already sufficient in macs?

kfarr 2y ago
Looks like the ability to use MacBook mic when not using Macos
everfrustrated 2y ago
Apple did it as a software function so it's not in hardware, hence this implementation for people wanting to run (presumably) Asahi Linux.
Aissen 2y ago
It's a component of Asahi Linux. It's integrated and enabled by default if you have the right laptop.
rogerbinns 2y ago
This is how Apple addressed audio hardware and do something similar for speakers. Instead of trying to make speakers that have the desired frequency response or microphones that produce the desired signal, they let the analog hardware do whatever it does. Then in software they use digital signal processing. For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response, and for the microphones they do this work to extract the desired signal. If Linux addressed the speakers as is, you would get unpleasant sound, and if it read the microphones as is, it would get a lot of noise. That is why Asahi had to add digital signal processing to the audio input and output, to get the "correct" audio. It does mean the processing is specific to the analogue audio hardware in each of the different Mac models. The processing could be done in additional hardware, but why bother when you have a very good CPU that can do the work.
- codedokode 2y ago
  > For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response As I understand, this is not a magic pill: it probably won't help to pull out frequencies which are suppressed by 30-40 dB and I assume that if the frequency response graph is too wavy (lot of narrow peaks and dips), it won't help either. Also, you need to have calibration files to use this method, right?
  - cship2 2y ago
    Interesting, does that means Mac speakers may be great for certain sounds, but not others.
  - crazygringo 2y ago
    I mean, Apple uses high quality speakers to begin with, as far as laptops go. I'm sure they're not making 40 dB corrections, that would be ginormous. Yes, I would be very surprised if they weren't using specific calibrations for each model. That's pretty basic.
  - rogerbinns 2y ago
    Yes you need calibration files for supported models. You can see the details and explanation at the asahi audio repository. They also criticize the MacOS curves, and point out how some Windows vendors are doing the same DSP approach. https://github.com/AsahiLinux/asahi-audio https://github.com/AsahiLinux/asahi-audio
    - codedokode 2y ago
      By the way I now realized that simply adding an equalizer before the amp might be not enough; speakers typically produce different sound in different directions, so for a perfect sound you need to somehow track location of the head and adjust filter curves.
grayhatter 2y ago
your question was non specific so guessing a bit at what you're asking, because some of it is already answered in the docs... but conceptually it's similar how gps triangulation works, but in the other direction, (information flows from the source point, speaker in this case, to the mic array) and with audio waves instead of rf waves. Each mic will have a slightly different view of the audio coming in, and using the timing between them, you can use the wave form that one mic records to figure out what's to early or too late to be audio from directly in front of the laptop. And then delete that audio, leaving just audio from the speaker directly in front of the laptop. eg A ------ MIC1 --- B --- MIC2 ------ C any sound coming from A, will be picked up by MIC1 well before MIC2, same for sounds coming from C. If you delete that audio from the income waveform, you have beam forming. And thus much better audio noise filtering. And as it says in the link, Apple decided to implement this is software, not hardware, so you'd need to reimplement it if you're not using macos.

by swyx 2y ago

kfarr 2y ago

Looks like the ability to use MacBook mic when not using Macos

everfrustrated 2y ago

Apple did it as a software function so it's not in hardware, hence this implementation for people wanting to run (presumably) Asahi Linux.

Aissen 2y ago

It's a component of Asahi Linux. It's integrated and enabled by default if you have the right laptop.

rogerbinns 2y ago

This is how Apple addressed audio hardware and do something similar for speakers. Instead of trying to make speakers that have the desired frequency response or microphones that produce the desired signal, they let the analog hardware do whatever it does. Then in software they use digital signal processing. For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response, and for the microphones they do this work to extract the desired signal. If Linux addressed the speakers as is, you would get unpleasant sound, and if it read the microphones as is, it would get a lot of noise. That is why Asahi had to add digital signal processing to the audio input and output, to get the "correct" audio. It does mean the processing is specific to the analogue audio hardware in each of the different Mac models. The processing could be done in additional hardware, but why bother when you have a very good CPU that can do the work.

codedokode 2y ago
> For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response As I understand, this is not a magic pill: it probably won't help to pull out frequencies which are suppressed by 30-40 dB and I assume that if the frequency response graph is too wavy (lot of narrow peaks and dips), it won't help either. Also, you need to have calibration files to use this method, right?
- cship2 2y ago
  Interesting, does that means Mac speakers may be great for certain sounds, but not others.
- crazygringo 2y ago
  I mean, Apple uses high quality speakers to begin with, as far as laptops go. I'm sure they're not making 40 dB corrections, that would be ginormous. Yes, I would be very surprised if they weren't using specific calibrations for each model. That's pretty basic.
- rogerbinns 2y ago
  Yes you need calibration files for supported models. You can see the details and explanation at the asahi audio repository. They also criticize the MacOS curves, and point out how some Windows vendors are doing the same DSP approach. https://github.com/AsahiLinux/asahi-audio https://github.com/AsahiLinux/asahi-audio
  - codedokode 2y ago
    By the way I now realized that simply adding an equalizer before the amp might be not enough; speakers typically produce different sound in different directions, so for a perfect sound you need to somehow track location of the head and adjust filter curves.

grayhatter 2y ago

your question was non specific so guessing a bit at what you're asking, because some of it is already answered in the docs... but conceptually it's similar how gps triangulation works, but in the other direction, (information flows from the source point, speaker in this case, to the mic array) and with audio waves instead of rf waves. Each mic will have a slightly different view of the audio coming in, and using the timing between them, you can use the wave form that one mic records to figure out what's to early or too late to be audio from directly in front of the laptop. And then delete that audio, leaving just audio from the speaker directly in front of the laptop. eg A ------ MIC1 --- B --- MIC2 ------ C any sound coming from A, will be picked up by MIC1 well before MIC2, same for sounds coming from C. If you delete that audio from the income waveform, you have beam forming. And thus much better audio noise filtering. And as it says in the link, Apple decided to implement this is software, not hardware, so you'd need to reimplement it if you're not using macos.