4 ms·

I have a Python script on my mac that detects when I press-and-hold the right option key, and records audio while it's pressed. On release, it transcribes it wi

I have a Python script on my mac that detects when I press-and-hold the right option key, and records audio while it's pressed. On release, it transcribes it with whispercpp and pastes it. Makes it very easy to record quick voice notes. Here it is: https://github.com/corlinp/whisperer/tree/whisper.cpp https://github.com/corlinp/whisperer/tree/whisper.cpp I was working on a native version in the form of a taskbar app with customizable prompt and all. However I quickly realized that the behaviors I want the app to do require a bunch of accessibility permissions that would block it from the app store and require more setup steps. Would anybody still find that useful?

rpastuszak 3y ago
> However I quickly realized that the behaviors I want the app to do require a bunch of accessibility permissions Which behaviours specifically? Personally, I wouldn't worry too much about the App Store. I'm distributing Enso (http://enso.sonnet.io http://enso.sonnet.io) via gumroad.com, and people download/pay for it. I think it's easier than using the App Store Connect route anyway. Here's a good intro: https://rambo.codes/posts/2021-01-08-distributing-mac-apps-outside-the-app-store https://rambo.codes/posts/2021-01-08-distributing-mac-apps-o...
- tornato7 3y ago
  Detecting an alt-key push even when it's not an active window, and editing the selected text field are both accessibility permissions. Thanks for the info about your app. It looks great!
  - alin23 3y ago
    Editing the field definitely needs the permissions, but detecting Alt-key holding should not. You can do that using something like: var reactOnOptionKeyHeld: DispatchWorkItem? { didSet { oldValue?.cancel() } } NSEvent.addGlobalMonitorForEvents(matching: .flagsChanged) { (event) in guard event.modifierFlags == [.option] else { reactOnOptionKeyHeld = nil return } reactOnOptionKeyHeld = DispatchWorkItem { // start recording } // schedule to run if held for at least 1 second DispatchQueue.main.asyncAfter(deadline: .now() + 1, execute: reactOnOptionKeyHeld!) } I see you're using Python with pynput though, which is creating a full key listener so I guess that is why you need the permissions.
    - tornato7 3y ago
      Thanks! Yes, to be clear I was rewriting it in Swift which I'm new at, so I appreciate the expert advice there.
callalex 3y ago
Does anyone actually download their Mac software from the App Store?
- Angostura 3y ago
  Yup, sometimes. Sometimes not.
- palmer_fox 3y ago
  I prefer to because it centralizes updates for apps from different makers. Way more convenient than manually checking each piece of software.
- quitit 3y ago
  Unless it's a trusted recommendation I'll look on the app store first. Sometimes I'll still get it from the app store anyway: - The store itself is convenient for browsing and discovery. - I don't need to do any kind of background checking on the developer prior to running their app - Similarly they don't get access to my credit card details, so I don't have to be concerned about them storing it incorrectly or abusing it later - It's also easier for me to pay them, as often foreign transactions are blocked despite me specifically approving them. - I don't need to hand over my email or other personal details, small developers seem really bad at storing information and CC details properly. I use custom emails for everyone and numerous times I've seen my data on-sold, stolen by ex-employees, or simply thefted from the company by hacking groups. - I don't have to do any reviews when there is an update, I can just accept it knowing that the developer is still trusted and that the project hasn't been hijacked such as the numerous painful times that popular open source projects have had malware snuck into them. - If the app doesn't properly do what claims (or at least what I thought it would), it's a few clicks to get a refund from Apple. - Apple carrot and stick developers to keep their apps up to date with the system. First carrot, and eventually the stick (delisting). Some may argue that some of these things can still happen with an app store, but it's demonstrably less and there are processes in place to deal with that. It's not a popular opinion, especially on HN, but there are plenty of developers who, whether through frustration or dealing with pedantic/rude* customer requests, treat their customers like shit, the store prevents that. The app store ain't perfect, and there are plenty of functions which apps can't have if they're sold through the app store, but for its flaws it's trustworthy and helps me utilise a far larger number of utilities than I'd normally be comfortable with personally maintaining. * I sit on enough discords to see how breathtakingly rude and demanding some users are without noticing it, even for totally free software.
breakfastduck 3y ago
Not being able to be on the app store isn't an issue, it wouldn't put anyone off using it or downloading it. Majority of my apps arent from there, I imagine most long time users are the same.
novok 3y ago
You need to add the "DIR" var in run.sh or it fails if you run outside of the src dir: SRC_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" python3.10 "$SRC_DIR/whisperer.py" $@ Also did you solve the python app icon from bouncing all the time?
- tornato7 3y ago
  Thanks for actually trying it out! I must admit, I didn't pay much attention to UX for installation here since it was mostly for my own use, but that's great you got it working! What do you think? I have not solved the bouncing icon. That's one of the reasons it needs to be rewritten in Swift!
  - novok 3y ago
    I think it's pretty cool to have a hotkey to type STT text anywhere locally. It also helps with using LLMs, it opens you to using more run times since most don't have a whisper plugin, and using the whisper plug is usually an awkward UX too.
mxstbr 3y ago
I want this taskbar app! (ideally with streaming transcription vs. at the end) How can I get it?
GavinMcG 3y ago
I haven’t used MacOS native dictation, but I thought it could be used anywhere text could be input. Does your script have different functionality? Edit: looks like Mac native runs locally only with dapple Silicon, and maybe has a more limited geographic/linguistic reach?
- leodriesch 3y ago
  macOS native dictation is not as good as Whisper in terms of accuracy, however that is probably going to change with macOS Sonoma since they will switch the model for speech recognition to a better one (Transformer based iirc).
  - Obscurity4340 3y ago
    Was it ever addressed that even when you had the microphone turned off it could still detect audial stimuli and reflected that in the oscillating sound wave visual? Makes me wonder if it was/is always listening even with Hey Siri disabled
    - lallysingh 3y ago
      That "sounds" to me like the mic was properly cut off electrically, but the rest of the system as active, so you'd get electrical noise coming in. E.g., the mic and amp are powered down, but the ADC is still active. My old Sun Ultra 40 M2 had a ton of electrical noise on my headphone jack, and I could def. tell when the CPU was busy from what I was hearing.
      - Obscurity4340 3y ago
        I meant like, even after toggling it off when you made noise or spoke, there was a visual representation/feedback for that shown in the wavy graph thing. Not just ambient/moving parts type noise. Just thought it was weird, never really thought about it too much.
  - phren0logy 3y ago

rpastuszak 3y ago

> However I quickly realized that the behaviors I want the app to do require a bunch of accessibility permissions Which behaviours specifically? Personally, I wouldn't worry too much about the App Store. I'm distributing Enso (http://enso.sonnet.io http://enso.sonnet.io) via gumroad.com, and people download/pay for it. I think it's easier than using the App Store Connect route anyway. Here's a good intro: https://rambo.codes/posts/2021-01-08-distributing-mac-apps-outside-the-app-store https://rambo.codes/posts/2021-01-08-distributing-mac-apps-o...

tornato7 3y ago
Detecting an alt-key push even when it's not an active window, and editing the selected text field are both accessibility permissions. Thanks for the info about your app. It looks great!
- alin23 3y ago
  Editing the field definitely needs the permissions, but detecting Alt-key holding should not. You can do that using something like: var reactOnOptionKeyHeld: DispatchWorkItem? { didSet { oldValue?.cancel() } } NSEvent.addGlobalMonitorForEvents(matching: .flagsChanged) { (event) in guard event.modifierFlags == [.option] else { reactOnOptionKeyHeld = nil return } reactOnOptionKeyHeld = DispatchWorkItem { // start recording } // schedule to run if held for at least 1 second DispatchQueue.main.asyncAfter(deadline: .now() + 1, execute: reactOnOptionKeyHeld!) } I see you're using Python with pynput though, which is creating a full key listener so I guess that is why you need the permissions.
  - tornato7 3y ago
    Thanks! Yes, to be clear I was rewriting it in Swift which I'm new at, so I appreciate the expert advice there.

callalex 3y ago

Does anyone actually download their Mac software from the App Store?

Angostura 3y ago
Yup, sometimes. Sometimes not.
palmer_fox 3y ago
I prefer to because it centralizes updates for apps from different makers. Way more convenient than manually checking each piece of software.
quitit 3y ago
Unless it's a trusted recommendation I'll look on the app store first. Sometimes I'll still get it from the app store anyway: - The store itself is convenient for browsing and discovery. - I don't need to do any kind of background checking on the developer prior to running their app - Similarly they don't get access to my credit card details, so I don't have to be concerned about them storing it incorrectly or abusing it later - It's also easier for me to pay them, as often foreign transactions are blocked despite me specifically approving them. - I don't need to hand over my email or other personal details, small developers seem really bad at storing information and CC details properly. I use custom emails for everyone and numerous times I've seen my data on-sold, stolen by ex-employees, or simply thefted from the company by hacking groups. - I don't have to do any reviews when there is an update, I can just accept it knowing that the developer is still trusted and that the project hasn't been hijacked such as the numerous painful times that popular open source projects have had malware snuck into them. - If the app doesn't properly do what claims (or at least what I thought it would), it's a few clicks to get a refund from Apple. - Apple carrot and stick developers to keep their apps up to date with the system. First carrot, and eventually the stick (delisting). Some may argue that some of these things can still happen with an app store, but it's demonstrably less and there are processes in place to deal with that. It's not a popular opinion, especially on HN, but there are plenty of developers who, whether through frustration or dealing with pedantic/rude* customer requests, treat their customers like shit, the store prevents that. The app store ain't perfect, and there are plenty of functions which apps can't have if they're sold through the app store, but for its flaws it's trustworthy and helps me utilise a far larger number of utilities than I'd normally be comfortable with personally maintaining. * I sit on enough discords to see how breathtakingly rude and demanding some users are without noticing it, even for totally free software.

breakfastduck 3y ago

Not being able to be on the app store isn't an issue, it wouldn't put anyone off using it or downloading it. Majority of my apps arent from there, I imagine most long time users are the same.

novok 3y ago

You need to add the "DIR" var in run.sh or it fails if you run outside of the src dir: SRC_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" python3.10 "$SRC_DIR/whisperer.py" $@ Also did you solve the python app icon from bouncing all the time?

tornato7 3y ago
Thanks for actually trying it out! I must admit, I didn't pay much attention to UX for installation here since it was mostly for my own use, but that's great you got it working! What do you think? I have not solved the bouncing icon. That's one of the reasons it needs to be rewritten in Swift!
- novok 3y ago
  I think it's pretty cool to have a hotkey to type STT text anywhere locally. It also helps with using LLMs, it opens you to using more run times since most don't have a whisper plugin, and using the whisper plug is usually an awkward UX too.

mxstbr 3y ago

I want this taskbar app! (ideally with streaming transcription vs. at the end) How can I get it?

GavinMcG 3y ago

I haven’t used MacOS native dictation, but I thought it could be used anywhere text could be input. Does your script have different functionality? Edit: looks like Mac native runs locally only with dapple Silicon, and maybe has a more limited geographic/linguistic reach?

leodriesch 3y ago
macOS native dictation is not as good as Whisper in terms of accuracy, however that is probably going to change with macOS Sonoma since they will switch the model for speech recognition to a better one (Transformer based iirc).
- Obscurity4340 3y ago
  Was it ever addressed that even when you had the microphone turned off it could still detect audial stimuli and reflected that in the oscillating sound wave visual? Makes me wonder if it was/is always listening even with Hey Siri disabled
  - lallysingh 3y ago
    That "sounds" to me like the mic was properly cut off electrically, but the rest of the system as active, so you'd get electrical noise coming in. E.g., the mic and amp are powered down, but the ADC is still active. My old Sun Ultra 40 M2 had a ton of electrical noise on my headphone jack, and I could def. tell when the CPU was busy from what I was hearing.
    - Obscurity4340 3y ago
      I meant like, even after toggling it off when you made noise or spoke, there was a visual representation/feedback for that shown in the wavy graph thing. Not just ambient/moving parts type noise. Just thought it was weird, never really thought about it too much.
- phren0logy 3y ago