4 ms·
Super cool, congrats on the launch - will be trying this soon! I noticed it’s using Tauri - what are your main takeaways from building a local inference desktop
by yunohn 1y ago
Super cool, congrats on the launch - will be trying this soon! I noticed it’s using Tauri - what are your main takeaways from building a local inference desktop app with it?
- yujonglee 1y agoThanks. I learned that running a server on the Rust side and calling it from a TypeScript frontend is a good approach. For example, we run an OpenAI-compatible server using a Tauri plugin (https://github.com/fastrepl/hyprnote/tree/main/plugins/local-llm/src https://github.com/fastrepl/hyprnote/tree/main/plugins/local...) and call it using the Vercel AI SDK.
- yunohn 1y agoI’ve been thinking about ways to reuse Vercel’s AI SDK, so this is a great one - thanks for sharing!
- yujonglee 1y agoAlso you might find this interesting: https://github.com/fastrepl/hyprnote/blob/main/packages/utils/src/ai.ts https://github.com/fastrepl/hyprnote/blob/main/packages/util....
- yunohn 1y agoSuper neat! You’ve made an impressive amount of cool specialized crates - have you considered making them generally usable to the wider community and licensing them under LGPL/MIT instead of GPL?
- yujonglee 1y agowe def considered it. our line of thinking is start with more restrictive license, and later switch to MIT or something once we figure things out :)
- ashishact 1y agoI wanted to build this for myself. Could never figure out how to get audio output from Mac. Tried almost all audio loopback driver (Blackhole, Soundflower ...). There was problem everywhere wrt security. Even tried making a teams meeting bot. But Teams doesn't give live audio to developer unless you are a special partner. Glad you made this. Will play around