4 ms·

This is such a fascinating topic. I do think we will see the plot of "Her" playing out in the next 12-24 months. For your specific case, I agree OpenAI moderati

This is such a fascinating topic. I do think we will see the plot of "Her" playing out in the next 12-24 months. For your specific case, I agree OpenAI moderation is an annoying overreach and maybe switching to Mistral could be a possible way to move forward. The 7B is perhaps not good enough currently but we are fast getting there and will likely have a GPT-4 level capability in an open source < 20B model soon.

brucethemoose2 3y ago
Mistral 7B finetunes are extremely good. More than good enough for an AI friend app, with long context to boot. I would suggest Toppy-M or OpenHermes 2.5, or one of the new YARN extended models. And, there is the potential to run the model on the user's device with MLC-LLM and whisper, to save on API costs (and privacy concerns) for users with newer phones. This could give the app a true "free" tier to help it proliferate while making charging for API requests kinda doable. I'm not sure if any good TTS models have been ported yet, but I'm sure its a matter of time.
- bilater 3y ago
  Yes - ASR on browser/device has reached the quality threshold pretty much with distil whisper/ whisper 3. Quality TTS with fast latency is still not there but getting better (you can use Tortoise but its slow and compute expensive as far as I know). Bark is another option but has mixed results. I played around with a recent transformers.js one (using SpeechT5) you can run in your browser and am optimistic where we can go with some improvements: https://tinyllms.vercel.app/dashboard/tts https://tinyllms.vercel.app/dashboard/tts
  - brucethemoose2 3y ago
    I was recently pointed to xtts: https://coqui.ai/blog/tts/open_xtts https://coqui.ai/blog/tts/open_xtts I dunno about performance, but the model is 1.86GB.
    - bilater 3y ago
      Yes - that one is also a good option. Easy to use with services like Replicate: https://replicate.com/lucataco/xtts-v2 https://replicate.com/lucataco/xtts-v2

brucethemoose2 3y ago

Mistral 7B finetunes are extremely good. More than good enough for an AI friend app, with long context to boot. I would suggest Toppy-M or OpenHermes 2.5, or one of the new YARN extended models. And, there is the potential to run the model on the user's device with MLC-LLM and whisper, to save on API costs (and privacy concerns) for users with newer phones. This could give the app a true "free" tier to help it proliferate while making charging for API requests kinda doable. I'm not sure if any good TTS models have been ported yet, but I'm sure its a matter of time.

bilater 3y ago
Yes - ASR on browser/device has reached the quality threshold pretty much with distil whisper/ whisper 3. Quality TTS with fast latency is still not there but getting better (you can use Tortoise but its slow and compute expensive as far as I know). Bark is another option but has mixed results. I played around with a recent transformers.js one (using SpeechT5) you can run in your browser and am optimistic where we can go with some improvements: https://tinyllms.vercel.app/dashboard/tts https://tinyllms.vercel.app/dashboard/tts
- brucethemoose2 3y ago
  I was recently pointed to xtts: https://coqui.ai/blog/tts/open_xtts https://coqui.ai/blog/tts/open_xtts I dunno about performance, but the model is 1.86GB.
  - bilater 3y ago
    Yes - that one is also a good option. Easy to use with services like Replicate: https://replicate.com/lucataco/xtts-v2 https://replicate.com/lucataco/xtts-v2