4 ms·

Created a script to share screen with gpt 4 and asking it to guide through blender. Latency could be better if openai tts api supported a streaming text input

worldsayshi 3y ago
Yes I feel like the delay seen in the video is about the same length as I would expect when making a request without streaming.
- Suneel478 3y ago
  Yeah, elevenlabs supports input text streaming but in practice it seems to wait for quite a while before it starts streaming. Should look into ways to make TTS much more instant
  - cooper_ganglia 3y ago
    Play.ht has a great Turbo model that produces audio chunks within less than 200ms from request. It doesn’t sound quite as good as ElevenLabs, but it’s about 90% as good and is much faster. Might be worth checking out.
dr_dshiv 3y ago
Can you share the script? It’s a great idea
- Suneel478 3y ago
  Will clean up scripts and share soon on this thread - https://twitter.com/suneel_matham/status/1722538037551530069 https://twitter.com/suneel_matham/status/1722538037551530069
dbish 3y ago
Pretty cool. I wonder if there's enough interest from other devs to hook into something like this or build "apps" on top of this ability. Are you trying to "productionize" or build a platform for others on this or was it a cool hack that you don't plan on spending much more time on? Did you send screenshots at a certain FPS or do some other decision making on when to use the bigger LLM? My little startup has system that can stream apps to an ML processing service with both a video of what's on screen and things like the context around it and what you clicked on. We run an LLM on top of this after a bunch of other processing (OCR, delta change detection, speech reco, etc.) for our knowledge capture purposes for our single app. It would be really straight forward to make such a platform available to others to build apps like what you're showing on more then just a browser, pretty much any application you run on a desktop or working across multiple of them, we haven't since before the new GPT4 with vision, most people weren't working on anything where that would help. Anyway, I think we've solved a bunch of the heavy lifting to make this possible, feel free to email me, or anyone else reading this who like this space, if you're a company or dev that might want that layer so you can build something cool on top of it like this (diamond@augmend.com).

worldsayshi 3y ago

Yes I feel like the delay seen in the video is about the same length as I would expect when making a request without streaming.

Suneel478 3y ago
Yeah, elevenlabs supports input text streaming but in practice it seems to wait for quite a while before it starts streaming. Should look into ways to make TTS much more instant
- cooper_ganglia 3y ago
  Play.ht has a great Turbo model that produces audio chunks within less than 200ms from request. It doesn’t sound quite as good as ElevenLabs, but it’s about 90% as good and is much faster. Might be worth checking out.

dr_dshiv 3y ago

Can you share the script? It’s a great idea

Suneel478 3y ago
Will clean up scripts and share soon on this thread - https://twitter.com/suneel_matham/status/1722538037551530069 https://twitter.com/suneel_matham/status/1722538037551530069

dbish 3y ago

Pretty cool. I wonder if there's enough interest from other devs to hook into something like this or build "apps" on top of this ability. Are you trying to "productionize" or build a platform for others on this or was it a cool hack that you don't plan on spending much more time on? Did you send screenshots at a certain FPS or do some other decision making on when to use the bigger LLM? My little startup has system that can stream apps to an ML processing service with both a video of what's on screen and things like the context around it and what you clicked on. We run an LLM on top of this after a bunch of other processing (OCR, delta change detection, speech reco, etc.) for our knowledge capture purposes for our single app. It would be really straight forward to make such a platform available to others to build apps like what you're showing on more then just a browser, pretty much any application you run on a desktop or working across multiple of them, we haven't since before the new GPT4 with vision, most people weren't working on anything where that would help. Anyway, I think we've solved a bunch of the heavy lifting to make this possible, feel free to email me, or anyone else reading this who like this space, if you're a company or dev that might want that layer so you can build something cool on top of it like this (diamond@augmend.com).