7 ms·
Talk-to-ChatGPT
- xony 4y ago[dead]
- riskneutral 4y agoThis is great, thanks!!!
- dddrh 4y agoBeen noodling on this idea and love to have an example/inspiration. Thanks for sharing. (Edit) Specifically glad to learn about the web speech APIs. As a hobby programmer by night and Product Manager by day learning about more and more capabilities I can leverage in toy projects is awesome. I was considering trying out whisperAI but this seems like a better stepping stone for “simpler” starter projects.
- mmaia 4y agoAwesome. I saw someone asking for this feature to practice other languages just yesterday. In Firefox, it only supports reading which is already cool. Here's a CDN script in case someone wants to load it in JS Console: https://cdn.jsdelivr.net/gh/C-Nedelcu/talk-to-chatgpt@main/chrome-extension/content.js https://cdn.jsdelivr.net/gh/C-Nedelcu/talk-to-chatgpt@main/c... Edit: script url
- joovuu 4y agoShameless self plug but I built something similar: https://heylaing.com https://heylaing.com . Basically I got fed up of not being able to practice my language skills realistically (fed up of always getting precanned responses etc). I also had limited time to get a language buddy and so built this. Originally it was just a python script, then a terrible Flask App that me and some friends used, then posted it on reddit and it went somewhere so started to develop it. Any feedback is welcome! I'm currently working on an iOS and Android app but never built in react-native before so it's a steep learning curve.
- riskneutral 4y agoIt would be good to implement an initial command phrase to begin dictating, like "Hey Alexa," "OK Google," or in this case e.g. "Hey GPT" Also, I feel like it sends the text to ChatGPT too quickly, for me at least. Wish it would wait a bit longer in case I have anything to add. A command phrase to end the sentence might be too much.
- de6u99er 4y agoYour Code says it uses the browser's speech recognition API. ``` // Start speech recognition using the browser's speech recognition API function CN_StartSpeechRecognition() { ``` As far as I know, speech recognition in Chrome, unlike on Android phones, is being done online. Means audio is being sent to Google servers. How does this comply with GDPR?
- renewiltord 4y agoWhy don't you tell us how it violates GDPR? And then you can make an issue on Github to tell the guy to stop offering his thing in the EU.
- jcims 4y agoDoes it claim to be compliant anywhere? I would assume it isn’t.
- itcrowd 4y agoGdpr compliance is not optional for services offered in the EU
- deleted 4y ago[deleted]
- shagie 4y agoDoes every client side javascript project someone builds on GitHub for fun need to be verified if it is GDPR compliant? And what if the source someone shares is using APIs that aren't GDPR complaint? That you can download, compile it, run it on your machine and then say that it was offered to you and so must be GDPR compliant. I'm not in the EU and not familiar with the particulars of GDPR but that feels like it is stretching for a reason to complain.
- deleted 4y ago[deleted]
- wizzwizz4 4y ago
- zaptrem 4y agoHas anyone built a Siri integration/Shortcut for this? I'm referring to the actual ChatGPT model, not via the normal OpenAI API.
- ericHosick 4y agoHave not looked into this but someone I know said they got it working: https://www.youtube.com/watch?v=gePhjvKdUro https://www.youtube.com/watch?v=gePhjvKdUro
- LeoPanthera 4y agoChatGPT does not (yet) have an API. If there are any, they would have to resort to screen-scraping. An API for ChatGPT is apparently coming.
- celdon25 4y agohttps://github.com/acheong08/ChatGPT https://github.com/acheong08/ChatGPT
- LeoPanthera 4y agoThat's really no different to screen scraping. It could break at any moment. And I'm pretty sure it violates the OpenAI TOS too.
- celdon25 4y agoDarn, I guess GP will have to go without then
- anitakirkovska 4y ago[flagged]
- fnordpiglet 4y agoI’m surprised no one has made a chatgpt alexa skill. Although I realize throttling and costs probably stop that.
- lgas 4y agoIf you google, you will find many, eg. https://www.chatgptalexa.com/ https://www.chatgptalexa.com/
- bilater 4y agoNice - I think Web Speech APIs are a super powerful tool that a lot of devs would be surprised to learn they get out of the box (I was).
- Animats 4y agoThis is using Google's text to speech and speech to text, right? Isn't that a paid service?
- russellbeattie 4y agoChrome provides basic speech to text for free via the browser's implementation of the Web Speech API. There is a different Google Cloud version, which is a commercial service that provides a better model for more accuracy and optional data logging. The page below can be used by Safari as well, which sends the data to Apple's servers for processing instead. https://www.google.com/intl/en/chrome/demos/speech.html https://www.google.com/intl/en/chrome/demos/speech.html
- leobg 4y agoCool. I built something similar for myself. Though it uses the GPT-3 API rather than ChatGPT: Whisper.cpp for text input. Google WaveNet Voice for text output. One button for the user to start and stop speaking, like Siri. Allows me and my daughter to literally talk to GPT-3 and have real conversations with it. (Though I’d never let her do that without supervision. Also, she has learned very quickly that she needs to take everything it says with a grain of salt, and that it’s important to fact-check its answers.) I’d be happy to show it, but Whisper is quite CPU intensive. I don’t know how to host it so it can handle any meaningful number of concurrent users without breaking the bank. If anyone has suggestions or wants to help, let me know.
- louison11 4y agoUse a cloud service for speech to text, like Google’s API, and you should be able to handle a pretty good number of users without breaking the bank. (I believe they have a free tier, then pretty reasonable pricing. You’d just have to set up a rate limiter on your server to make sure nobody’s abusing it).
- jsharf 4y agoTheoretically, one could compile whisper.cpp to run in the browser using emscripten, maybe made faster with webgl... I think this would be quite a heavy page load time for a website, but if the model file gets cached, and the user has a decent CPU/GPU, it... could work?
- leobg 4y agoThat is a cool idea! And I just realized the Whisper.cpp guy has already built something like this. Whisper WASM. [1] Unfortunately, on my iPhone this runs so slow that I might just as well send my mic recordings to a typist. And mobile is important here. Being able to quickly pull this up on your phone really is part of the fun. [1] https://whisper.ggerganov.com/ https://whisper.ggerganov.com/
- youssefabdelm 4y agoNobody has done this well enough yet. What's required: 1. Transcribe your speech using Whisper (in that case you don't have to make an effort to speak clearly so long as you're in a relatively quiet room) 2. Get a TTS system that actually sounds good (e.g. Descript, Eleven Labs, etc.) 3. Have RAPID responses like a normal human conversation (mostly on OpenAI's side... so hopefully ChatGPT Plus fixes that)
- ericlewis 4y agorapid responses is basically the biggest problem, it is quite hard but also because you can't stream tokens into any TTS system and have it sound good. The more "complete" the corpus the better it seems to be at using the right sort of pausing and such. So it is more like a: LLM needs to be directly connected to a TTS type of issue, somehow.
- nojs 4y agoThe bottleneck is currently TTS. The best option is probably Eleven Labs, but response times are unpredictable. GPT response times can be worked around by falling back to a faster model, but you can't do that with TTS because the voice needs to be consistent. It seems like current state of the art are diffusion models ala DALL-E, see e.g. [1] (the developer, James Betker now incidentally works for OpenAI). It's nontrivial to turn this into something that works in real-time without a decent budget, though. Whisper (for transcription) is insanely fast and good. 1. https://github.com/neonbjb/tortoise-tts https://github.com/neonbjb/tortoise-tts
- ilaksh 4y agoThe Eleven Labs guy told me they put out an update that made it more consistent, which maybe it is slightly better, but still unpredictable like you say. Maybe you could ask them about it again. I did suggest they make a higher queue priority available for higher level paid tiers.
- babyshake 4y agoIf you want to have rapid responses, you basically want to have the first part of the response ready to go as soon as it is detected the user is no longer speaking. Were they asking a question? While the actual answer is being processed, start responding with "let me answer that for you..." or something like that.
- ericlewis 4y agoeveryone is figuring it out :P https://www.youtube.com/watch?v=ky9L1eGxj_k&t=1s https://www.youtube.com/watch?v=ky9L1eGxj_k&t=1s
- bemmu 4y agoYeah I also made a chrome extensions for this for personal use, but realized that there wasn't a good trade-off between having it sound good and getting rapid responses. You don't want to start speaking before you get a full sentence, or the TTS will sound weird. You don't want to wait for a full sentence, because it would be too slow. Besides that, speech recognition wasn't working well enough, and I heard (haven't tried) Whisper has the same delay problem where you again cannot stream it, but must wait for the user to have spoken for a while.
- jug 4y agoFunny. I was just today thinking of this but on a Raspberry Pi with mic and speakers. Google Bard is surely coming to Google Home but it would be a fun project to get a head start of sorts!
- pandominium 4y agorip to the corporate voice assistants!
- standardUser 4y agoSpectacular plugin, great work. This will make it fun and easy to make my friends try out ChatGPT when they come over. I wonder what kind of work is needed to get a more human-like voice?