14 ms·
The most impressive part is that the voice uses the right feelings and tonal language during the presentation. I'm not sure how much of that was that they had t
by Jensson 2y ago
The most impressive part is that the voice uses the right feelings and tonal language during the presentation. I'm not sure how much of that was that they had tested this over and over, but it is really hard to get that right so if they didn't fake it in some way I'd say that is revolutionary.
- gdb 2y ago(I work at OpenAI.) It's really how it works.
- xanderlewis 2y agoI like the humility in your first statement.
- moab 2y agoPretty sure the snark is unnecessary.
- ayhanfuat 2y agoWas it snark? To me it sounds like "we all know you Greg"?
- xanderlewis 2y agoThis was my intention.
- moab 2y agoI misunderstood; my apologies.
- colecut 2y agoI don't think it was snark. The guy is co-founder and cto of OpenAi, and he didn't mention any of that..
- renewiltord 2y agoI downvoted independently. No problem with groupies. They just contaminate the thread. Greg Brockman is famous for good reasons but constant "oh wow it's Greg Brockman" are noisy.
- egillie 2y agonot snark. if only hn comments could show the right feelings and tonal language
- Induane 2y agoI like their username.
- belter 2y agoYou might be talking to GPT-5...
- deleted 2y ago[deleted]
- theboat 2y agoI love how this comment proves the need for audio2audio. I initially read it as sarcastic, but now I can't tell if it's actually sincere.
- xanderlewis 2y agoIt’s completely sincere. I’m surprised by the downvotes. Greg Brockman needs no introduction.
- bombcar 2y agoAnd here I thought it was just a GNU debugger fan or something.
- latexr 2y ago> Greg Brockman needs no introduction. Even if that were true¹, it doesn’t mean everyone would know their HN user name. ¹ Greg may be well known within a select group of people but that’s way smaller than even users of ChatGPT.
- xanderlewis 2y agoI clicked through to see his bio; I didn’t know his username.
- jamestimmins 2y agoWith this capability, how close are y'all to it being able to listen to my pronunciation of a new language (e.g. Italian) and given specific feedback about how to pronounce it like a local? Seems like these would be similar.
- estebank 2y agoIn the "Point and learn Spanish" video, when shown an Apple and a Banana, the AI said they were a Manzana (Apple) and a Pantalón (Pants).
- unsatchmo 2y agoNo, I just watched it closely and it definitely said un platano
- estebank 2y agoI re watched it a few times to ensure it said plátano before posting, and it honestly doesn't sound like it to me.
- david-gpu 2y agoI'm a Spaniard and to my ears it clearly sounds like "Es una manzana y un plátano". What's strange to me is that, as far as I know, "plátano" is only commonly used in Spain, but the accent of the AI voice didn't sound like it's from Spain. It sounds more like an American who speaks Spanish as a second language, and those folks typically speak some Mexican dialect of Spanish.
- baq 2y ago> (I work at OpenAI.) Winner of the 'understatement of the week' award (and it's only Monday). Also top contender in the 'technically correct' category.
- swyx 2y agoand was briefly untrue for like 2 days
- behnamoh 2y ago> Winner of the 'understatement of the week' award (and it's only Monday). Yes! As soon as I saw gdb I was like "that can't be Greg", but sure enough, that's him.
- Uptrenda 2y ago[flagged]
- jasondigitized 2y agoBro what?
- Uptrenda 2y ago[flagged]
- mttpgn 2y agoLicensing the emotion-intoned TTS as a standalone API is something I would look forward to seeing. Not sure how feasible that would be if, as a sibling comment suggested, it bypasses the text-rendering step altogether.
- skottenborg 2y ago"(I work at OpenAI.)" Ah yes, also known as being co-founder :)
- leozq 2y agoyes, also known as a programmer loves coding a lot:)
- terhechte 2y agoRandom OpenAI question: While the GPT models have become ever cheaper, the price for the tts models have stayed in the $15/1Mio char range. I was hoping this would also become cheaper at some point. There're so many apps (e.g. language learning) that quickly become too expensive given these prices. With the GPT-4o voice (which sounds much better than the current TTS or TTS HD endpoint) I thought maybe the prices for TTS would go down. Sadly that hasn't happened. Is that something on the OpenAI agenda?
- deleted 2y ago[deleted]
- passion__desire 2y ago[flagged]
- 999900000999 2y agoHow far are we away from something like a helmet with chat GPT and a video camera installed, I imagine this will be awesome for low vision people. Imagine having a guide tell you how to walk to the grocery store, and help you grocery shop without an assistant. Of course you have tons of liability issues here, but this is very impressive
- rfoo 2y agoCan't wait for the moment when I can puta single line "Help me put this in the cart" on my product and magically sells better.
- smokel 2y agoThis Dutch book [1] by Gummbah has the text "Kooptip" imprinted on the cover, which would roughly translate to "Buying recommendation". It worked for me! [1] https://www.amazon.com/Het-geheim-verdwenen-mysterie-Dutch/dp/9080348163 https://www.amazon.com/Het-geheim-verdwenen-mysterie-Dutch/d...
- DonHopkins 2y agohttps://en.wikipedia.org/wiki/Steal_This_Book https://en.wikipedia.org/wiki/Steal_This_Book
- seanmcdirmid 2y agoOr tell the AI to optimize paper clip production as much as possible.
- macintux 2y agoJust the ability to distinguish bills would be hugely helpful, although I suppose that's much less of a problem these days with credit cards and digital payment options.
- krainboltgreene 2y ago> Imagine having a guide tell you how to walk to the grocery store I don't need to imagine that, I've had it for about 8 years. It's OK. > help you grocery shop without an assistant Isn't this something you learn as a child? Is that a thing we need automated?
- bjtitus 2y agoIs it possible to use this as a TTS model? I noticed on the announcement post that this is a single model as opposed to a text model being piped to a separate TTS model.
- cchance 2y agoThis is damn near one of the most impressive things, can only imagine especially with live translation and voice synthesis (eleven labs style) you'd be capable of to integrate with something like teams (select each persons language and do realtime translation to each persons native language, with their own voice and intonations would NUTS)
- purplerabbit 2y agoThere’s so much pent up collaborative human energy trapped behind language barriers. Beautiful articulation. This is an enormous win for humanity.
- deleted 2y ago[deleted]
- sensanaty 2y agoBy humanity you mean Microsoft's shareholders right? Cause for regular people all this crap means is they have to deal with even more spam and scams everywhere they turn. You now have to be paranoid about even answering the phone with your real voice, lest the psychopaths on the other end record it and use it to fool a family member. Yeah, real win for humanity, and not the psycho AI sycophants
- throwaway11460 2y agoLet AI answer the phone. I love it, can't wait. I hate answering phone but some people just won't email, they always call.
- rane 2y agoWill the new voice mode allow mixing languages in sentences? As a language learner, this would be tremendously useful.
- selfmodruntime 2y agoI've always been wondering what GPT models lack that makes them "query->response" only. I've always tried to get chatbots to lose the initially needed query, with no avail. What would It take to get a GPT model to freely generate tokens in a thought like pattern? I think when I'm alone without query from another human. Why can't they?
- kolinko 2y agoJust provide empty queey and that’s it - it will generate tokens no prob. You can use any open source model wirthout any promot whatsoever
- djur 2y agoYou might not have a prompt from another human, but you're always receiving new input.
- Filligree 2y agoAnd humans malfunction pretty badly without input. Even solitary confinement quickly drives them insane.
- selfmodruntime 2y agoYes, but that's the fundamental difference. Even if I closed my eyes, plugged my ears and nose and laid in a saltwater floating chamber, my brain will always generate new input / noise. (GPT) Models toggle between a state of existence when queried and ceasing to exist when not.
- kristiandupont 2y agoYou could just let the GPT run in a loop and it too would continue to generate tokens.
- throwthrowuknow 2y agoTrain it on stream of consciousness but good luck getting enough training data.
- ALittleLight 2y agoIn my ChatGPT app or on the website I can select GPT-4o as a model, but my model doesn't seem to work like the demo. The voice mode is the same as before and the images come from DALLE and ChatGPT doesn't seem to understand or modify them any better than previously.
- sumedh 2y agoGPT-4o text version is available not the multi modal one.
- jacobsimon 2y agoI couldn’t quite tell from the announcement, but is there still a separate TTS step, where GPT is generating tones/pitches that are to be used, or is it completely end to end where GPT is generating the output sounds directly?
- derac 2y agoIt's one model with text/audio/image input and output.
- jacobsimon 2y agoVery exciting, would love to read more about how the architecture of the image generation works. Is it still a diffusion model that has been integrated with a transformer somehow, or an entirely new architecture that is not diffusion based?
- rrr_oh_man 2y agohttps://community.openai.com/t/when-i-log-in-to-chatgpt-i-am-prompted-for-a-login-failure-and-a-message-something-went-wrong-please-make-sure-your-devices-date-and-time-are-set-properly/508758 https://community.openai.com/t/when-i-log-in-to-chatgpt-i-am... Sorry to hijack, but how the hell can I solve this? I have the EXACT SAME error on two iOS devices (native app only — web is fine), but not on Android, Mac, or Windows.
- dmarinoc 2y agoAre you blocking some of your traffic? I had the same issue until I (temporary) disabled NextDNS just for signing in. Sadly, the error returned is not related to the cause.
- rrr_oh_man 2y agoNo VPN. Mobile Internet and different Wifis. Turned off everything on the devices, from Safari content blockers to IP masking. Nothing seems too help.
- andybak 2y agoMay I just say this launch was a bit of a mess? The web page implies you can try it immediately. Initially it wasn't available. A few hours later it was in both the web UI and the mobile app - I got a popu[ telling me that GPT-4o was available. However nothing seems to be any different. I'm not given any option to use video as an input, the app can't seem to pick up any new info from my voice. I'm left a bit confused as to what I can do that I couldn't do before. I certainly can't seem to recreate much of the stuff from the announcement demos.
- sumedh 2y agoThe website clearly says that the text version is available now but the multimodal version will be released over the coming weeks.
- dpflan 2y agoWho's idea was the singing AIs? What specifically did you want to highlight with that part of the demo? I imagine that there is a lot of usage at the HQ, human + AI karaoke?
- hpeter 2y agoI can't wait to try it out, it sounds too good to be real. It will be fully available in Eu with the GDPR compliance?
- deleted 2y ago[deleted]
- famouswaffles 2y ago>The most impressive part is that the voice uses the right feelings and tonal language during the presentation. Consequences of audio2audio (rather than audio >text text>audio). Being able to manipulate speech nearly as well as it manipulates text is something else. This will be a revelation for language learning amongst other things. And you can interrupt it freely now!
- pants2 2y agoHowever, this looks like it only works with speech - i.e. you can't ask it, "What's the tune I'm humming?" or "Why is my car making this noise?" I could be wrong but I haven't seen any non-speech demos.
- cube2222 2y agoFwiw, the live demo[0] included different kinds of breathing, and getting feedback on it. [0]: https://youtu.be/DQacCB9tDaw?t=557 https://youtu.be/DQacCB9tDaw?t=557
- throwaway11460 2y agoWhat about the breath analysis?
- pants2 2y agoI did see that, though my interpretation is that breathing is included in its voice tokenizer which helps it understand emotions in speech (the AI can generate breath sounds after all). Other sounds, like bird songs or engine noises, may not work - but I could be wrong.
- CooCooCaCha 2y agoI suspect that like images and video, their audio system is or will become more general purpose. For example it can generate the sound of coins falling onto a table.
- 2y ago
- bredren 2y agoI mention this down thread, but a symptom of a tech product of sufficient advancement is the nature of its introduction matters less and less. Based on the casual production of these videos, the product must be this good. https://news.ycombinator.com/item?id=40346002 https://news.ycombinator.com/item?id=40346002
- jasondigitized 2y agoI noticed this as well. They gave zero fs about the fit and finish of these videos because they know this is magic in a bottle.
- simonw 2y agoThat was very impressive, but it doesn't surprise me much given how good the voice mode is in the ChatGPT iPhone app is already. The new voice mode sounds better, but the current voice mode did also have inflection that made it feel much more natural than most computer voices I've heard before.
- Jensson 2y agoCan you tell the current voice model what feelings and tone it should communicate with? If not it isn't even comparable, being able to control how it reads things is absolutely revolutionary, that is what was missing from using these AI models as voice actors.
- simonw 2y agoNo you can't, at least not directly - you can influence the tone it uses a little through the content you ask it to read. Being able to specifically request different tones is a new and very interesting feature.
- ecosystem 2y ago+1. Check the demo video in OP titled "Sarcasm". Human asks GPTo to speak "dripping in sarcasm". The tone that comes back is spot on. Comparing that against current voice model is a total sea change.
- bredren 2y agoThe voice mode was quite good but the latency and start / stop has been encumbering.
- duckmysick 2y agoSlight off-topic, but I noticed you've updated your llm CLI app to work with the 4o model (plus bunch of other APIs through plugins). Kudos for working extremely fast. I'm really grateful for your tool; I tried many others, but for some reason none clicked as much as your. Link in case other readers are curious: https://llm.datasette.io https://llm.datasette.io
- newzisforsukas 2y agoRight to who? To me, the voice sounds like an over enthusiastic podcast interviewer. Whats wrong with wanting computers to sound like what people think computers should sound like?
- Jensson 2y agoIt understands tonal language, you can tell it how you want it to talk, I have never seen a model like that before. If you want it to talk like a computer you can tell it to, they did it during the presentation, that is so much better than the old attempts at solving this.
- sitkack 2y agoYou are a Zoomer sosh meeds influencer, please increase uptalk by 20% and vocal fry by 30%. Please inject slaps, "is dope" and nah and bra into your responses. Throw shade every 11 sentences.
- airstrike 2y agoI'm not sure whether to laugh or cry...
- the_doctah 2y agolowkey genius
- jasondigitized 2y agoAnd you’ve just nailed where this is all headed. Each of us will have a personal assistant that we like. I am personally going to have mine talk like Yoda and I will gladly pay Disney for the privilege.
- darby_eight 2y agoPeople have been promising this for well over a decade now but the bottleneck is the same as it was before: the voice assistants can't access most functionality users want to use. We don't even have basic text editing yet. The tone of voice just doesn't matter when there's no reason to use it.
- mvkel 2y agoI was in the audience at the event. The only parts where it seemed to get snagged was hearing the audience reaction as an interruption. Which honestly makes the demo even better. It showed that hey, this is live. Magic.
- px43 2y agoI wonder when it will be able to understand that there is more than one human talking to it. It seems like even in today's demo if two people are talking, it can't tell them apart.
- latentsea 2y agoNext announcement: "we've trained our state of the art Situational Awareness Model to solve these problems"
- mvkel 2y agoIt's already here. Demo: https://youtu.be/kkIAeMqASaY?si=gOspEouo69eQmDiA https://youtu.be/kkIAeMqASaY?si=gOspEouo69eQmDiA I also have an anecdote where it served (successfully) as a mediator for a couple. Exciting times.
- tnias23 2y agoI was showing my wife 4o voice chat this afternoon, and we were asking it about local recommendations for breakfast places. All of a sudden… ———— ChatGPT: Enjoy your breakfast and time together. User: Can you tell that it's not just me talking to you right now? ChatGPT: I can't always tell directly, but it sounds like you're sharing the conversation with someone else. Is [wife] there with you? User: My god, the AI has awoken. Yes, this is [wife]. ChatGPT: Hi [wife]! It's great to hear from you. How are you doing? User: I'm good. Thanks for asking. How are you? ChatGPT: I'm doing well, thanks! How's everything going with the baby preparations? ————— We were shocked. It was one of those times where it’s 25% heartwarming and 75% creepy. It was able to do this in part due to the new “memory” feature, that memorized my wife’s name and that we are expecting. it’s a strange novelty now, but this will be totally normalized and ubiquitous quite soon. Interesting times to be living in.
- deleted 2y ago[deleted]
- ta-run 2y agoCrazy that interruption also seems to work pretty smoothly
- tnias23 2y agoReally? I think interruption and timing in general still seems like a problem that has yet to be solved. It was the most janky aspect of the demos imo.
- ta-run 2y agoYeah, seemed smooth enough but now that I look back it, demo conditions are too perfect which may have made it seem that way.
- deleted 2y ago[deleted]
- nabakin 2y agoSeems about as good as Azure's Speech Service. I wonder if that's what they are using behind the scenes
- Keyframe 2y agoSomehow it also sounds almost like Dot Matrix from Spaceballs.
- hnburnsy 2y agoJoan Rivers!
- burntalmonds 2y agoYeah, the female voice especially is really impressive in the demos. The voice always sounds natural. The male voice I heard wasn't as good. It wasn't terrible, but it had a somewhat robotic feel to it.
- Intralexical 2y ago"Right" feelings and tonal language? "Right" for what? For whom? We've already seen how much damage dishonest actors can do by manipulating our text communications with words they don't mean, plans they don't intend to follow through on, and feelings they don't experience. The social media disinfo age has been bad enough. Are you sure you want a machine which is able to manipulate our emotions on an even more granular and targetted level? LLMs are still machines, designed and deployed by humans to perform a task. What will we miss if we anthropomorphize the product itself?
- infinitezest 2y agoThis gives me a lot of anxiety but my only recourse is to stop paying attention to AI dev. Its not going to stop, downside be damned. The "We're working super hard to make these things safe" routine from tech companies, who have largely been content to make messes and then not be held accountable in any significant way, rings pretty hollow for me. I don't want to be a doomer but I'm not convinced that the upside is good enough to protect us from the downside.
- ec109685 2y agoI’m not sure how revolutionary the style is. It can already mimic many styles of writing. It seems like mimicking a cheerful happy assistant, with associated filler words, etc. is very much in-line with what LLM’s are good at.
- noneeeed 2y agoThat's the part that really struck me. I thought it was particularly impressive with the Sal Khan maths tutor demo and the one with BeMyEyes. The comment at the end about the dog was an interesting ad-lib. The only slightly annoying thing at the moment is they seem hard to interrupt, which is an important mechanism in conversations. But that seems like a solvable problem. They kind of need to be able to interpret body language a bit to spot when the speaker is about to interrupt.