7 ms·

Very interesting and extremely impressive! I tried using the voice chat in their app previously and was disappointed. The big UX problem was that it didn't try

by Hugsun 2y ago

Very interesting and extremely impressive! I tried using the voice chat in their app previously and was disappointed. The big UX problem was that it didn't try to understand when I had finished speaking. English is a second language and I paused a bit too long thinking of a word and it just started responding to my obviously half spoken sentence. Trying again it just became stressful as I had to rush my words out to avoid an annoying response to an unfinished thought. I didn't try interrupting it but judging by the comments here it was not possible. It was very surprising to me to be so overtly exposed to the nuances of real conversation. Just this one thing of not understanding when it's your turn to talk made the interaction very unpleasant, more than I would have expected. On that note, I noticed that the AI in the demo seems to be very rambly. It almost always just kept talking and many statements were reiterations of previous ones. It reminded me of a type of youtuber that uses a lot of filler phrases like "let's go ahead and ...", just to be more verbose and lessen silences. Most of the statements by the guy doing the demo were interrupting the AI. It's still extremely impressive but I found this interesting enough to share. It will be exciting to see how hard it is to reproduce these abilities in the open, and to solve this issue.

luminen 2y ago
"I paused a bit too long thinking of a word and it just started responding to my obviously half spoken sentence. Trying again it just became stressful as I had to rush my words out to avoid an annoying response to an unfinished thought." I'm a native speaker and this was my experience as well. I had better luck manually sending the message with the "push to hold" button.
- dionian 2y ago
  Same. It cuts off really quick when I'm trying to phrase a complex prompt and I wind up losing my work. So I go to the manual dictation mode, which works really nicely but doesnt give me the hands off mode. I admit, the hands off mode is often better for simpler interactions, but even then, I am frequently cut off mid-prompt.
ijidak 2y ago
> I noticed that the AI in the demo seems to be very rambly I know this is a serious conversation, but when the presenters had to cut it off, I got flashbacks to Data in Star Trek TNG!! And 3PO in Star Wars! Human: "Shut up" Robot: "Shutting up sir" Turns out rambling AI was an accurate prediction!
- yreg 2y ago
  There needs to be an override for this. When you tell Siri to shut up, it either apologizes or complains about your behaviour. When you tell Alexa to shut up, it immediately goes silent. I prefer the latter when it comes to computers.
- Hugsun 2y ago
  That's funny and a very good point. It also tangentially reminds me of an excellent video I re-watched recently called The Tragedy of Droids[1]. I highly recommend it. It raises interesting moral questions about the nature of droids in the star wars universe. [1]: https://youtu.be/WD2UrB7zepo https://youtu.be/WD2UrB7zepo
yreg 2y ago
I have the same ESL UX problem with all the AI assistants. I do my work in english and talk to people just fine, but with machines it's usually awkward for me. Also on your other note (demo seems to be very rambly), it bothered me as well. I don't want the AI to continue speaking, while having nothing to say until I interrupt it. Be brief. That can be solved through prompts at least.
obmelvin 2y ago
This was such a huge issue for me and made it very difficult to use conversation mode to practice any sort of professional interaction or presentation
hbn 2y ago
> I noticed that the AI in the demo seems to be very rambly That's been a major issue for me with LLMs this whole time. They can't just give me an answer, they have to spout a whole preamble that usually includes spitting my query back at me and trying to impress me with how many words it's saying, like it's a requirement. You can tell it e.g. "don't give me anything other than the list" but it's annoying to have to ask it every time. Every AI chat needs a "no yap mode"
- ianbicking 2y ago
  They really need a "hidden yap" mode. LLMs perform better on difficult questions or interactions when they have "room to think". An introductory paragraph is like that, and it's as much for the LLM to form its thoughts as it's for the user. But for all that the intro paragraph doesn't have to be _read_ by the user, it just has to be emitted by GPT and put into the transcript.
  - Hugsun 2y ago
    That's very true. I believe it's a variation of chain of thought prompting they're doing. ChatGPT seems trained to do this for one.
- jabits 2y ago
  I’m kind of glad it does this so I know that it understood what I asked. A good presenter will do this as well when responding to questions.
  - Hugsun 2y ago
    I almost always find it too verbose and unnaturally mimicy when bouncing the question back. It doesn't paraphrase my request. It's more like restating it. What I notice most is that almost always repeats verbatim unnaturally long parts of my requests. This might be more useful to people that do lazy prompting. My nature compels me to be clear and specific in all written text.
- PoignardAzur 2y ago
ianbicking 2y ago
This makes me wonder: if you tell 4o something like "listen to me until I say all done" will it be able to suppress its output until it hears that? I'm guessing not quite possible now, just because I'm guessing patiently waiting is a different band of information that they haven't implemented. But I really don't know.
- Hugsun 2y ago
  I'd bet it can't do it now. I'd be curious to hear what it says in response. Partially because it requires time dependent reasoning about being a participant in a conversation. It shouldn't be too hard to make this work though. If you make the AI start by emitting either a "my turn to talk" or "still listening" token it should be able to listen patiently. If trained correctly.
- tr3ntg 2y ago
  I don’t think so. The listening and responding isn’t being managed by an LLM. That’s just the app listening with a microphone and timer. Stop talking for x sec = process response. I bet the bot would wholeheartedly agree that it would definitely wait for you to finish talking, then just not do it. It doesn’t know anything about the app it’s “in.” At least at a deep level. I agree that all this is impressive, but with odd, unclear bounds that sometimes confuse users.
sznio 2y ago
> many statements were reiterations of previous ones SEO spam got into the training data.
- chaorace 2y ago
  The funny thing actually is that repeating yourself is a really important communication skill. The model seems to have internalized that, but isn't yet quite at the level where it properly understands why/when to repeat. It's probably related to GPT's more general sycophant inclinations. Acting like a doormat is apparently easier to teach than nuanced politeness -- much in the same way that repeating yourself ad nauseum is easier than intuiting specific points of emphasis.
zdko 2y ago
FYI: In the current voice chat available in their app, if you press and hold the white circle, it will listen for you until you lift your finger. This pretty much fixed my issues with it!

by Hugsun 2y ago

luminen 2y ago

"I paused a bit too long thinking of a word and it just started responding to my obviously half spoken sentence. Trying again it just became stressful as I had to rush my words out to avoid an annoying response to an unfinished thought." I'm a native speaker and this was my experience as well. I had better luck manually sending the message with the "push to hold" button.

dionian 2y ago
Same. It cuts off really quick when I'm trying to phrase a complex prompt and I wind up losing my work. So I go to the manual dictation mode, which works really nicely but doesnt give me the hands off mode. I admit, the hands off mode is often better for simpler interactions, but even then, I am frequently cut off mid-prompt.

ijidak 2y ago

> I noticed that the AI in the demo seems to be very rambly I know this is a serious conversation, but when the presenters had to cut it off, I got flashbacks to Data in Star Trek TNG!! And 3PO in Star Wars! Human: "Shut up" Robot: "Shutting up sir" Turns out rambling AI was an accurate prediction!

yreg 2y ago
There needs to be an override for this. When you tell Siri to shut up, it either apologizes or complains about your behaviour. When you tell Alexa to shut up, it immediately goes silent. I prefer the latter when it comes to computers.
Hugsun 2y ago
That's funny and a very good point. It also tangentially reminds me of an excellent video I re-watched recently called The Tragedy of Droids[1]. I highly recommend it. It raises interesting moral questions about the nature of droids in the star wars universe. [1]: https://youtu.be/WD2UrB7zepo https://youtu.be/WD2UrB7zepo

yreg 2y ago

I have the same ESL UX problem with all the AI assistants. I do my work in english and talk to people just fine, but with machines it's usually awkward for me. Also on your other note (demo seems to be very rambly), it bothered me as well. I don't want the AI to continue speaking, while having nothing to say until I interrupt it. Be brief. That can be solved through prompts at least.

obmelvin 2y ago

This was such a huge issue for me and made it very difficult to use conversation mode to practice any sort of professional interaction or presentation

hbn 2y ago

> I noticed that the AI in the demo seems to be very rambly That's been a major issue for me with LLMs this whole time. They can't just give me an answer, they have to spout a whole preamble that usually includes spitting my query back at me and trying to impress me with how many words it's saying, like it's a requirement. You can tell it e.g. "don't give me anything other than the list" but it's annoying to have to ask it every time. Every AI chat needs a "no yap mode"

ianbicking 2y ago
They really need a "hidden yap" mode. LLMs perform better on difficult questions or interactions when they have "room to think". An introductory paragraph is like that, and it's as much for the LLM to form its thoughts as it's for the user. But for all that the intro paragraph doesn't have to be _read_ by the user, it just has to be emitted by GPT and put into the transcript.
- Hugsun 2y ago
  That's very true. I believe it's a variation of chain of thought prompting they're doing. ChatGPT seems trained to do this for one.
jabits 2y ago
I’m kind of glad it does this so I know that it understood what I asked. A good presenter will do this as well when responding to questions.
- Hugsun 2y ago
  I almost always find it too verbose and unnaturally mimicy when bouncing the question back. It doesn't paraphrase my request. It's more like restating it. What I notice most is that almost always repeats verbatim unnaturally long parts of my requests. This might be more useful to people that do lazy prompting. My nature compels me to be clear and specific in all written text.
PoignardAzur 2y ago

ianbicking 2y ago

This makes me wonder: if you tell 4o something like "listen to me until I say all done" will it be able to suppress its output until it hears that? I'm guessing not quite possible now, just because I'm guessing patiently waiting is a different band of information that they haven't implemented. But I really don't know.

Hugsun 2y ago
I'd bet it can't do it now. I'd be curious to hear what it says in response. Partially because it requires time dependent reasoning about being a participant in a conversation. It shouldn't be too hard to make this work though. If you make the AI start by emitting either a "my turn to talk" or "still listening" token it should be able to listen patiently. If trained correctly.
tr3ntg 2y ago
I don’t think so. The listening and responding isn’t being managed by an LLM. That’s just the app listening with a microphone and timer. Stop talking for x sec = process response. I bet the bot would wholeheartedly agree that it would definitely wait for you to finish talking, then just not do it. It doesn’t know anything about the app it’s “in.” At least at a deep level. I agree that all this is impressive, but with odd, unclear bounds that sometimes confuse users.

sznio 2y ago

> many statements were reiterations of previous ones SEO spam got into the training data.

chaorace 2y ago
The funny thing actually is that repeating yourself is a really important communication skill. The model seems to have internalized that, but isn't yet quite at the level where it properly understands why/when to repeat. It's probably related to GPT's more general sycophant inclinations. Acting like a doormat is apparently easier to teach than nuanced politeness -- much in the same way that repeating yourself ad nauseum is easier than intuiting specific points of emphasis.

zdko 2y ago

FYI: In the current voice chat available in their app, if you press and hold the white circle, it will listen for you until you lift your finger. This pretty much fixed my issues with it!