6 ms·

OpenAI, Jony Ive struggle with technical details on secretive new AI gadget

32 points by apparent 1y ago

duxup 1y ago
>One issue is ensuring the device only chimes in when useful, preventing it from talking too much or not knowing when to finish the conversation—an ongoing issue with ChatGPT. This device that just sits and listens all the time and interrupts ... that's nothing like I use LLMs now. That's a completely different mode of operation / my tolerance for mistakes would be near 0 with that. It's not clear to me if an LLM can even DO these things conversationally well enough to work / not be a nuisance. Siri might be the best example, you use it a few times, it's useless a few times and then I just use it to set timers and forget about it otherwise.
- apparent 1y ago
  I still can't understand my my HomePod is constantly responding (uselessly, often just saying it can't send a message for me) when I am talking to my phone that is in my hand. It even happens when I have AirPods in my ears. How hard is it to have a priority listing for: 1: AirPods-linked devices go first when AirPods are present 2: closest device goes first in general 3: if there are multiple devices that could respond, and only some of them can do the requested task, one of the devices that can do the task should respond
  - dimal 1y ago
    I had the same problem but in reverse. I’d want to play a song on the HomePod but my iPad sitting on the coffee table would start playing it on its tinny little speaker. The solution was to set Siri on the devices to not respond to “Siri”. To use Siri on the handheld devices I press and hold the side button until I get the ugly gradient border. It’s pretty bad that Apple released this feature in such a uselessly broken state.
  - bookofjoe 1y ago
    Ditto. I tell my Apple Watch to set an alarm and my HomePod — in another room, 30 feet away — says "I've set your alarm." FAIL
  - deleted 1y ago
    [deleted]
- danielfoster 1y ago
  It sounds like this device could become the iPod Shuffle of AI devices— cute and clever, but not particularly useful.
- estimator7292 1y ago
  Just imagine the unlimited utility of your device pinging at random points throughout the day to declare "Ah, this is a classic problem with <SUBJECT>" who wouldn't need that?
  - rsynnott 1y ago
    See, this is the sort of thing that makes me suspect that maybe I'm in hell (in the Good Place or Iain Banks' Surface Detail mode rather than the devils-poking-people-with-pitchforks one); that, as a concept, could only possibly exist to torture people.
- rsynnott 1y ago
  > and then I just use it to set timers and forget about it otherwise. Much as the Apple Watch, in practice, was really a good fitness tracker, and kinda pointless for everything else, the natural place for voice assistant gadgets is setting timers. I'm fairly sure that's all that 99% of Alexa users use their talking cylinder for, say. Also, isn't this just that Humane.ai gadget? And there was another one, IIRC. They were all the rage as CES 2024 IIRC.
bko 1y ago
> One person said the device would be “always on” rather than triggered by a word or prompt. The device’s sensors would gather data throughout the day that would help to build its virtual assistant’s “memory.” That sounds like a terrible idea. It's hard to say, but I like to think that something like the iPhone always sounded like a good idea. Basically, the entire web in your pocket. Check football scores while you're on the john, text someone instead of calling, GPS, etc. There were some unknowns, like can we get touch screen to work well enough, will people pay $600 for a phone when typical phones cost $100-200 at the time. But the product itself was obviously good. But this is some puck that records everything and periodically chimes in? Sounds awful. What problem is this trying to solve? iPhone solved the problem of camera, entertainment and phone in one device. Is anyone walking around thinking gee I wish I had some AI talk to me right now but I don't want to press a button?
- iamleppert 1y ago
  How on earth are they going to manage battery life for an always on device? At this point they might as well just rig up a cheap 2-way short range radio, with a microphone always on and connected to OpenAI HQ.
- ratg13 1y ago
  I think advertising is the problem being solved. If they can make it remotely useful enough to allow you to record all of your conversations, that is extremely valuable to advertisers.
- scuff3d 1y ago
  And people thought MS Recall was bad.
- snowwrestler 1y ago
  Here’s an example of an AI that interrupts people: turn-by-turn directions. The interruptions are tolerated because the info is so timely and useful. Heck, drivers will shush friends and family to hear the AI instead. So that gives you a sense for the bar for other AI interruptions. It had better be very timely and very valuable if I’m going to accept an AI interruption. That probably means the interruptions are rare in general, at least at first. The iPhone was popular from the moment it was introduced, but people back then did not really think about it the way we do now. The biggest cheers were for a phone (one big demo was voicemail LOL), and a “widescreen iPod.” There was tepid confused applause for it as a “break-through Internet communications device.” Which is 99% of how everyone uses it today.
flave 1y ago
> struggling with […] privacy Yeah I can imagine that a device which records everyone around you, all the time, struggles with that.
joshstrange 1y ago
> whose alluring designs of the iMac, iPod, and iPhone helped turn Apple into one of the most valuable companies in the world "Helped" is doing a lot of heavy lifting in that sentence. I'm still not sure if those these things did well because of Jony or due to Steve. What we saw from Jony after Steve (few ports, butterfly keyboard, thinness at the expense of everything else) makes me think that without a good editor (of which Altman may or may not be) Jony isn't able to create greatness on his own. I also find Jony's ventures post-Apple to be laughable and not interesting in the slightest. > “The concept is that you should have a friend who’s a computer who isn’t your weird AI girlfriend... like [Apple’s digital voice assistant] Siri but better,” Uh, ok, that sentence took a weird turn. I'm not sure if anyone has ever called Siri a "weird AI girlfriend", I've never seen it marketed that way or people using it that way. Now, the OpenAI API, on the other hand, has clearly helped enable whole slews of AI Girl/boyfriends. Odd dig to throw at Apple here. I'm sure whatever they unveil will be interesting, just not clear if it will have the same fate as R1/Humane/Friend. Personally, I hope they fail based on the video they released when OpenAI first bought "io", cringe does not begin to describe it.
- intothemild 1y ago
  I really don't get the cult of Jony. As you touched upon, without Steve, the design decisions were baffling.
  - alwahi 1y ago
    but have you considered the trashcan mac?
- turtlebits 1y ago
  If not the designer and/or CEO, who are you going to give credit to?
  - joshstrange 1y ago
    I do give credit to Steve Jobs. I was saying that what Jony has done on his own is less impressive IMHO. If they manage to pull this off (OpenAI/io) then I will, begrudgingly, have to give credit to Altman, despite not being a huge fan of him (then again, Jobs wasn't a "good guy" in many aspect of his life either). My point being, Jony needs an editor, when he was unchecked at Apple (or his own design company "Love From") I found that much of his output came across as an exercise in smelling one’s own farts.
bonthron 1y ago
is it weird they're running TV commercial with Jony Ive even though the product doesn't exist yet?
grim_io 1y ago
There is no hardware format I can imagine that would make sense for something like this. Maybe that's just my limited imagination, but since no one has made anything useful in this space...
- ninininino 1y ago
  If drones weren't incredibly noisy and battery wasn't so scarce, a floating personal assistant orb w/ video and mic input and a speaker/audio output, sort of a modern techno take on the fairy Navi from the series Zelda would be great.
  - walthamstow 1y ago
    The insta360 content-creator camera kind of functions like this but, instead of floating, people hold it away from their body on a stick all day
  - mrguyorama 1y ago
    The fairy Navi who is famously annoying and who everyone wishes they could literally murder?
- claudiulodro 1y ago
  A fob on a lanyard would work well I think. It would be nerdy as hell, but in a practical sense, it's better than carrying around another phone-like thing or wearing something on your face.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
greatgib 1y ago
Looks like the usual fad that never will never pass the reality check. It could have been great for them to work on something like smart glasses, a competitor to Meta one with AI. Or a nice AI super-smart watch.
yahoozoo 1y ago
This is going to be dead on arrival.
- bookofjoe 1y ago
  Before arrival.
constantcrying 1y ago
Screens are easily the most useful way to communicate complex information. Why would getting rid of them be beneficial at all? What does the device want to accomplish that something with a screen couldn't immediately do better? Voice is inherently linear, even the most basic voice only menus are tedious to navigate.
jonplackett 1y ago
> Their aim is to create a palm-sized device without a screen that can take audio and visual cues from the physical environment and respond to users’ requests So… a ‘phone’ - but no screen.
StopDisinfo910 1y ago
This "article" is a collection of vague quotes from "one person" supposedly "briefed on the plan" about what an hypothetical device in development is allegedly doing. That’s pretty thin.
more_corn 1y ago
Holy crap! Hugely overhyped and oberpriced AI pin thing like the last ai pin thing but better because different, faces all the same limitations of old pin thing just like every sane person predicted. So crazy how this works.
alwahi 1y ago
why could this gadgets functionality not be integrated into the chatgpt app? im asking rhetorically.
rsynnott 1y ago
> Multiple people familiar with the plans said OpenAI and Ive were working on a device roughly the size of a smartphone that users would communicate with through a camera, microphone and speaker. One person suggested it might have multiple cameras. Finally, someone has, er, invented the smartphone? Except without the most useful bit? Why do people keep trying to make special-purpose LLM gadgets? What on earth is wrong with just having an app on a phone?
browningstreet 1y ago
It feels like there's nothing really new to say about what's proposed in this report that hasn't already been said about the Humane, Friend, and R1 except "except Sam & Jony".

32 points by apparent 1y ago

>One issue is ensuring the device only chimes in when useful, preventing it from talking too much or not knowing when to finish the conversation—an ongoing issue with ChatGPT. This device that just sits and listens all the time and interrupts ... that's nothing like I use LLMs now. That's a completely different mode of operation / my tolerance for mistakes would be near 0 with that. It's not clear to me if an LLM can even DO these things conversationally well enough to work / not be a nuisance. Siri might be the best example, you use it a few times, it's useless a few times and then I just use it to set timers and forget about it otherwise.

apparent 1y ago
I still can't understand my my HomePod is constantly responding (uselessly, often just saying it can't send a message for me) when I am talking to my phone that is in my hand. It even happens when I have AirPods in my ears. How hard is it to have a priority listing for: 1: AirPods-linked devices go first when AirPods are present 2: closest device goes first in general 3: if there are multiple devices that could respond, and only some of them can do the requested task, one of the devices that can do the task should respond
- dimal 1y ago
  I had the same problem but in reverse. I’d want to play a song on the HomePod but my iPad sitting on the coffee table would start playing it on its tinny little speaker. The solution was to set Siri on the devices to not respond to “Siri”. To use Siri on the handheld devices I press and hold the side button until I get the ugly gradient border. It’s pretty bad that Apple released this feature in such a uselessly broken state.
- bookofjoe 1y ago
  Ditto. I tell my Apple Watch to set an alarm and my HomePod — in another room, 30 feet away — says "I've set your alarm." FAIL
- deleted 1y ago
  [deleted]
danielfoster 1y ago
It sounds like this device could become the iPod Shuffle of AI devices— cute and clever, but not particularly useful.
estimator7292 1y ago
Just imagine the unlimited utility of your device pinging at random points throughout the day to declare "Ah, this is a classic problem with <SUBJECT>" who wouldn't need that?
- rsynnott 1y ago
  See, this is the sort of thing that makes me suspect that maybe I'm in hell (in the Good Place or Iain Banks' Surface Detail mode rather than the devils-poking-people-with-pitchforks one); that, as a concept, could only possibly exist to torture people.
rsynnott 1y ago
> and then I just use it to set timers and forget about it otherwise. Much as the Apple Watch, in practice, was really a good fitness tracker, and kinda pointless for everything else, the natural place for voice assistant gadgets is setting timers. I'm fairly sure that's all that 99% of Alexa users use their talking cylinder for, say. Also, isn't this just that Humane.ai gadget? And there was another one, IIRC. They were all the rage as CES 2024 IIRC.

bko 1y ago

> One person said the device would be “always on” rather than triggered by a word or prompt. The device’s sensors would gather data throughout the day that would help to build its virtual assistant’s “memory.” That sounds like a terrible idea. It's hard to say, but I like to think that something like the iPhone always sounded like a good idea. Basically, the entire web in your pocket. Check football scores while you're on the john, text someone instead of calling, GPS, etc. There were some unknowns, like can we get touch screen to work well enough, will people pay $600 for a phone when typical phones cost $100-200 at the time. But the product itself was obviously good. But this is some puck that records everything and periodically chimes in? Sounds awful. What problem is this trying to solve? iPhone solved the problem of camera, entertainment and phone in one device. Is anyone walking around thinking gee I wish I had some AI talk to me right now but I don't want to press a button?

iamleppert 1y ago
How on earth are they going to manage battery life for an always on device? At this point they might as well just rig up a cheap 2-way short range radio, with a microphone always on and connected to OpenAI HQ.
ratg13 1y ago
I think advertising is the problem being solved. If they can make it remotely useful enough to allow you to record all of your conversations, that is extremely valuable to advertisers.
scuff3d 1y ago
And people thought MS Recall was bad.
snowwrestler 1y ago
Here’s an example of an AI that interrupts people: turn-by-turn directions. The interruptions are tolerated because the info is so timely and useful. Heck, drivers will shush friends and family to hear the AI instead. So that gives you a sense for the bar for other AI interruptions. It had better be very timely and very valuable if I’m going to accept an AI interruption. That probably means the interruptions are rare in general, at least at first. The iPhone was popular from the moment it was introduced, but people back then did not really think about it the way we do now. The biggest cheers were for a phone (one big demo was voicemail LOL), and a “widescreen iPod.” There was tepid confused applause for it as a “break-through Internet communications device.” Which is 99% of how everyone uses it today.

flave 1y ago

> struggling with […] privacy Yeah I can imagine that a device which records everyone around you, all the time, struggles with that.

joshstrange 1y ago

> whose alluring designs of the iMac, iPod, and iPhone helped turn Apple into one of the most valuable companies in the world "Helped" is doing a lot of heavy lifting in that sentence. I'm still not sure if those these things did well because of Jony or due to Steve. What we saw from Jony after Steve (few ports, butterfly keyboard, thinness at the expense of everything else) makes me think that without a good editor (of which Altman may or may not be) Jony isn't able to create greatness on his own. I also find Jony's ventures post-Apple to be laughable and not interesting in the slightest. > “The concept is that you should have a friend who’s a computer who isn’t your weird AI girlfriend... like [Apple’s digital voice assistant] Siri but better,” Uh, ok, that sentence took a weird turn. I'm not sure if anyone has ever called Siri a "weird AI girlfriend", I've never seen it marketed that way or people using it that way. Now, the OpenAI API, on the other hand, has clearly helped enable whole slews of AI Girl/boyfriends. Odd dig to throw at Apple here. I'm sure whatever they unveil will be interesting, just not clear if it will have the same fate as R1/Humane/Friend. Personally, I hope they fail based on the video they released when OpenAI first bought "io", cringe does not begin to describe it.

intothemild 1y ago
I really don't get the cult of Jony. As you touched upon, without Steve, the design decisions were baffling.
- alwahi 1y ago
  but have you considered the trashcan mac?
turtlebits 1y ago
If not the designer and/or CEO, who are you going to give credit to?
- joshstrange 1y ago
  I do give credit to Steve Jobs. I was saying that what Jony has done on his own is less impressive IMHO. If they manage to pull this off (OpenAI/io) then I will, begrudgingly, have to give credit to Altman, despite not being a huge fan of him (then again, Jobs wasn't a "good guy" in many aspect of his life either). My point being, Jony needs an editor, when he was unchecked at Apple (or his own design company "Love From") I found that much of his output came across as an exercise in smelling one’s own farts.

bonthron 1y ago

is it weird they're running TV commercial with Jony Ive even though the product doesn't exist yet?

grim_io 1y ago

There is no hardware format I can imagine that would make sense for something like this. Maybe that's just my limited imagination, but since no one has made anything useful in this space...

ninininino 1y ago
If drones weren't incredibly noisy and battery wasn't so scarce, a floating personal assistant orb w/ video and mic input and a speaker/audio output, sort of a modern techno take on the fairy Navi from the series Zelda would be great.
- walthamstow 1y ago
  The insta360 content-creator camera kind of functions like this but, instead of floating, people hold it away from their body on a stick all day
- mrguyorama 1y ago
  The fairy Navi who is famously annoying and who everyone wishes they could literally murder?
claudiulodro 1y ago
A fob on a lanyard would work well I think. It would be nerdy as hell, but in a practical sense, it's better than carrying around another phone-like thing or wearing something on your face.

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

greatgib 1y ago

Looks like the usual fad that never will never pass the reality check. It could have been great for them to work on something like smart glasses, a competitor to Meta one with AI. Or a nice AI super-smart watch.

yahoozoo 1y ago

This is going to be dead on arrival.

bookofjoe 1y ago
Before arrival.

constantcrying 1y ago

Screens are easily the most useful way to communicate complex information. Why would getting rid of them be beneficial at all? What does the device want to accomplish that something with a screen couldn't immediately do better? Voice is inherently linear, even the most basic voice only menus are tedious to navigate.

jonplackett 1y ago

> Their aim is to create a palm-sized device without a screen that can take audio and visual cues from the physical environment and respond to users’ requests So… a ‘phone’ - but no screen.

StopDisinfo910 1y ago

This "article" is a collection of vague quotes from "one person" supposedly "briefed on the plan" about what an hypothetical device in development is allegedly doing. That’s pretty thin.

more_corn 1y ago

Holy crap! Hugely overhyped and oberpriced AI pin thing like the last ai pin thing but better because different, faces all the same limitations of old pin thing just like every sane person predicted. So crazy how this works.

alwahi 1y ago

why could this gadgets functionality not be integrated into the chatgpt app? im asking rhetorically.

rsynnott 1y ago

> Multiple people familiar with the plans said OpenAI and Ive were working on a device roughly the size of a smartphone that users would communicate with through a camera, microphone and speaker. One person suggested it might have multiple cameras. Finally, someone has, er, invented the smartphone? Except without the most useful bit? Why do people keep trying to make special-purpose LLM gadgets? What on earth is wrong with just having an app on a phone?

browningstreet 1y ago

It feels like there's nothing really new to say about what's proposed in this report that hasn't already been said about the Humane, Friend, and R1 except "except Sam & Jony".