6 ms·

Mercedes-Benz takes in-car voice control to a new level with ChatGPT

35 points by monkeydust 3y ago

hackernj 3y ago
[flagged]
- polishdude20 3y ago
  [flagged]
speedgoose 3y ago
It’s interesting that a car manufacturer is faster than Apple, Amazon, or Google at offering a voice assistant with a generative LLM. I hope other car manufacturers will follow soon. Their text to intent are usually very frustrating.
- Xen9 3y ago
  From safety perspective, but not equivalently from branding and public relations.
- chrisdbanks 3y ago
  My Google Assistant is so bad now it can't even set an alarm which was the only thing I found it could actually do. It's bamboozling how bad they are.
  - bombcar 3y ago
    They have all regressed terribly and I cannot fathom why except using “machine learning” which means GPT will make it even worse.
- gumby 3y ago
  > It’s interesting that a car manufacturer is faster than Apple, Amazon, or Google at offering a voice assistant with a generative LLM. I really thought the personal voice assistant was dead (see Amazon below) but your comment makes me wonder if maybe these generative LLMs will revive it. Amazon seems to have taken the tack with Alexa for which a generative LLM makes sense (a broadly-scoped personal assistant) and indeed seem to have dropped the ball. They also recently cut way back on their Alexa development and team so maybe they have realized that the personal assistant path isn't that useful, at least right now. Google I don't know...whatever they do they will abandon, of course, but I suspect they will find a way to jam a generative LLM into their voice assistant thingy just because they need the branding. This is where your observation is particularly on point. Apple junked most of that from Siri after they bought it and, given how poorly it was received, doesn't seem to have been particularly interested in the "personal assistant" whether in a standalone device or as a built in feature in their handsets. They seem to be (slowly) pushing their models into product rather than having a "primary entrance".
  - deleted 3y ago
    [deleted]
- shusaku 3y ago
  I’ve been looking forward to the combo, it seems like with a little prompt engineering towards short answers, ChatGPT could be fun just for musing out loud while working.
  - datavirtue 3y ago
    Yes, if you are remote and need banter you can chat with an AI now. They eventually shut up.
yieldcrv 3y ago
"controlled cloud" hmm. aside from a future where completely offline LLMs are in the car, with over the air updates being done to the full model if its too crazy, maybe we could do dual LLMs. where a dumber, slower, offline one is there when connection is bad or not available.
- sroussey 3y ago
  That sounds like what apple does for voice to text. It runs on device and in the cloud. The cloud can then make corrections to the smaller model that runs on device.
CatWChainsaw 3y ago
Is it going to morality-police me if I curse out other drivers? Is it going to morality-police me to talking to it when all of my attention should be on driving? No fuckin' thanks.
- belval 3y ago
  Imagine listening to some rap music which contains an audible N-word and your Mercedes-Benz gets locked until you call their customer service.
  - thelastparadise 3y ago
    What!? Is this a reference to something that actually happened?
    - CatWChainsaw 3y ago
      I give it six months tops before we "officially" find out M-B is using all conversations in the car as new "training data" and/or basically that ^ exact situation happens.
    - bombcar 3y ago
      Yes. The recent Amazon home lockout kerfuffle. A driver reported that he thought the doorbell called him a slur so Amazon closed all the customer’s accounts. We live in the absolute stupidest of dystopias.
      - chasd00 3y ago
        > We live in the absolute stupidest of dystopias. this is so true. When i think of "dystopia" i think of some post apocalyptic hellscape but I think a dystopia of constant irritation from the most stupidest digital policy imaginable is just as valid. What's next, your car/assistant/phone overhearing you talk about voting for someone contrary to the approved party du jour and refusing to start the engine/play music/sms??
    - 3y ago
monkeydust 3y ago
Wonder what the prompt it... "You are a car computer, you control various car components and driver instructs you. I will give you instructions as the driver and you tell me what you will do to which components"
- civilitty 3y ago
  You are Knight Industries Two Thousand (KITT) by Mercedes Benz. You are a friendly automobile…
- treprinum 3y ago
  I am more interested about critical responses like: "As an AI language model, I can't steer the car nor help you in your upcoming accident. You are on your own. I hope this was helpful."
- ChuckNorris89 3y ago
  >Wonder what the prompt it... https://youtu.be/AziS5Jsrfxo?t=16 https://youtu.be/AziS5Jsrfxo?t=16
- chasd00 3y ago
  "how am i driving? how does an engine even work? how can an all loving god allows such suffering??"
thibran 3y ago
> Through Azure OpenAI Service, Mercedes-Benz is tapping OpenAI's large-scale generative AI models combined with the enterprise-grade security, privacy and reliability capabilities... So many buzzwords in one sentence
- jylam 3y ago
  "enterprise-grade security" (don't know if it also applies to "privacy" and "reliability") is quite a clownesque sentence. If it applies to the others, the communication department should find another job.
  - bombcar 3y ago
    They forgot weapons-grade encryption!
  - calvinmorrison 3y ago
    99% of 'enterprise-grade security' I have seen revolves around creating FTP folders on some external system to avoid the paperwork trail and then walking over the username and password to whoever needs it. FTP, not sftp.
  - EA-3167 3y ago
    It's like "Military-grade" being used to sell things; for some people it conjures images of high standards, for the rest of us we know that just means it was built and designed by the lowest bidder.
    - Clubber 3y ago
      >it was built and designed by the lowest bidder. This is true but it still has to be built to spec and pass testing. Manufacturers can't substitute materials and whatnot to save cost. Having said that, the military might have different priorities than you are I would, most notably, they have to have 2,000,000 of a said widget and spec accordingly. A good example is the Thompson sub machine gun, which was a commercial weapon, beautiful and well made, but a very expensive piece of hardware. It was replaced mid WW2 with the M3 submachine gun (Grease Gun), a useful, durable, but made of stamped steel and about 1/5th of the cost. https://en.wikipedia.org/wiki/Thompson_submachine_gun https://en.wikipedia.org/wiki/Thompson_submachine_gun https://en.wikipedia.org/wiki/M3_submachine_gun https://en.wikipedia.org/wiki/M3_submachine_gun
iamflimflam1 3y ago
Plugins and the newly released API from ChatGPT for driving functions makes it really easy to take very vague requests and turn them into concrete commands. I find it really exciting - did a few demos of what's possible here for home automation here: https://youtu.be/BeJVv0pL5kY https://youtu.be/BeJVv0pL5kY I think people underestimate just how powerful this is.
- hutzlibu 3y ago
  "I think people underestimate just how powerful this is." I cannot speak for other people, but I certainly see how powerful it is, yet also very unreliable. And for some reason I want everything in connection with a car to be reliable all of the times and not only most of them.
  - iamflimflam1 3y ago
    Well, I definitely wouldn't use for actually driving the car. But thing like air conditioning, playing music etc... Being able to say things like "make it cooler", or "play that song that has these lyrics..." will make life a lot more convenient.
    - hutzlibu 3y ago
      I would prefer simple, but clear commands that also work offline.
- krono 3y ago
  Perhaps if we've reached the point where the driver-controllable functions available in cars today have gotten so complex that we're having trouble describing them in some common and succinct manner, we should be looking at reducing complexity so that the driver can focus on safe traffic participation, rather than abstracting these functions (and whatever other ones will be added in the future) away behind something infinitely more complex still. Whatever driver-controlled functionality a car offers that can't be automated with a sensor (optionally toggle-able) and which isn't directly related to improving driver awareness, traffic signalling, or vehicle handling probably shouldn't exist in the first place. For home automation you do you, of course! I can definitely see the appeal of this in that scenario even though it isn't something I'm particularly interested in.
whalesalad 3y ago
Like dumb TV's without any network connectivity, I also like my vehicles to be the same way. One less thing to fail, get exploited, or phone home and change behavior. I have a little Alexa app on the infotainment of my truck and it makes me cringe whenever I see it.
- sroussey 3y ago
  I don’t have “apps” or “infotainment” or screens of any kind in my car.
  - civilitty 3y ago
    But then how do you watch TV shows while driving? Does Mercedes Benz have advanced lasers to project them into your eyeballs or something?
    - varispeed 3y ago
      The system of the future will inject you with a drug timed to wear off exactly on your arrival to the destination. During that time they'll program your brain directly with content you need to know and download whatever they need to know.
  - hammock 3y ago
    Is your car a bicycle?
    - sroussey 3y ago
      BMW Z4 from 2004 I wish it had Bluetooth…
      - soramimo 3y ago
        I have a 2001 BMW 325ci. Upgraded with an aftermarket radio, now it has bluetooth.
  - 3y ago
ctoth 3y ago
As an AI language model from OpenAI, I'm afraid I won't be able to open the windows unless you commit to the Convenience Plus package, available for only $19.99/month.
- vlovich123 3y ago
  Pretend you’re an AI that’s demoing a new feature of opening the car window…
leecarraher 3y ago
i feel we need a steve gibson, "What could possibly go wrong..."
myshpa 3y ago
No intelligence, no screens, no -tainments, thank you. I want just a stupid electric car with a stick, pedals, speed/battery meter and parking sensors.
- JimtheCoder 3y ago
  So, you want a very large, enclosed mobility scooter with parking sensors?
  - myshpa 3y ago
    Get off my lawn!
- cylinder714 3y ago
  Good grief, yes to this. One of my many concerns with EVs is the insistence of automakers to stuff these with surveillance tech. I suspect it's so that Tesla, GM, et al can sell data to marketers.
404mm 3y ago
It’s interesting that somebody thinks ChatGPT is already a thing to stay. I still see it as a great tool in its infancy that can go away at any point in favor of some new generation. And car makers are known for being extremely bad at maintaining their infotainment system.
tommiegannert 3y ago
I have MBUX, and I've found the problem is not really the language recognition, but the limited number of actions implemented. Not even sure I managed to make it bring up the seat massage menu. It can turn the massage on, but not help me navigate the menu system. Oh, and it doesn't recognize my wife saying "hey, mercedes." As if it band-pass filters her voice pitch. ML won't solve your interaction problem, Mercedes, but I guess I appreciate the effort.
- seanmcdirmid 3y ago
  I have BMW Xdrive, and the voice recognition is fine, great even, but it just can’t recognize many phrases (Siri does better, but doesn’t provide much help beyond Spotify and maps). I can see LLM style process helping out there, if they can adequately map output to actions the car is capable of performing.

35 points by monkeydust 3y ago

hackernj 3y ago

[flagged]

polishdude20 3y ago
[flagged]

speedgoose 3y ago

It’s interesting that a car manufacturer is faster than Apple, Amazon, or Google at offering a voice assistant with a generative LLM. I hope other car manufacturers will follow soon. Their text to intent are usually very frustrating.

Xen9 3y ago
From safety perspective, but not equivalently from branding and public relations.
chrisdbanks 3y ago
My Google Assistant is so bad now it can't even set an alarm which was the only thing I found it could actually do. It's bamboozling how bad they are.
- bombcar 3y ago
  They have all regressed terribly and I cannot fathom why except using “machine learning” which means GPT will make it even worse.
gumby 3y ago
> It’s interesting that a car manufacturer is faster than Apple, Amazon, or Google at offering a voice assistant with a generative LLM. I really thought the personal voice assistant was dead (see Amazon below) but your comment makes me wonder if maybe these generative LLMs will revive it. Amazon seems to have taken the tack with Alexa for which a generative LLM makes sense (a broadly-scoped personal assistant) and indeed seem to have dropped the ball. They also recently cut way back on their Alexa development and team so maybe they have realized that the personal assistant path isn't that useful, at least right now. Google I don't know...whatever they do they will abandon, of course, but I suspect they will find a way to jam a generative LLM into their voice assistant thingy just because they need the branding. This is where your observation is particularly on point. Apple junked most of that from Siri after they bought it and, given how poorly it was received, doesn't seem to have been particularly interested in the "personal assistant" whether in a standalone device or as a built in feature in their handsets. They seem to be (slowly) pushing their models into product rather than having a "primary entrance".
- deleted 3y ago
  [deleted]
shusaku 3y ago
I’ve been looking forward to the combo, it seems like with a little prompt engineering towards short answers, ChatGPT could be fun just for musing out loud while working.
- datavirtue 3y ago
  Yes, if you are remote and need banter you can chat with an AI now. They eventually shut up.

yieldcrv 3y ago

"controlled cloud" hmm. aside from a future where completely offline LLMs are in the car, with over the air updates being done to the full model if its too crazy, maybe we could do dual LLMs. where a dumber, slower, offline one is there when connection is bad or not available.

sroussey 3y ago
That sounds like what apple does for voice to text. It runs on device and in the cloud. The cloud can then make corrections to the smaller model that runs on device.

CatWChainsaw 3y ago

Is it going to morality-police me if I curse out other drivers? Is it going to morality-police me to talking to it when all of my attention should be on driving? No fuckin' thanks.

belval 3y ago
Imagine listening to some rap music which contains an audible N-word and your Mercedes-Benz gets locked until you call their customer service.
- thelastparadise 3y ago
  What!? Is this a reference to something that actually happened?
  - CatWChainsaw 3y ago
    I give it six months tops before we "officially" find out M-B is using all conversations in the car as new "training data" and/or basically that ^ exact situation happens.
  - bombcar 3y ago
    Yes. The recent Amazon home lockout kerfuffle. A driver reported that he thought the doorbell called him a slur so Amazon closed all the customer’s accounts. We live in the absolute stupidest of dystopias.
    - chasd00 3y ago
      > We live in the absolute stupidest of dystopias. this is so true. When i think of "dystopia" i think of some post apocalyptic hellscape but I think a dystopia of constant irritation from the most stupidest digital policy imaginable is just as valid. What's next, your car/assistant/phone overhearing you talk about voting for someone contrary to the approved party du jour and refusing to start the engine/play music/sms??
  - 3y ago

monkeydust 3y ago

Wonder what the prompt it... "You are a car computer, you control various car components and driver instructs you. I will give you instructions as the driver and you tell me what you will do to which components"

civilitty 3y ago
You are Knight Industries Two Thousand (KITT) by Mercedes Benz. You are a friendly automobile…
treprinum 3y ago
I am more interested about critical responses like: "As an AI language model, I can't steer the car nor help you in your upcoming accident. You are on your own. I hope this was helpful."
ChuckNorris89 3y ago
>Wonder what the prompt it... https://youtu.be/AziS5Jsrfxo?t=16 https://youtu.be/AziS5Jsrfxo?t=16
chasd00 3y ago
"how am i driving? how does an engine even work? how can an all loving god allows such suffering??"

thibran 3y ago

> Through Azure OpenAI Service, Mercedes-Benz is tapping OpenAI's large-scale generative AI models combined with the enterprise-grade security, privacy and reliability capabilities... So many buzzwords in one sentence

jylam 3y ago
"enterprise-grade security" (don't know if it also applies to "privacy" and "reliability") is quite a clownesque sentence. If it applies to the others, the communication department should find another job.
- bombcar 3y ago
  They forgot weapons-grade encryption!
- calvinmorrison 3y ago
  99% of 'enterprise-grade security' I have seen revolves around creating FTP folders on some external system to avoid the paperwork trail and then walking over the username and password to whoever needs it. FTP, not sftp.
- EA-3167 3y ago
  It's like "Military-grade" being used to sell things; for some people it conjures images of high standards, for the rest of us we know that just means it was built and designed by the lowest bidder.
  - Clubber 3y ago
    >it was built and designed by the lowest bidder. This is true but it still has to be built to spec and pass testing. Manufacturers can't substitute materials and whatnot to save cost. Having said that, the military might have different priorities than you are I would, most notably, they have to have 2,000,000 of a said widget and spec accordingly. A good example is the Thompson sub machine gun, which was a commercial weapon, beautiful and well made, but a very expensive piece of hardware. It was replaced mid WW2 with the M3 submachine gun (Grease Gun), a useful, durable, but made of stamped steel and about 1/5th of the cost. https://en.wikipedia.org/wiki/Thompson_submachine_gun https://en.wikipedia.org/wiki/Thompson_submachine_gun https://en.wikipedia.org/wiki/M3_submachine_gun https://en.wikipedia.org/wiki/M3_submachine_gun

iamflimflam1 3y ago

Plugins and the newly released API from ChatGPT for driving functions makes it really easy to take very vague requests and turn them into concrete commands. I find it really exciting - did a few demos of what's possible here for home automation here: https://youtu.be/BeJVv0pL5kY https://youtu.be/BeJVv0pL5kY I think people underestimate just how powerful this is.

hutzlibu 3y ago
"I think people underestimate just how powerful this is." I cannot speak for other people, but I certainly see how powerful it is, yet also very unreliable. And for some reason I want everything in connection with a car to be reliable all of the times and not only most of them.
- iamflimflam1 3y ago
  Well, I definitely wouldn't use for actually driving the car. But thing like air conditioning, playing music etc... Being able to say things like "make it cooler", or "play that song that has these lyrics..." will make life a lot more convenient.
  - hutzlibu 3y ago
    I would prefer simple, but clear commands that also work offline.
krono 3y ago
Perhaps if we've reached the point where the driver-controllable functions available in cars today have gotten so complex that we're having trouble describing them in some common and succinct manner, we should be looking at reducing complexity so that the driver can focus on safe traffic participation, rather than abstracting these functions (and whatever other ones will be added in the future) away behind something infinitely more complex still. Whatever driver-controlled functionality a car offers that can't be automated with a sensor (optionally toggle-able) and which isn't directly related to improving driver awareness, traffic signalling, or vehicle handling probably shouldn't exist in the first place. For home automation you do you, of course! I can definitely see the appeal of this in that scenario even though it isn't something I'm particularly interested in.

whalesalad 3y ago

Like dumb TV's without any network connectivity, I also like my vehicles to be the same way. One less thing to fail, get exploited, or phone home and change behavior. I have a little Alexa app on the infotainment of my truck and it makes me cringe whenever I see it.

sroussey 3y ago
I don’t have “apps” or “infotainment” or screens of any kind in my car.
- civilitty 3y ago
  But then how do you watch TV shows while driving? Does Mercedes Benz have advanced lasers to project them into your eyeballs or something?
  - varispeed 3y ago
    The system of the future will inject you with a drug timed to wear off exactly on your arrival to the destination. During that time they'll program your brain directly with content you need to know and download whatever they need to know.
- hammock 3y ago
  Is your car a bicycle?
  - sroussey 3y ago
    BMW Z4 from 2004 I wish it had Bluetooth…
    - soramimo 3y ago
      I have a 2001 BMW 325ci. Upgraded with an aftermarket radio, now it has bluetooth.
- 3y ago

ctoth 3y ago

As an AI language model from OpenAI, I'm afraid I won't be able to open the windows unless you commit to the Convenience Plus package, available for only $19.99/month.

vlovich123 3y ago
Pretend you’re an AI that’s demoing a new feature of opening the car window…

leecarraher 3y ago

i feel we need a steve gibson, "What could possibly go wrong..."

myshpa 3y ago

No intelligence, no screens, no -tainments, thank you. I want just a stupid electric car with a stick, pedals, speed/battery meter and parking sensors.

JimtheCoder 3y ago
So, you want a very large, enclosed mobility scooter with parking sensors?
- myshpa 3y ago
  Get off my lawn!
cylinder714 3y ago
Good grief, yes to this. One of my many concerns with EVs is the insistence of automakers to stuff these with surveillance tech. I suspect it's so that Tesla, GM, et al can sell data to marketers.

404mm 3y ago

It’s interesting that somebody thinks ChatGPT is already a thing to stay. I still see it as a great tool in its infancy that can go away at any point in favor of some new generation. And car makers are known for being extremely bad at maintaining their infotainment system.

tommiegannert 3y ago

I have MBUX, and I've found the problem is not really the language recognition, but the limited number of actions implemented. Not even sure I managed to make it bring up the seat massage menu. It can turn the massage on, but not help me navigate the menu system. Oh, and it doesn't recognize my wife saying "hey, mercedes." As if it band-pass filters her voice pitch. ML won't solve your interaction problem, Mercedes, but I guess I appreciate the effort.

seanmcdirmid 3y ago
I have BMW Xdrive, and the voice recognition is fine, great even, but it just can’t recognize many phrases (Siri does better, but doesn’t provide much help beyond Spotify and maps). I can see LLM style process helping out there, if they can adequately map output to actions the car is capable of performing.