15 ms·

Okay the bike example is cute and impressive, but the human interaction seems to be obfuscating the potentially bigger application. With a few tweaks this is a

by TOMDM 3y ago

Okay the bike example is cute and impressive, but the human interaction seems to be obfuscating the potentially bigger application. With a few tweaks this is a general purpose solver for robotics planning. There are still a few hard problems between this and a working solution, but it is one of hard problems solved. Will we be seeing general purpose robots performing simple labor powered by chatgpt within the next half decade?

amelius 3y ago
> With a few tweaks this is a general purpose solver for robotics planning. Yeah, but with an enormous ecological footprint. Also, not suitable for small lightweight robots like drones.
- TOMDM 3y ago
  Even on something the size of a car chatgpt won't be running locally, the car and drone are equally capable of hitting openai's API in a well connected environment. What needs to happen with the response is a different matter though.
- dist-epoch 3y ago
  What's the ecological footprint of a human doing the same job? Especially when you factor in 18+ years of preparing.
  - bamboozled 3y ago
    Humans don't spend 18+ years preparing how to lower a seat post or drive a truck or even do pretty much most jobs. No one is solely training for 18 years to do anything. Most of those 18 years are having a fucking great time (being young is freakin awesome) and living a great life is never a waste or a negative ecological footprint. Society artificially slows education down so it takes 18 years to finish school because parents need to be off at work, so 18 years of baby sitting is preferred. By 18, kids are at the age where they will no longer be told what to do so it's off to the next waste of time, college, then 30 years of staring at a blinking box...or whatever. When I was 12, I decided I wanted to drive a car, I'd never driven a car in my life, but I took my parents car and drove it around wherever I liked with absolutely no issue or prior instruction. I did this for years. The youth are very capable, we just don't want them to be too capable...
psbp 3y ago
Google demoed this a few months ago https://www.deepmind.com/blog/rt-2-new-model-translates-vision-and-language-into-action?utm_source=keywordblog&utm_medium=referral&utm_campaign=rt2 https://www.deepmind.com/blog/rt-2-new-model-translates-visi...
- m3kw9 3y ago
  They are really good at keeping demos as demos
  - michelb 3y ago
    It’s just a hiring article.
    - rmbyrro 3y ago
      Hiring to produce more demos, to hire more to produce even more demos...
      - TeMPOraL 3y ago
        Yes. As long as the hirees do some actual work in between producing demos, this even makes sense as a hiring approach.
      - michelb 3y ago
        As long as they choose not to work elsewhere, it seems effective for Google.
  - Workaccount2 3y ago
    The implementation that manifests itself as an extremely creepy, downright concerning level of dubious moral transgressions isn't nearly as publicly glamorous as their tech demos.
  - dragonwriter 3y ago
    I don’t know, a lot of Google demos and papers introduce techniques that are productized fairly soon, just usually not by Google.
usaar333 3y ago
That bike example seemed a mix of underwhelming (for being the demo video) and even confusing. 1. It's not smart enough to recognize from the initial image this is a bolt style seat lock (which a human can). 2. The manual is not shown to the viewer, so I can't infer how the model knows this is a 4mm bolt (or if it is just guessing given that's the most likely one). 3. I don't understand how it can know the toolbox is using metric allen wrenches. Additionally is this just the same vision model that exists in bing chat?
- gisely 3y ago
  Yep. This example basically convinced me that they were unable to figure out anything actually useful to do with the model's new capabilities. Which makes me wonder how capable the new model in fact is.
  - usaar333 3y ago
    Yah, pretty sure it is the same feature that's been in Bing Chat for 2 months now. Which feels really like there's only one pass of feature extraction from the image, preventing any detailed analysis beyond a course "what do you see". (Follow-up questions of things it likely didn't parse are highly hallucinated). This is why they can't extract the seat post information directly from the bike when the user asks. There's no "going back and looking at the image". Edit: nope, it's a better image analyzer than Bing
    - famouswaffles 3y ago
      >Yah, pretty sure it is the same feature that's been in Bing Chat for 2 months now. It's not. Feel free to try these queries: https://twitter.com/ComicSociety/status/1698694653845848544?t=m9W8JdSxSi9o1uJL_zx24g&s=19 https://twitter.com/ComicSociety/status/1698694653845848544?... (comic book page in particular, from a be my eyes user) Or these https://imgur.com/a/iOYTmt0 https://imgur.com/a/iOYTmt0 (graph analysis in particular, last example) and see Bing fail them.
- biot 3y ago
  The bike shown in the first image is Specialized Sirrus X. You can make out from the image of the manual that it says "spacer/axle/bolt specifications". Searching for this yields the following Specialized bike manual which is similar: https://www.manualslib.com/manual/1974494/Specialized-Epic-Evo.html?page=9#manual https://www.manualslib.com/manual/1974494/Specialized-Epic-E... -- there are some notable differences, but the Specialized Sirrus X manuals that are online aren't in the same style. The prior page (8) shows "SEAT COLLAR 4mm HEX" and, based on looking up seat collar in an image search, the part in question matches. In terms of the toolbox, note that it only identified the location of the Allen wrench set. The advice was just "Within that set, find the 4 mm Allen (Hex) key". Had they replied with "I don't see any sizes in mm", the conversation could've continued with "Your Allen keys might be using SAE sizing. A compatible size will be 5/32, do you see that in your set?"
RivieraKid 3y ago
This is what I'm most excited about. There's been a minor breakthrough recently: https://pressroom.toyota.com/toyota-research-institute-unveils-breakthrough-in-teaching-robots-new-behaviors/ https://pressroom.toyota.com/toyota-research-institute-unvei...
famouswaffles 3y ago
There are already a few research demos. For driving - https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...
hereonout2 3y ago
I feel they could have used a more convincing example to be honest. Yeah it's cool it recognises so much but how useful is the demo in reality? You have someone with a tool box and a manual (seriously who has a manual for their bike), asking the most basic question on how to lower a seatpost. My 5 year old kid knows how to do that. Surely there's a better way to demonstrate the ground breaking impacts of ai on humanity than this. I dunno, something like how do I tie my shoelace.

by TOMDM 3y ago

amelius 3y ago

> With a few tweaks this is a general purpose solver for robotics planning. Yeah, but with an enormous ecological footprint. Also, not suitable for small lightweight robots like drones.

TOMDM 3y ago
Even on something the size of a car chatgpt won't be running locally, the car and drone are equally capable of hitting openai's API in a well connected environment. What needs to happen with the response is a different matter though.
dist-epoch 3y ago
What's the ecological footprint of a human doing the same job? Especially when you factor in 18+ years of preparing.
- bamboozled 3y ago
  Humans don't spend 18+ years preparing how to lower a seat post or drive a truck or even do pretty much most jobs. No one is solely training for 18 years to do anything. Most of those 18 years are having a fucking great time (being young is freakin awesome) and living a great life is never a waste or a negative ecological footprint. Society artificially slows education down so it takes 18 years to finish school because parents need to be off at work, so 18 years of baby sitting is preferred. By 18, kids are at the age where they will no longer be told what to do so it's off to the next waste of time, college, then 30 years of staring at a blinking box...or whatever. When I was 12, I decided I wanted to drive a car, I'd never driven a car in my life, but I took my parents car and drove it around wherever I liked with absolutely no issue or prior instruction. I did this for years. The youth are very capable, we just don't want them to be too capable...

psbp 3y ago

Google demoed this a few months ago https://www.deepmind.com/blog/rt-2-new-model-translates-vision-and-language-into-action?utm_source=keywordblog&utm_medium=referral&utm_campaign=rt2 https://www.deepmind.com/blog/rt-2-new-model-translates-visi...

m3kw9 3y ago
They are really good at keeping demos as demos
- michelb 3y ago
  It’s just a hiring article.
  - rmbyrro 3y ago
    Hiring to produce more demos, to hire more to produce even more demos...
    - TeMPOraL 3y ago
      Yes. As long as the hirees do some actual work in between producing demos, this even makes sense as a hiring approach.
    - michelb 3y ago
      As long as they choose not to work elsewhere, it seems effective for Google.
- Workaccount2 3y ago
  The implementation that manifests itself as an extremely creepy, downright concerning level of dubious moral transgressions isn't nearly as publicly glamorous as their tech demos.
- dragonwriter 3y ago
  I don’t know, a lot of Google demos and papers introduce techniques that are productized fairly soon, just usually not by Google.

usaar333 3y ago

That bike example seemed a mix of underwhelming (for being the demo video) and even confusing. 1. It's not smart enough to recognize from the initial image this is a bolt style seat lock (which a human can). 2. The manual is not shown to the viewer, so I can't infer how the model knows this is a 4mm bolt (or if it is just guessing given that's the most likely one). 3. I don't understand how it can know the toolbox is using metric allen wrenches. Additionally is this just the same vision model that exists in bing chat?

gisely 3y ago
Yep. This example basically convinced me that they were unable to figure out anything actually useful to do with the model's new capabilities. Which makes me wonder how capable the new model in fact is.
- usaar333 3y ago
  Yah, pretty sure it is the same feature that's been in Bing Chat for 2 months now. Which feels really like there's only one pass of feature extraction from the image, preventing any detailed analysis beyond a course "what do you see". (Follow-up questions of things it likely didn't parse are highly hallucinated). This is why they can't extract the seat post information directly from the bike when the user asks. There's no "going back and looking at the image". Edit: nope, it's a better image analyzer than Bing
  - famouswaffles 3y ago
    >Yah, pretty sure it is the same feature that's been in Bing Chat for 2 months now. It's not. Feel free to try these queries: https://twitter.com/ComicSociety/status/1698694653845848544?t=m9W8JdSxSi9o1uJL_zx24g&s=19 https://twitter.com/ComicSociety/status/1698694653845848544?... (comic book page in particular, from a be my eyes user) Or these https://imgur.com/a/iOYTmt0 https://imgur.com/a/iOYTmt0 (graph analysis in particular, last example) and see Bing fail them.
biot 3y ago
The bike shown in the first image is Specialized Sirrus X. You can make out from the image of the manual that it says "spacer/axle/bolt specifications". Searching for this yields the following Specialized bike manual which is similar: https://www.manualslib.com/manual/1974494/Specialized-Epic-Evo.html?page=9#manual https://www.manualslib.com/manual/1974494/Specialized-Epic-E... -- there are some notable differences, but the Specialized Sirrus X manuals that are online aren't in the same style. The prior page (8) shows "SEAT COLLAR 4mm HEX" and, based on looking up seat collar in an image search, the part in question matches. In terms of the toolbox, note that it only identified the location of the Allen wrench set. The advice was just "Within that set, find the 4 mm Allen (Hex) key". Had they replied with "I don't see any sizes in mm", the conversation could've continued with "Your Allen keys might be using SAE sizing. A compatible size will be 5/32, do you see that in your set?"

RivieraKid 3y ago

This is what I'm most excited about. There's been a minor breakthrough recently: https://pressroom.toyota.com/toyota-research-institute-unveils-breakthrough-in-teaching-robots-new-behaviors/ https://pressroom.toyota.com/toyota-research-institute-unvei...

famouswaffles 3y ago

There are already a few research demos. For driving - https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...

hereonout2 3y ago

I feel they could have used a more convincing example to be honest. Yeah it's cool it recognises so much but how useful is the demo in reality? You have someone with a tool box and a manual (seriously who has a manual for their bike), asking the most basic question on how to lower a seatpost. My 5 year old kid knows how to do that. Surely there's a better way to demonstrate the ground breaking impacts of ai on humanity than this. I dunno, something like how do I tie my shoelace.