3 ms·

I think you're giving it too much credit for solving examples that are obviously wrong. Not only a non-expert but even most children know those pairings don't g

I think you're giving it too much credit for solving examples that are obviously wrong. Not only a non-expert but even most children know those pairings don't go together. Given that the discussion in this post is around torquing to the right specifications, I don't know if just fine tuning is enough. It might need more serious training on videos of assembly. Even then, can it distinguish between the right torque or not from video? I think it's interesting, but you should deliberately try to make it fail to see where the edges are. Like hold a wrench of the obviously wrong size next to a nut and ask if that will work. Force perspective so they look the same size. See if it will prevent you from mixing potentially dangerous chemical combinations. Will it warn you to wear protective eyewear when using a circular saw?

consumer451 3y ago
Oh definitely, I was giving it some softballs. Maybe I am also giving too much credit to what fine tuning can achieve. But in a manufacturing environment, if one labelled wrench sizes with different colors, then things like noticing wrench size would be easier. Also, I imagined that the camera would of known (same) geometry during training and inference in a Boeing implementation. I also wonder if multimodal LLMs are blowing my mind unnecessarily. It really feels like a huge leap to me though.
- plagiarist 3y ago
  They are a pretty huge leap! I just think for manufacturing and assembly where it is critical to get specific values right we cannot rely on neural nets in their current form. I think there are digital wrenches that will record the torque they applied, you don't need to mess around with colors. But at that point, it's sort of like, why bother teaching the AI to notice if people are doing it right, stick a camera on the wrench and QR sticker on the bolt. Make a more formal verification process that can be guaranteed to match specifications.

consumer451 3y ago

Oh definitely, I was giving it some softballs. Maybe I am also giving too much credit to what fine tuning can achieve. But in a manufacturing environment, if one labelled wrench sizes with different colors, then things like noticing wrench size would be easier. Also, I imagined that the camera would of known (same) geometry during training and inference in a Boeing implementation. I also wonder if multimodal LLMs are blowing my mind unnecessarily. It really feels like a huge leap to me though.

plagiarist 3y ago
They are a pretty huge leap! I just think for manufacturing and assembly where it is critical to get specific values right we cannot rely on neural nets in their current form. I think there are digital wrenches that will record the torque they applied, you don't need to mess around with colors. But at that point, it's sort of like, why bother teaching the AI to notice if people are doing it right, stick a camera on the wrench and QR sticker on the bolt. Make a more formal verification process that can be guaranteed to match specifications.