9 ms·

This sub and blog post are all talking about ChatGPT which is no doubt amazing and far ahead of the curve. However I would also point to Metas new vision model:

This sub and blog post are all talking about ChatGPT which is no doubt amazing and far ahead of the curve. However I would also point to Metas new vision model: SAM (Segment Anything): it is so far beyond any other vision model, I actually believe vision will be solved in a few years now. People don’t realize that there was an industry of publishing paper with incremental improvements in small datasets in CVPR that has been completely invalidated by this paper. I’ve seen engineers in Cruise segmentation team, say Metas new model seems to work better than the in house models they developed and that they should build on top of this. I’ve worked in Tesla Autopilot before and saw it hit a mannequin because we never had mannequin in our dataset before ( we might have had it in the data but it was not a part of our ontology for the network to predict). One approach to mitigate this was OpenAI’s clip that used the English Language as classification labels but Meta’s SAM is so much better where it detects objects without need to specify language. It just understands scenes and objects, at a fundamental level, it can detect anything in a picture if you prompt it right. Honestly it feels a lot like GPT1 which was also ignored by most. If you prompt it right you can get it to segment anything in an image, but prompting it right requires human input. However I can imagine the third or fourth version, with some RL sprinkled in just working zero-shot on complete pixel understanding of any image in the world. This was one of the holy grail of computer vision, that we are seeing solved right in front of our eyes

nextworddev 4y ago
Yep. We came a long way from bounding boxes, not sure if community here is ready for vision + 3d + video models to have the “chatgpt” moment this year
mitthrowaway2 4y ago
There seem to be a lot of breakthroughs being announced right now. My guess is that after ChatGPT, companies like Meta pushed their teams to get their own projects out the door which they'd already been working on for some time. SAM and Zero-1-to-3 (https://news.ycombinator.com/item?id=35242193 https://news.ycombinator.com/item?id=35242193) are incredibly impressive new projects in the image comprehension side. Then of course there's all the stuff happening with Midjourney/Stable Diffusion. And DeepMind's AlphaCode.
- sashank_1509 4y ago
  I’d say after ChatGPT’s success every other company is focusing far more on scaling up than coming up with new methods. SAM is basically the method they came up with in 2021, but scaled up to 1 billion masks, which was not something Meta might have been willing to do before ChatGPT
mach1ne 4y ago
What does it mean to solve vision though?
- kajecounterhack 4y ago
  It's 2023 and tesla still can't reliably read traffic lights. Sounds like you know vision has a long way to go but I'd expect in the future it'll be harder to make that case. Maybe that's all that's meant.
  - evdubs 4y ago
    It works for me. Do you have an article or something discussing the Tesla limitations?
    - kajecounterhack 4y ago
      https://www.ndtv.com/offbeat/watch-tesla-autopilot-feature-mistakes-moon-for-yellow-traffic-light-2495804 https://www.ndtv.com/offbeat/watch-tesla-autopilot-feature-m... This is 2021, but these systems are still not very smart today.
      - deleted 4y ago
        [deleted]
      - leoh 4y ago
        Their tech is crap. Not even L2 driving really.
        kajecounterhack 4y ago
        You should give them more credit than that, but even if you don't you'd be surprised how bad even companies like Cruise and Waymo can be at this kind of thing.
AussieWog93 4y ago
That sounds amazing. I have a robot vacuum cleaner with "AI" computer vision, and you can just tell that it doesn't see the world in the way a living being does. It'll detect and avoid shoes, cables and dog turds, but is completely blind to anything not in its specific library of objects (we have a two year old, it chokes all the time on things like ribbons and toys). Being able to recognise objects that cause it to choke, even if it doesn't what they're called in English, is souch more significant to robotics than being able to label things.
greeny7373 4y ago
Yep Sam is really really good. Already have use cases for that too!
ModernMech 4y ago
> I’ve worked in Tesla Autopilot before and saw it hit a mannequin because we never had mannequin in our dataset before ( we might have had it in the data but it was not a part of our ontology for the network to predict). Incredible. To be clear, your group and the company you work for saw fit to release your beta-quality autopilot onto public roads, as it was hitting human-looking objects in your test labs. Is that what you're admitting here? Did anyone in your group object to this? Were you personally concerned? Instead of moving fast and breaking things, what if you had not rushed your autopilot out, but waited on technology to improve to the point that your product wouldn't hit a human-looking object it had never seen before?
- sashank_1509 4y ago
  There’s too much insanity over building new tech these days, and I find a lot of hate directed on Autopilot, to stem from hate directed on Elon. Let’s go over the basics for Autopilot: 1. It costs thousands of dollars a year + 15k one time and it’s very easy to get banned from autopilot for life. They have a 3 strike rule + 1000 miles driven on your Tesla with a good safety before you’re allowed to access autopilot 2. I’ve seen 2 types of customers use autopilot. One is rich dudes, who buy it just to have all bells and whistles. They use it less than once a month and honestly it’s a waste of money for them. The other are passionate early adopters, they regularly make YouTube videos, constantly stress test our tech and are huge contributors of our tech itself. I’d say 10-20% of our users are the latter. The kind of group who don’t use autopilot? The regular old Joe, who perhaps would like Autopilot for some practical use case. It’s too expensive, has a lot of restrictions (like you have to grab the steering once every few minutes or autopilot disengages and then you get banned out of it even if you pay) that it doesn’t make sense for him to buy this tech anyway. In essence, this is not tech that is being used by regular people who have a chance of misusing it. Ever since the Uber self driving crash, heads roll if a self driving car crashes and as an engineer I don’t get any access to Tesla legal but it’s my understanding that in none of the cases filed against Tesla, did they prove Autopilot was active (forget Autopilot being the cause). So yes, we’re not building new tech that’s killing hundreds, no one has any ethical dilemmas here. We’re building tech that a passionate group of users really want to see succeed and help us do that, and rest of the users just give us money for some reason even though they don’t really use it or trust it. I would frankly be more torn about working in a place like Waymo where the user has almost no control over the car (they don’t even sit at the steering wheel), and they have to solve the problem one shot before releasing it to the public while Tesla can keep iterating step by step (with its passionate user base supporting us and showing love all the way) Edit: Changed this comment to make it smaller
ragazzina 4y ago
I wonder if this means no more stupid select-all-squares-with captchas.
- phito 4y ago
  Captcha will soon be useless

nextworddev 4y ago

Yep. We came a long way from bounding boxes, not sure if community here is ready for vision + 3d + video models to have the “chatgpt” moment this year

mitthrowaway2 4y ago

There seem to be a lot of breakthroughs being announced right now. My guess is that after ChatGPT, companies like Meta pushed their teams to get their own projects out the door which they'd already been working on for some time. SAM and Zero-1-to-3 (https://news.ycombinator.com/item?id=35242193 https://news.ycombinator.com/item?id=35242193) are incredibly impressive new projects in the image comprehension side. Then of course there's all the stuff happening with Midjourney/Stable Diffusion. And DeepMind's AlphaCode.

sashank_1509 4y ago
I’d say after ChatGPT’s success every other company is focusing far more on scaling up than coming up with new methods. SAM is basically the method they came up with in 2021, but scaled up to 1 billion masks, which was not something Meta might have been willing to do before ChatGPT

mach1ne 4y ago

What does it mean to solve vision though?

kajecounterhack 4y ago
It's 2023 and tesla still can't reliably read traffic lights. Sounds like you know vision has a long way to go but I'd expect in the future it'll be harder to make that case. Maybe that's all that's meant.
- evdubs 4y ago
  It works for me. Do you have an article or something discussing the Tesla limitations?
  - kajecounterhack 4y ago
    https://www.ndtv.com/offbeat/watch-tesla-autopilot-feature-mistakes-moon-for-yellow-traffic-light-2495804 https://www.ndtv.com/offbeat/watch-tesla-autopilot-feature-m... This is 2021, but these systems are still not very smart today.
    - deleted 4y ago
      [deleted]
    - leoh 4y ago
      Their tech is crap. Not even L2 driving really.
      - kajecounterhack 4y ago
        You should give them more credit than that, but even if you don't you'd be surprised how bad even companies like Cruise and Waymo can be at this kind of thing.

AussieWog93 4y ago

That sounds amazing. I have a robot vacuum cleaner with "AI" computer vision, and you can just tell that it doesn't see the world in the way a living being does. It'll detect and avoid shoes, cables and dog turds, but is completely blind to anything not in its specific library of objects (we have a two year old, it chokes all the time on things like ribbons and toys). Being able to recognise objects that cause it to choke, even if it doesn't what they're called in English, is souch more significant to robotics than being able to label things.

greeny7373 4y ago

Yep Sam is really really good. Already have use cases for that too!

ModernMech 4y ago

> I’ve worked in Tesla Autopilot before and saw it hit a mannequin because we never had mannequin in our dataset before ( we might have had it in the data but it was not a part of our ontology for the network to predict). Incredible. To be clear, your group and the company you work for saw fit to release your beta-quality autopilot onto public roads, as it was hitting human-looking objects in your test labs. Is that what you're admitting here? Did anyone in your group object to this? Were you personally concerned? Instead of moving fast and breaking things, what if you had not rushed your autopilot out, but waited on technology to improve to the point that your product wouldn't hit a human-looking object it had never seen before?

sashank_1509 4y ago
There’s too much insanity over building new tech these days, and I find a lot of hate directed on Autopilot, to stem from hate directed on Elon. Let’s go over the basics for Autopilot: 1. It costs thousands of dollars a year + 15k one time and it’s very easy to get banned from autopilot for life. They have a 3 strike rule + 1000 miles driven on your Tesla with a good safety before you’re allowed to access autopilot 2. I’ve seen 2 types of customers use autopilot. One is rich dudes, who buy it just to have all bells and whistles. They use it less than once a month and honestly it’s a waste of money for them. The other are passionate early adopters, they regularly make YouTube videos, constantly stress test our tech and are huge contributors of our tech itself. I’d say 10-20% of our users are the latter. The kind of group who don’t use autopilot? The regular old Joe, who perhaps would like Autopilot for some practical use case. It’s too expensive, has a lot of restrictions (like you have to grab the steering once every few minutes or autopilot disengages and then you get banned out of it even if you pay) that it doesn’t make sense for him to buy this tech anyway. In essence, this is not tech that is being used by regular people who have a chance of misusing it. Ever since the Uber self driving crash, heads roll if a self driving car crashes and as an engineer I don’t get any access to Tesla legal but it’s my understanding that in none of the cases filed against Tesla, did they prove Autopilot was active (forget Autopilot being the cause). So yes, we’re not building new tech that’s killing hundreds, no one has any ethical dilemmas here. We’re building tech that a passionate group of users really want to see succeed and help us do that, and rest of the users just give us money for some reason even though they don’t really use it or trust it. I would frankly be more torn about working in a place like Waymo where the user has almost no control over the car (they don’t even sit at the steering wheel), and they have to solve the problem one shot before releasing it to the public while Tesla can keep iterating step by step (with its passionate user base supporting us and showing love all the way) Edit: Changed this comment to make it smaller

ragazzina 4y ago

I wonder if this means no more stupid select-all-squares-with captchas.

phito 4y ago
Captcha will soon be useless