6 ms·

VideoPrism: A foundational visual encoder for video understanding

106 points by kmdupree 2y ago

adzm 2y ago
YouTube is going to be a gold mine for training data.
- hrdwdmrbl 2y ago
  is a gold mine* present tense :)
  - mtillman 2y ago
    Sold a company in 2014 that categorized 65M videos a month using a number of techniques such as object detection (soccer ball), scene classification (soccer pitch), and face recognition (Ronaldo). Used YouTube as a training ground. Marketed as ad words for video. Way ahead of its time. Did $60M in revenue the year after we sold it. Face. Palm.
    - bingleboy 2y ago
      I built something that used surveillance cams in warehouses in 2018 and almost got 25m for a subset of what our company did. I took those same ideas to some long-term connects and we turned it into something worth much more that to us. Be glad about where you are knowing that you won't have to do as much as others to pay down your sins in this industry.
- hwbunny 2y ago
  And since everyone turns their own profession to pennies by uploading the know how of their craft, it will properly wreck entire industries once the AI+Robotics marriage takes off.
  - devwastaken 2y ago
    Video describing practices does not translate to AI performing those practices. Creating a community of knowledge progresses that knowledge. Sitting on it only benefits few and removes the market. Without people sharing their work on YouTube their own market would be far less.
  - Richyrichy3234 2y ago
    And? This is only a problem in a dystopian shit future were we are too stupid to get around this. Work doesn't make people free. and yes i do absolutly believe that the pressure will be good for us as a society as we will have to rethink.
    - hwbunny 2y ago
      Just watch. People without purpose are dead.
      - Richyrichy3234 2y ago
        People are forced to make work their purpose because we still need to do it as our primary activity. Plenty of people spend their time different. Plenty of people will learn how to do something else if they actually don't have to work.
transformi 2y ago
[flagged]
- deleted 2y ago
  [deleted]
vessenes 2y ago
So frustrating. "We hope VideoPrism paves the way for future breakthroughs at the intersection of AI and video analysis, helping to realize the potential of ViFMs across domains such as scientific discovery, education, and healthcare." ... for those of us internal to Google Research, and possibly large corporate clients when we choose. Custom internal dataset, no model weights, no API access, just "FYI, this works pretty well when you have our data." I do appreciate publishing of successful architectures, genuinely, and VP looks like it has some nice ideas, and it is very useful to know that training such a thing is probably not a dead end, so for all that, thanks. At the end of the day, with large competitors committed to open models/weights, I think momentum is on the side of needing to push out open weights at least, but my guess is GOOG will be the last to transition to this of the big tech cos. I could understand it as a business decision if they were quickly opening this stuff up on their cloud, and competing that way, but they still seem to be in this world where the DeepMind crew continues to push out novel and interesting research projects, and the cloud crew struggles to deliver a competitive LLM at the same time. I wonder where they'll be in a year. I'd like to hope up to competitive standards so that we've got an additional player, but progress seems slow to me right now.
- deleted 2y ago
  [deleted]
- bingbingbing777 2y ago
  Why are you so frustrated that a company decided to not completely release something they have built internally?
  - GaggiX 2y ago
    Because it's presented as research, but the results are not reproducible without at least information about the dataset.
    - nolok 2y ago
      Well Google did that for years and where did that get them? To companies using their research to build something but not share to the same extent their research so they could keep a leg up. So yeah, they learned their lesson. You want to complain, complain to those who did not play the game fair and square.
deleted 2y ago
[deleted]
allpaca 2y ago
[dead]

106 points by kmdupree 2y ago

adzm 2y ago

YouTube is going to be a gold mine for training data.

hrdwdmrbl 2y ago
is a gold mine* present tense :)
- mtillman 2y ago
  Sold a company in 2014 that categorized 65M videos a month using a number of techniques such as object detection (soccer ball), scene classification (soccer pitch), and face recognition (Ronaldo). Used YouTube as a training ground. Marketed as ad words for video. Way ahead of its time. Did $60M in revenue the year after we sold it. Face. Palm.
  - bingleboy 2y ago
    I built something that used surveillance cams in warehouses in 2018 and almost got 25m for a subset of what our company did. I took those same ideas to some long-term connects and we turned it into something worth much more that to us. Be glad about where you are knowing that you won't have to do as much as others to pay down your sins in this industry.
hwbunny 2y ago
And since everyone turns their own profession to pennies by uploading the know how of their craft, it will properly wreck entire industries once the AI+Robotics marriage takes off.
- devwastaken 2y ago
  Video describing practices does not translate to AI performing those practices. Creating a community of knowledge progresses that knowledge. Sitting on it only benefits few and removes the market. Without people sharing their work on YouTube their own market would be far less.
- Richyrichy3234 2y ago
  And? This is only a problem in a dystopian shit future were we are too stupid to get around this. Work doesn't make people free. and yes i do absolutly believe that the pressure will be good for us as a society as we will have to rethink.
  - hwbunny 2y ago
    Just watch. People without purpose are dead.
    - Richyrichy3234 2y ago
      People are forced to make work their purpose because we still need to do it as our primary activity. Plenty of people spend their time different. Plenty of people will learn how to do something else if they actually don't have to work.

transformi 2y ago

[flagged]

deleted 2y ago
[deleted]

vessenes 2y ago

So frustrating. "We hope VideoPrism paves the way for future breakthroughs at the intersection of AI and video analysis, helping to realize the potential of ViFMs across domains such as scientific discovery, education, and healthcare." ... for those of us internal to Google Research, and possibly large corporate clients when we choose. Custom internal dataset, no model weights, no API access, just "FYI, this works pretty well when you have our data." I do appreciate publishing of successful architectures, genuinely, and VP looks like it has some nice ideas, and it is very useful to know that training such a thing is probably not a dead end, so for all that, thanks. At the end of the day, with large competitors committed to open models/weights, I think momentum is on the side of needing to push out open weights at least, but my guess is GOOG will be the last to transition to this of the big tech cos. I could understand it as a business decision if they were quickly opening this stuff up on their cloud, and competing that way, but they still seem to be in this world where the DeepMind crew continues to push out novel and interesting research projects, and the cloud crew struggles to deliver a competitive LLM at the same time. I wonder where they'll be in a year. I'd like to hope up to competitive standards so that we've got an additional player, but progress seems slow to me right now.

deleted 2y ago
[deleted]
bingbingbing777 2y ago
Why are you so frustrated that a company decided to not completely release something they have built internally?
- GaggiX 2y ago
  Because it's presented as research, but the results are not reproducible without at least information about the dataset.
  - nolok 2y ago
    Well Google did that for years and where did that get them? To companies using their research to build something but not share to the same extent their research so they could keep a leg up. So yeah, they learned their lesson. You want to complain, complain to those who did not play the game fair and square.

deleted 2y ago

[deleted]

allpaca 2y ago

[dead]