3 ms·

the next step would be to produce descriptions of each scene in the video and search on that. this is already possible for photos/images, so getting this for v

by em-bee 3y ago

the next step would be to produce descriptions of each scene in the video and search on that. this is already possible for photos/images, so getting this for video is just a matter of time, but i don't know how far along the technology is at this point and how much resources it will take.

viddi 3y ago
There is a service that produces descriptions of each scene: https://www.videogist.co/ https://www.videogist.co/ But I don't know about searchability.
yoeven 3y ago
You read my mind! Beyond indexing the audio of the video, taking snapshots of the video content and indexing that would be pretty interesting as well. I've seen what GPT-4 vision can do for images, I'll be looking for an open source alternative that I can build around
- em-bee 3y ago
  for image analysis there are these two options: https://www.adept.ai/blog/fuyu-8b https://www.adept.ai/blog/fuyu-8b https://github.com/THUDM/CogVLM https://github.com/THUDM/CogVLM
  - yoeven 3y ago
    Oh nice!! Thanks, this helps :)