4 ms·
V-JEPA: Video Joint Embedding Predictive Architecture (V-JEPA) Model
- jimmySixDOF 3y agoThis would have been bigger news except for Gemini 1.5, Sora, and the Magic investment all happening at the same time. Gemini can do needle in a haystack reliably in the 3hrs of video they tested against.
- cma 3y agoMagic investment?
- stormfather 3y agoMagic.dev got ~100MM investment because the ex CEO of github was super impressed by their progress towards super long context, code optimized LLMs to replace developers...
- sebastiennight 3y agoWhat is the "needle in a haystack" test you're referring to?
- jimmySixDOF 3y agoIt started off as an improvised test for longer context windows I think Anthropic/Claud did some early on but Greg Kamradt seems to have got it established as a benchmark [1]. Basically they hide a phrase (in this Gemini case it was text flashed on the screen) at various points in a document and then compare performance across LLMs by retrieval accuracy as the size of the document increases. [1] https://x.com/GregKamradt/status/1758149501188210932 https://x.com/GregKamradt/status/1758149501188210932
- mptest 3y agoTo expand on this, it was so perfect at the haystack benchmark that they invented a new one, basically a hundred needle in the haystack benchmark. they had to invent new benchmarks for the model because it blew all existing ones out of the water. [0] [0]https://youtu.be/Cs6pe8o7XY8 https://youtu.be/Cs6pe8o7XY8
- bitshiftfaced 3y agoLook at how Alpha Go started with human data, and then they found a way to train it without that. I've been wondering if it might be possible to do a similar thing with LLMs by grounding them on real world video by having them predict what happens in the video. I suppose you'd still need some minimal language ability to bootstrap it from, but imagine it learning the laws of physics and mathematics from the ground up.