2 ms·
This is important, even though the Twitter part is useless. Here's the paper, "Revisiting Feature Prediction for Learning Visual Representations from Video"[1]
by Animats 3y ago
This is important, even though the Twitter part is useless. Here's the paper, "Revisiting Feature Prediction for Learning Visual Representations from Video"[1]
A big problem with machine learning so far has been the lack of some underlying, more abstract model of the subject matter. I don't understand much of this, but apparently something called "representation", a purely mathematical concept, is able to help with this.[2] This turns some kinds of abstract problems into linear algebra problems, which means matrix operations, the things GPUs do so well, help.
Code is available, from links in [2].
Would someone comment on this, please. Is this the beginning of a huge breakthrough? One where, instead of working in text token space or image pixel space, it's possible to work in an automatically generated more abstract space?
[1] https://scontent-sjc3-1.xx.fbcdn.net/v/t39.2365-6/427986745_768441298640104_1604906292521363076_n.pdf?_nc_cat=103&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=Lpq5IeF5ftUAX9eazCc&_nc_ht=scontent-sjc3-1.xx&oh=00_AfA9FKLM1Sfp86PgoyfCHg3UmLOH8stjie1QLOICKMqUkA&oe=65D69EB1 https://scontent-sjc3-1.xx.fbcdn.net/v/t39.2365-6/427986745_...
[2] https://en.wikipedia.org/wiki/Representation_theory https://en.wikipedia.org/wiki/Representation_theory
- darthoctopus 3y agoThe "representation" in "representation theory" that the wikipedia article refers to is a term of art, referring to a class of one-to-one maps between groups (sets endowed with a binary operation with specific properties) and square matrices, in such a way that matrix multiplication preserves the properties of this binary operation between the original group elements being mapped to matrices. There is no mention of this sort of algebraic operation whatsoever in the paper you have linked, which, as far as I can tell, uses the word "representation" only in a nontechnical descriptive sense. I believe you might be misunderstanding the nature of this breakthrough.
- esafak 3y agohttps://en.wikipedia.org/wiki/Feature_learning https://en.wikipedia.org/wiki/Feature_learning