3 ms·
I just found out it gets worse: turns out GPT-4 isn't processing images so much as arrays of pixels! And worse: turns out GPT-4 isn't processing pixels so much
by riwsky 3y ago
I just found out it gets worse: turns out GPT-4 isn't processing images so much as arrays of pixels!
And worse: turns out GPT-4 isn't processing pixels so much as integers representing in a position in some color space like RGB!
And worse! turns out GPT-4 isn't processing integers so much as series of ones and zeroes!
Now that this is public knowledge, I'm willing to bet this was the ugly "less than candid" truth that the board sacked Sam Altman over.
- zarzavat 3y agoAs they say, quantity has a quality all of its own. If the framerate of a video is so slow as to be a slideshow, then it’s arguably not video anymore. Video has a connotation of appearing temporally continuous to the naked eye.
- omneity 3y agoThere is a significant difference. Video has a temporal component (frames tend to be correlated with previous ones), and vision LLMs do not have some sort of hidden states to keep track of that temporal component. Using captions to bridge this only works to a certain extent (you're giving text descriptions of what happened in the past, not what had _actually_ happened).