3 ms·
I need you guys help. Is there some theorem stating something like random few-hot vectors can always be combined linearly to match any signal with a low p-valu
by Ey7NFZ3P0nzAe 2y ago
I need you guys help.
Is there some theorem stating something like random few-hot vectors can always be combined linearly to match any signal with a low p-value?
I thought I encountered it sometimes in my experiments and that this might be happening in this llm x neuroscience trend of matching llm internals to brain signals.
- Ey7NFZ3P0nzAe 2y agoReply to myself to add more context as I'm at work: what I mean is that I suspect that the average "shape" of vectors in the llm is dependent on the depth inside the llm. In effect: vectors initially are gaussian-random but as the model gets trained they end up closer and closer to few-shot vectors, that are in effect easier to correlate to any signal including brain signals. In effect: any kind of llm activation could be correlated to brain signals even though it's just a sophisticated p mapping and would not correspond to anything useful. Iirc the team at meta ai of jean remi king shown that even randomly trained LLMs could be fitted.