3 ms·
it's just that tokens are things like "tele" "vis" "ion" etc., represented by some numbers/vectors but instead of that, we can have those vectors/numbers repre
by xucian 2y ago
it's just that tokens are things like "tele" "vis" "ion" etc., represented by some numbers/vectors
but instead of that, we can have those vectors/numbers represent the concepts themselves. for example, the concept of "television" would be represented by a single vector, and guess what, you then also get translation capabilities by design, since 99% of the concepts are language-agnostic; and you (probably) also get smaller models, since they don't need to be "remember" all the translations for every word. imagine the same parts of the NN are working regardless of whether you talk to it in English or Japanese (after some interface LLM converts them to this "Meaning Space" I'm talking about, which isn't a trivial task, but it's a translation task more than a comprehension task). how cool is that?
I know LLMs are deceiving because they know a king is to a queen what a man is to a woman, but, put it another way, they're just "saying what we want to hear", they're not "explaining us the concept". I am really happy with where we've got, we're already extracting enormous value from them. I'm just excited about this completely new approach in arriving at the same destination, which, for me, it seems to consume less resources while coming closer to what we define as 'understanding'
we need it for web4 (brainnet) anyway