4 ms·
intelligence != memory
by codr7 1y ago
intelligence != memory
- esafak 1y agoML is not memorization. Besides, how much memory do you think this model has?
- codr7 1y agoI know, it's worse.
- TeMPOraL 1y agoIt's understanding.
- rhdjsjebshjffn 1y agoSure, if you still think the word has meaning.
- TeMPOraL 1y agoYes, I do. Any way you slice this term, it looks close to what ML models are learning through training. I'd go as far as saying LLMs are meaning made incarnate - that huge tensor of floats represents a stupidly high-dimensional latent space, which encodes semantic similarity of every token, and combinations of tokens (up to a limit). That's as close as reifying the meaning of "meaning" itself as we ever come. (It's funny that we got there through brute force instead of developing philosophy, and it's also nice that we get a computational artifact out of it that we can poke and study, instead of incomprehensible and mostly bogus theories.)
- rhdjsjebshjffn 1y ago[flagged]
- croes 1y agoLLMs neither understand nor reason, that has been shown multiple times.
- croes 1y agoSeems like some don’t like that LLMs aren’t really intelligent. https://neurosciencenews.com/llm-ai-logic-27987/ https://neurosciencenews.com/llm-ai-logic-27987/
- otabdeveloper4 1y agoWe're at the peak of the hype cycle right now. Ask these questions again in two years when the next winter happens.
- dinfinity 1y agoThe study tested transformers, not LLMs. They trained models on only task specific data, not on a general dataset and certainly not on the enormous datasets frontier models are trained on. "Our training sets consist of 2.9M sequences (120M tokens) for shortest paths; 31M sequences (1.7B tokens) for noisy shortest paths; and 91M sequences (4.7B tokens) for random walks. We train two types of transformers [38] from scratch using next-token prediction for each dataset: an 89.3M parameter model consisting of 12 layers, 768 hidden dimensions, and 12 heads; and a 1.5B parameter model consisting of 48 layers, 1600 hidden dimensions, and 25 heads."
- KoolKat23 1y ago
- rhdjsjebshjffn 1y agoML is a kind of memorization, though.
- onlyrealcuzzo 1y agoAnything can be a kind of something since that's subjective...
- croes 1y agoBut it’s more kind of memorization than understanding and reasoning
- rhdjsjebshjffn 1y agoYea, anything can be a kind of something else. :clown: Bruh. Do you need to be paid to interact in good faith or were you raised to be social