4 ms·
Imagine a model smarter than most humans that fits on your phone. edit: I seem to be the only one excited by the possibilities of such small yet powerful model
by esafak 1y ago
Imagine a model smarter than most humans that fits on your phone.
edit: I seem to be the only one excited by the possibilities of such small yet powerful models. This is an iPhone moment: a computer that fits in your pocket, except this time it's smart.
- codr7 1y agointelligence != memory
- esafak 1y agoML is not memorization. Besides, how much memory do you think this model has?
- codr7 1y agoI know, it's worse.
- TeMPOraL 1y agoIt's understanding.
- rhdjsjebshjffn 1y agoSure, if you still think the word has meaning.
- TeMPOraL 1y agoYes, I do. Any way you slice this term, it looks close to what ML models are learning through training. I'd go as far as saying LLMs are meaning made incarnate - that huge tensor of floats represents a stupidly high-dimensional latent space, which encodes semantic similarity of every token, and combinations of tokens (up to a limit). That's as close as reifying the meaning of "meaning" itself as we ever come. (It's funny that we got there through brute force instead of developing philosophy, and it's also nice that we get a computational artifact out of it that we can poke and study, instead of incomprehensible and mostly bogus theories.)
- rhdjsjebshjffn 1y ago[flagged]
- croes 1y agoLLMs neither understand nor reason, that has been shown multiple times.
- croes 1y agoSeems like some don’t like that LLMs aren’t really intelligent. https://neurosciencenews.com/llm-ai-logic-27987/ https://neurosciencenews.com/llm-ai-logic-27987/
- otabdeveloper4 1y agoWe're at the peak of the hype cycle right now. Ask these questions again in two years when the next winter happens.
- dinfinity 1y agoThe study tested transformers, not LLMs. They trained models on only task specific data, not on a general dataset and certainly not on the enormous datasets frontier models are trained on. "Our training sets consist of 2.9M sequences (120M tokens) for shortest paths; 31M sequences (1.7B tokens) for noisy shortest paths; and 91M sequences (4.7B tokens) for random walks. We train two types of transformers [38] from scratch using next-token prediction for each dataset: an 89.3M parameter model consisting of 12 layers, 768 hidden dimensions, and 12 heads; and a 1.5B parameter model consisting of 48 layers, 1600 hidden dimensions, and 25 heads."
- KoolKat23 1y ago
- rhdjsjebshjffn 1y agoML is a kind of memorization, though.
- onlyrealcuzzo 1y agoAnything can be a kind of something since that's subjective...
- croes 1y agoBut it’s more kind of memorization than understanding and reasoning
- rhdjsjebshjffn 1y agoYea, anything can be a kind of something else. :clown: Bruh. Do you need to be paid to interact in good faith or were you raised to be social
- goatlover 1y agoWhy are we imagining? That leads to technologies being overhyped.
- rhdjsjebshjffn 1y agoI can't speak for anyone else, but these models only seem about as smart as google search, with enormous variability. I can't say I've ever had an interaction with a chatbot that's anything redolent of interaction with intelligence. Now would I take AI as a trivia partner? Absolutely. But that's not really the same as what I look for in "smart" humans.
- hmapple 1y agoHave you tried any SOTA models like o3? If not, I strongly encourage you to discuss your area of expertise with it and rate based on that It is incredibly competent
- koakuma-chan 1y ago[flagged]
- levzzz 1y ago[dead]
- rhdjsjebshjffn 1y agoI'm not really sure what to look for, frankly. It makes a rather uninteresting conversation partner and its observations of the world bland and mealy-mouthed. But potentially maybe I'm just not looking for a trivia partner in my software.
- int_19h 1y agoSOTA models can be pretty smart, but this particular model is a very far cry from anything SOTA.
- sureglymop 1y agoThe image description capabilities are pretty insane, crazy to think it's all happening on my phone. I can only imagine how interesting this is accessibility wise, e.g. for vision impaired people. I believe there are many more possible applications for these on a smartphone than just chatting with them.
- nsonha 1y agoYou seem to be the only one expected that model to be "smarter than most human" Leave that part out, I'm excited. I'd love to see this plays some roles in "inference caching", to reduce dependencies on external services. If only agents can plan and match patterns of tasks locally, and only needs real intelligence for doing self-contained/computationally heavy tasks.