3 ms·
Scientists Made a Mind-Bending Discovery About How AI Works
- sharemywin 4y agoThe researchers conducted their experiment by giving the model synthetic data, or prompts the program never could have seen before. Despite this, the language model was able to generalize and then extrapolate knowledge from them, said Akyürek. This led the team to hypothesize that AI models that exhibit in-context learning actually create smaller models inside themselves to achieve new tasks.
- ImHereToVote 4y agoI guess information processing has tipping points similar to matter.
- aaroninsf 4y agoMaybe I'm wrong, but this does not seem so much discovery as confirmation and more of simple illumination of something that has appeared to me at least to be self-evident, so much so that it's a core part of my explanation to lay people about what makes contemporary LLM and the like so powerful. I.e. it's a core feature, indeed it's the entire reason you have deep networks, to allow for abstractions. IMO the interesting ChatGPT stuff output is largely in this domain, it's quite obvious that much of what it is doing is not doable without something homologous to an internal representation.
- devmor 4y agoYeah this does not seem like a novel or shocking discovery. Perhaps I'm biased because I've been working with ML for so long but my first thought was "Well, yeah that's the point."
- mock-possum 4y agoIs it just me, or is this a real poorly-written article? Like it doesn’t seem to be saying much of anything at all, let alone deliver on the laughably aggrandized title. :/