3 ms·
> but it’s probably good to know LLMs are not just lookup tables. But there are almost literally lookup tables actually! The thing is they don't perform the l
by littlestymaar 3y ago
> but it’s probably good to know LLMs are not just lookup tables.
But there are almost literally lookup tables actually!
The thing is they don't perform the lookup on a single word as a key, but on the entire context, which is what makes them special. But besides that it's “just” a lookup table.
- kibibu 3y agoA deep attention network is absolutely not a lookup table
- holoduke 3y agoMultiple lookup tables
- mpeg 3y agoBit of an oversimplification though, no one would look at Postgres and go “it’s just a couple lookup tables put together”
- dragonwriter 3y agoAs an implementation? No. For fundamental understanding of the logical model? That its one big lookup table with a particular form of key nesting is... actually a pretty good model.
- littlestymaar 3y agoThat's exactly what DB indexes are though ;)
- littlestymaar 3y agoAn attention head is quite literally a lookup table!
- _akhe 3y agoExactly right. The others here are confusing LLM with "chat bot", and also they seem to be confusing token prediction with LLM. I have a feeling the mainstream won't really get it until a ChatGPT clone is online and ready to use lol and still it will be "This isn't a true chat bot, this is actually a Markovian language interface!"