4 ms·
> The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the
by dwaltrip 4y ago
> The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the same amount of data and get the same solution.
I'm pretty sure this is completely false?
- pifm_guy 4y agoCorrect. ML papers always try to find the most similar examples in the training set, and rarely find a close match.
- ClueslessTech99 4y agoHow so? What would be the difference? Other than being unviable.
- dwaltrip 4y agoI don’t have a deep understanding of how these language models work, but what do I know makes think that saying “It is basically a lookup table” is sort of like saying “A car is basically a bike”. Sure, there are some very vague similarities, but they are also incredibly different. A bike and a car both take you places. But the kinds of places they can take you and how effectively they do so differs vastly. Likewise, both lookup tables and LLMs take an input and then spit something out. But the similarities basically stop there.