3 ms·
> Some will be retrieving that from memory, but many will be using a generative process that generates the same code, because they've seen and done similar thi
by throwaway413 5y ago
> Some will be retrieving that from memory, but many will be using a generative process that generates the same code, because they've seen and done similar things thousands of times before.
How are these not both the exact same process of memory recollection? Can you elaborate on the difference between memory recall vs a generative process based on conditioning? I understand how these two are different in application, but not understand why one would say they are fundamentally different processes.
- nl 5y agoAnalogies start to break down once we are talking at this detailed level. The best I can come up with is this: Imagine you are implementing a system to give the correct answer to the addition of any two numbers between 1 and 100. One way to implement it would be to build a large database, loaded with "x" and "y" and their sum. Then when you want to find out what 1 + 2 is you do a lookup. The other method is to implement a "sum" function. Both give the same results. The first process is a database lookup, the second is akin to a generative process because it's doing calculation to come up with correct result. This analogy breaks down because a NN does have a token lookup as well. But the probabilistic computation is the major part of how a NN works, not the lookup part.