5 ms·
> The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics. So what? Can you propose another method to ma
by Eisenstein 2y ago
> The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics.
So what? Can you propose another method to make a computing device understand language? The method of the creation of the output does not stipulate anything about the nature of the thing creating it. If someone could map out a human brain and tell you how thoughts are made and added a 'all that is being done is' in front of it, does that make your thought creation trivial?
> An LLM on top of an LLM (i.e using context to generate inputs to an LLM) is just a fancier LLM.
This is called a tautology. You have not given any compelling reasons why an LLM cannot do anything, so calling something another LLM is not compelling either.
> To really understand all of this, all you need to do is look at how Transformer works, namely the attention block. There is no such thing as Query, Key, and Value in the sense of how they are implied to be used. The may as well be called A,B,C, as they are all learned in training, and can be freely interchanged in naming. All you do for inference is multiply the output vector by A,B,C to get 3 matrices, then multiply them together (technically with a scaling factor for 2 of them, but again, doesn't matter for which 2, and the scaling factor can be built into the matrix itself)
Here is how it works, so therefore it must meet some criteria I have imposed arbitrarily.
> So we have to rely on hacks like temperature to make it appear like reasoning, when its really not even close.
You still haven't produced any valid argument at all, for why one thing would be evidence of the other.
- ActorNightly 2y agoA good example of how to type a comment and yet not say anything. It should be pretty clear to anyone that human brains aren't just one giant compute functions with a limited set of outputs. There is no concept in your or my brain what 12074389762193867*2398720876324 is, but we can certainly figure it out, some even with good memory with complete sensory depravation. If you disagree with this, you are entitled to your opinion, but your comments on the state of AI are just irrelevant.
- Eisenstein 2y agoI don't remember making any comments on A.I. My post was pointing out basic flaws in reasoning and trying to provoke some thought about something where it appeared to be lacking. Unfortunately I did not succeed, since a myopic view made you hallucinate that I was saying something definitive about something I was not. Irrelevant, indeed.
- synarchefriend 2y agoo1 can also solve many arbitrary math problems that it could not have possibly seen in its training data. And it shows the steps that it uses to do so. How do you explain this without reasoning?