3 ms·
Why no one wants to hear what I'm saying? :) Since the beginning, I say, "what we say is the result of our neurons and their build up connections in our brain.
by theGeatZhopa 2y ago
Why no one wants to hear what I'm saying? :)
Since the beginning, I say, "what we say is the result of our neurons and their build up connections in our brain. Our brain consists of trillions of connections. These trillions are needed to get information from our "body" and to control it, to access knowledge, for reasoning... Just like our eyesight have its own Data-Autobahn directly into the brain, or, our hearing, tasting, tactile sensations.. all of that needs to be processed. That makes us individuals and makes us humans.
If one takes a look at an LLM, it isn't human. It doesn't walk around, it doesn't digest food for energy, it doesn't have to control foot and hand or the movement of the eyes/muscles.. so, take a human brain and remove all for such tasks necessary neurons and synapses. What will you have after? A brain consisting of neurons that "hold" our memories and our language expressiveness/knowledge in general. It doesn't hold anymore any neurons that are necessary to make a loud and to make the sound coming out of our mouths. Because LLM doesn't have that, too.
Then, you freeze that brain in it's state so it won't learn new things/build up new knowledge.
If you now think of a Interface to such a frozen state brain, so you can talk to it. Whenever you ask it something, what will it answer you? .. the same as another human being with the same knowledge would answer or the same what a human is reasoning about... In the beginning, it was a human brain, though.
Ok.. now we're here:
Everything we say and hear can also be written down on paper. When someone is reasoning about the AGI and writing an essay about it, that essay will have the writers chain of thoughts, his chosen vocabulary, put into a sentence by a ruling of the language's grammer. If you ask the Thinker & Writer few weeks later about his reasoning, he will bring up the same arguments (except new things and insights have been built up in between time). So, reasoning and thinking is a process strongly dictated by knowledge and insight.
Now. If one writes a book. That book will contain one's knowledge within a topic. Another one could read it and by that build up new knowledge and new insights and reasoning, basically learning from the first one.
Ok.. so far..
If you take this book now and index each argument. Then, some other asks through a Python script "give me index' number 4 contents." and get it returned exactly as it's written in the book (which is the factual chain of thoughts of the author) - you can't distinguish, whether the author responded by himself or it's a part of his book. What makes us able to differentiate it are the small deviations in language the real author may use. But if we build in a little variance words like "hmm.." "umpf.." "err...".. into the return of the python script - when the author uses this in his spoken or written answers- we can't distinguish machine or human, again...
So, the discussion should turn to "does hopping from neuron to neuron over kilometers of synapse connections to achieve a possible output makes us humans?"
If, now, one takes a look onto LLM. They use tokens. They use heuristics. But, what difference is there except the kilometers of synapse links? Neurons can have multiple in/out connections. A token does have it's probabilities.
One Neuron activates another one. A tokens probabilities are used to get from that token to the next token (to the one with the best probability).. as would a neuron do, too:
One Neuron starts to fire/ signaling depending on it's inputs strength and all the subsequent, connected to it, Neurons will also start to fire after receiving the signals. With their own different strength of the signal (!) which is encoded by the amount of incoming signals from a lot of connections.
So, how is the heuristic approach of probabilities is different from "hard coded" connections within our brains?:
Its less flexible in terms of the sensitivity:
Take two neurons. One fires, the other receive and fire too after that.
Take one token and have the probability set to 1 for next token in this simplification. You'll get the next token.
Now, extended this experiment of thoughts to three Neurons and three tokens. While the two Neurons connected to the first neuron are starting to fire at same strength upon "receiving" the first' signals , what will happen with the LLM tokens, where the probabilities are equal. You'll get two tokens returned.
And now, extend it to millions of millions. Our Neurons in our brain will have different connections and different "link-speeds". If one fires, a cascade will be started.
The tokens, however, are not built this way. They don't start to fire. They just give the next token, the one with the highest probability. But, in the end, it will be that token returned, that is necessary to shape a human understandable word, like „wo“ + "rde" -> „worde”!
It wasn't necessary to start a cascade of neurons firing to get to the end.
Probabilities are here the replacement of the cascade necessary to activate that one specific neuron, that completes, what the first neuron started.
Obviously, the difference is cascade & no cascade.
If one takes a look on the interlinking and the cascade in our brain. This empowers the brain to very very very very finely control which of the neurons in the brain will be activated. Probabilities can't do that.. as long they're not very granular defined..
In our thoughts experiment, reduce the amount of millions of Neurons back to three, but, assume their connections are very long and amount, having a lot of other neurons involved (but we now don't pay attention to this network). So you'll have slightly different situation, where the first one is firing, the Cascade does what it should, and as a result, one of the two successive neurons get a higher signal..
To reproduce this with tokens, one needs to reproduce the cascade in the background. But, still without the tokens firing..
One needs to train from small to big. On the same data. How ever it could be done.. not the problem of the thought experiment right now :)
But, if you achieve that. You'll have a complete representation of the neural network, but without involving cascades of links and Neurons.
So, in my thoughts, the probabilities are just a shortcut, a shorter way "to get to that one certain Neurons" from the starting Neuron on..
LLMs should be perfectly able to mimic the thoughts of human beings. They are trained on the very same, if it was written down and trained on it.. like a human can too mimic another human and a third one could never tell the difference, if the conversation is happening written instead of face to face talking."
So, the LLMs needs more training and should be based on the atomic parts of language - the chars. When there are other languages involved into the training, that would be, too, a good.
.. "
Why no one wants to hear what I say! Hahahaha