3 ms·

>Yes, this is a neural net model, that is what such models do and have done for decades already. I'm not sure why this is relevant. Do you argue that stable dif

>Yes, this is a neural net model, that is what such models do and have done for decades already. I'm not sure why this is relevant. Do you argue that stable diffusion is intelligent since it has emergent structures? Or an image recognition system is intelligent since it has emergent structures? Those are the same things. No I am saying there are models for intelligence within the neural net that is explicitly different from a stochastic parrot for english vocabulary. For example in one instance they identified a structure in an LLM that logically models the rules and strategy for an actual board game. Obviously I'm not referring to papers on plain old "neural networks" that shit is old news. I'm referring to new research on LLMs. Again I can provide you with papers provided you want evidence that will flip your stubborn viewpoint on this. It just depends on if your bias is flexible enough to accept such a sudden deconstruction of your own stance.

Jensson 4y ago
The fact that it adapts it state to fit the data isn't interesting in itself. An image recognition system forms a lot of macro structures around shapes or different logical parts of the image. Similarly an LLM forms a lot of macro structures around different kinds of text structures or words, including chess game series or song compositions or programming language tutorials. It is exactly the same kind of structures, just that some thinks those structures are sign of intelligence when they are applied to texts. Can such macro structures model intelligence in theory? Yes. But as we see in practice they aren't very logical. For example in this article we see that its markov chain didn't have enough programming language descriptions, so it veered into printing brace scoped code when it said the language had whitespace based scoping. Similarly in popular puzzles, just change the words around and it will start printing nonsense since it cares about what words you use and not what those words mean. Edit: Point is that existence of such structures doesn't make a model smart. You'd need to prove that these structures are smarter than before.
- HervalFreire 4y ago
  >Can such macro structures model intelligence in theory? Yes. So you agree it's possible. >Yes. But as we see in practice they aren't very logical. For example in this article we see that its markov chain didn't have enough programming language descriptions, Well as humans we have many separate models for all different components an aspects of the world around us. Clearly LLMs form many models that in practice are not accurate. But that does not mean all the models are defective. The fact that it can write blog posts indicates that many of these models are remarkably accurate and that it understands the concept of a "blog post". There is literal evidence of chatGPT answering questions as if it has an accurate underlying model of "understanding" as well as actual identified structures within the nerual net itself. There is also evidence for your point of chatGPT clearly forming broken and inaccurate models by answering questions with wrong answers that don't make sense. What gets me is that even when there is Clear and abundant evidence for both cases some people have to make the claim that chatGPT doesn't understand anything. The accurate answer is that LLMs understand some things and it doesn't understand other things.

Jensson 4y ago

The fact that it adapts it state to fit the data isn't interesting in itself. An image recognition system forms a lot of macro structures around shapes or different logical parts of the image. Similarly an LLM forms a lot of macro structures around different kinds of text structures or words, including chess game series or song compositions or programming language tutorials. It is exactly the same kind of structures, just that some thinks those structures are sign of intelligence when they are applied to texts. Can such macro structures model intelligence in theory? Yes. But as we see in practice they aren't very logical. For example in this article we see that its markov chain didn't have enough programming language descriptions, so it veered into printing brace scoped code when it said the language had whitespace based scoping. Similarly in popular puzzles, just change the words around and it will start printing nonsense since it cares about what words you use and not what those words mean. Edit: Point is that existence of such structures doesn't make a model smart. You'd need to prove that these structures are smarter than before.

HervalFreire 4y ago
>Can such macro structures model intelligence in theory? Yes. So you agree it's possible. >Yes. But as we see in practice they aren't very logical. For example in this article we see that its markov chain didn't have enough programming language descriptions, Well as humans we have many separate models for all different components an aspects of the world around us. Clearly LLMs form many models that in practice are not accurate. But that does not mean all the models are defective. The fact that it can write blog posts indicates that many of these models are remarkably accurate and that it understands the concept of a "blog post". There is literal evidence of chatGPT answering questions as if it has an accurate underlying model of "understanding" as well as actual identified structures within the nerual net itself. There is also evidence for your point of chatGPT clearly forming broken and inaccurate models by answering questions with wrong answers that don't make sense. What gets me is that even when there is Clear and abundant evidence for both cases some people have to make the claim that chatGPT doesn't understand anything. The accurate answer is that LLMs understand some things and it doesn't understand other things.