3 ms·

> it comes down to whether it can reason about things and whether it can draw new conclusions or create new information as a result Neither can humans, at leas

> it comes down to whether it can reason about things and whether it can draw new conclusions or create new information as a result Neither can humans, at least with our bare brains. We can do it by carefully observing the effects of our actions in the environment, but we are really studying the world and it takes time. Everything we know comes from the environment. The brain by itself invents or discovers nothing, it is the data-engine made by action-effect-feedback that teaches us all we know. Without the ability to push and prod, set up our experiments and carefully observe effects we wouldn't be at our current level. Environment is the teacher, but there is another important factor - language. Without it every one of us would have to rediscover from scratch. With it we can build upon other people to learn and cooperate. We encode everything we know in language. It acts like an evolutionary system of ideas. LLMs have what is necessary, they can learn language pretty well, but until now have not been exposed much to the world. There are millions of chats but very little in other kinds of environments - computers, simulators, games, robots. LLMs can create their own experiences and learn from each other, and from us. Open ended discovery is a grand project, a social process, it doesn't work well in one agent. Language is the linking element, and the world is the teacher. Some things are not written in any books, only the external world can teach us. Reasoning about things and drawing new conclusions depends on having access to an environment.

unclebucknasty 3y ago
I'm grappling too much with whether this argument has any meaning to give a cogent response. It kind of has a, "yeah, but if reality didn't exist" feel to it. Could humans reason if we somehow lived in a void? I don't know. But, then I guess it really wouldn't matter.
- visarga 3y ago
  You should think about how we get our training data. That is important, we don't have in our DNA that much information, most of it comes by learning. If you don't include the provenance of the information we use to train ourselves you don't really look at the whole process. The brain by itself does nothing just like computers without software and input data. On the other hand scientists need experimentation to come up with anything new. It never comes from pure thinking, there is always the experimental validation. The place we get all our knowledge from is the environment. Without environment as grounding we are just language hallucinators, remember the string theory - that was a nice ungrounded hallucination that kept physicists busy for decades. Without validation our theories are worthless. So a LLM trained purely on human text can't make radical discoveries but when it learns from the environment it can. AlphaGo discovered move 37 from its self-play experience.
  - unclebucknasty 3y ago
    Scientists do not "need experimentation to come up with anything new". The scientific method is explicitly starting with a hypothesis (i.e. something new), then experimenting to validate that hypothesis. Are these hypotheses frequently based on prior observations? Of course. But, reason is still being applied, so I don't know what you're trying to differentiate. You say alphago discovered move 37 from its self-play experience, but that explicitly has nothing to do with the environment. It's the equivalent of a human thinking through various scenarios (without experimentation or interacting with the environment) to come to a new conclusion. The environment is not required for that, which completely argues against what you seem to be saying. BTW, I have not made any statements about AI and reasoning, other than LLMs. AlphaGo, of course, is not LLM-based. Of course we don't all live in individual voids, so much of what we are concerned about are functions of our environments and will be expressed as such. I don't think there's anything groundbreaking in observing that a lot of our reasoning is applied to solving problems that have some intersection with our shared reality.
    - visarga 3y ago
      The go board and the opposing player are an environment. > then experimenting to validate that hypothesis. Proves my point that scientists can't secrete science from their pure brains, the environment is our teacher. We are just prying at its secrets with our limited brains, and usually need many of us to tackle one field, a distributed search for discoveries. None of us is smart enough to get even to Newtonian physics from scratch, we need to stand on the shoulders of previous generations to get anywhere. We're not smart enough to do it without the environment or without a large number of people working together.

unclebucknasty 3y ago

I'm grappling too much with whether this argument has any meaning to give a cogent response. It kind of has a, "yeah, but if reality didn't exist" feel to it. Could humans reason if we somehow lived in a void? I don't know. But, then I guess it really wouldn't matter.

visarga 3y ago
You should think about how we get our training data. That is important, we don't have in our DNA that much information, most of it comes by learning. If you don't include the provenance of the information we use to train ourselves you don't really look at the whole process. The brain by itself does nothing just like computers without software and input data. On the other hand scientists need experimentation to come up with anything new. It never comes from pure thinking, there is always the experimental validation. The place we get all our knowledge from is the environment. Without environment as grounding we are just language hallucinators, remember the string theory - that was a nice ungrounded hallucination that kept physicists busy for decades. Without validation our theories are worthless. So a LLM trained purely on human text can't make radical discoveries but when it learns from the environment it can. AlphaGo discovered move 37 from its self-play experience.
- unclebucknasty 3y ago
  Scientists do not "need experimentation to come up with anything new". The scientific method is explicitly starting with a hypothesis (i.e. something new), then experimenting to validate that hypothesis. Are these hypotheses frequently based on prior observations? Of course. But, reason is still being applied, so I don't know what you're trying to differentiate. You say alphago discovered move 37 from its self-play experience, but that explicitly has nothing to do with the environment. It's the equivalent of a human thinking through various scenarios (without experimentation or interacting with the environment) to come to a new conclusion. The environment is not required for that, which completely argues against what you seem to be saying. BTW, I have not made any statements about AI and reasoning, other than LLMs. AlphaGo, of course, is not LLM-based. Of course we don't all live in individual voids, so much of what we are concerned about are functions of our environments and will be expressed as such. I don't think there's anything groundbreaking in observing that a lot of our reasoning is applied to solving problems that have some intersection with our shared reality.
  - visarga 3y ago
    The go board and the opposing player are an environment. > then experimenting to validate that hypothesis. Proves my point that scientists can't secrete science from their pure brains, the environment is our teacher. We are just prying at its secrets with our limited brains, and usually need many of us to tackle one field, a distributed search for discoveries. None of us is smart enough to get even to Newtonian physics from scratch, we need to stand on the shoulders of previous generations to get anywhere. We're not smart enough to do it without the environment or without a large number of people working together.