3 ms·

It's the mechanical turk (not Amazon's, the medieval one) all over again. Instead of a small dude inside the machine, there is training data. Instead of pulls

It's the mechanical turk (not Amazon's, the medieval one) all over again. Instead of a small dude inside the machine, there is training data. Instead of pulls and levers connected to chess pieces, we have vectors and weights. If we scaled the turk to have millions of people inside, it would appear to be intelligent as well. I don't need it to reproduce everything. Just show me one convincing non-anectodal evidence that this is not another mechanical turk.

ben_w 2y ago
> Instead of pulls and levers connected to chess pieces, we have vectors and weights. And your brain does it with chemistry. The mechanism isn't important. > I don't need it to reproduce everything. Just show me one convincing non-anectodal evidence that this is not another mechanical turk. Define your test. Every single test that existed when I was a kid has been bested. All the new tests from all the domain experts to measure AI, are getting rapidly saturated. https://ourworldindata.org/grapher/test-scores-ai-capabilities-relative-human-performance?country=Handwriting+recognition~Speech+recognition~Image+recognition~Reading+comprehension~Language+understanding~Predictive+reasoning~Code+generation~Complex+reasoning~General+knowledge+tests~Math+problem-solving~Nuanced+language+interpretation~Reading+comprehension+with+unanswerable+questions https://ourworldindata.org/grapher/test-scores-ai-capabiliti... Coming up with a test is itself a challenge, these days.
- alganet 2y ago
  > And your brain does it with chemistry. Alledgedly. > Define your test. A pass is a model with no human training data input that is able to reason on human content. > Every single test that existed when I was a kid has been bested. The mechanical turk fooled humans for 80 years and passed all the tests at its time. As it turns out, the tests were not good enough. > Coming up with a test is itself a challenge, these days. My test is simple to understand and conceive. We just don't have any valid candidate to put through it. My requirements disqualify LLMs.
  - ben_w 2y ago
    > Alledgedly If you can't agree on our brains being chemical in nature, you're engaged in magical thinking and we have no more to discuss. > A pass is a model with no human training data input that is able to reason on human content. What does it mean to "reason on human content"? Both "reason" and separately "human content"? The former because some argue that "reason" is the mechanical application of logic, where a single transistor alone would be sufficient to pass; others require it to be a conscious application, at which point you've got to deal with there being about 40 different definitions of "consciousness". The latter, because a model with no human training data will not be able to read any human language for the same reason that no human alive can read Linear A… but also, our content includes the games we play, and AI trained on the rules of Go with zero examples of our games beat human Go players at the game of Go, which definitely seems to me to be doing things on "human content". > My test is simple to understand and conceive. We just don't have any valid candidate to put through it. My requirements disqualify LLMs. Where did you write this "simple to understand and conceive" test? Because what you've given here is not even a test. If you can't come up with a test, there's nothing to distinguish you personally from the AI that you're dismissing.
    - alganet 2y ago
      > If you can't agree on our brains being chemical in nature, you're engaged in magical thinking and we have no more to discuss. I agree that our brains have a lot of chemistry. I can't deny that. Science shows we can mess up brains by messing up their chemistry. So far, though, we still don't understand how it works. We can't use chemistry to improve it, just to break it or fix it. That alone should be a huge red flag for the reductionist point of view. It's our best hypothesis, but is not good enough to travel disciplines to IT. We don't know enough to compare the two, hence the skepticism. > What does it mean to "reason on human content"? Pass the turing test should be enough. I'm just changing the requirements (no tiny humans inside, complete or otherwise). > no human alive can read Linear A If we had a living human fluent in Linear A, I believe we could decipher it. Machines have us, living humans speaking a living language, to learn. > If you can't come up with a test, there's nothing to distinguish you personally from the AI that you're dismissing. I can. Open the machine and prove me that there are no tiny humans or human pieces inside.

ben_w 2y ago

> Instead of pulls and levers connected to chess pieces, we have vectors and weights. And your brain does it with chemistry. The mechanism isn't important. > I don't need it to reproduce everything. Just show me one convincing non-anectodal evidence that this is not another mechanical turk. Define your test. Every single test that existed when I was a kid has been bested. All the new tests from all the domain experts to measure AI, are getting rapidly saturated. https://ourworldindata.org/grapher/test-scores-ai-capabilities-relative-human-performance?country=Handwriting+recognition~Speech+recognition~Image+recognition~Reading+comprehension~Language+understanding~Predictive+reasoning~Code+generation~Complex+reasoning~General+knowledge+tests~Math+problem-solving~Nuanced+language+interpretation~Reading+comprehension+with+unanswerable+questions https://ourworldindata.org/grapher/test-scores-ai-capabiliti... Coming up with a test is itself a challenge, these days.

alganet 2y ago
> And your brain does it with chemistry. Alledgedly. > Define your test. A pass is a model with no human training data input that is able to reason on human content. > Every single test that existed when I was a kid has been bested. The mechanical turk fooled humans for 80 years and passed all the tests at its time. As it turns out, the tests were not good enough. > Coming up with a test is itself a challenge, these days. My test is simple to understand and conceive. We just don't have any valid candidate to put through it. My requirements disqualify LLMs.
- ben_w 2y ago
  > Alledgedly If you can't agree on our brains being chemical in nature, you're engaged in magical thinking and we have no more to discuss. > A pass is a model with no human training data input that is able to reason on human content. What does it mean to "reason on human content"? Both "reason" and separately "human content"? The former because some argue that "reason" is the mechanical application of logic, where a single transistor alone would be sufficient to pass; others require it to be a conscious application, at which point you've got to deal with there being about 40 different definitions of "consciousness". The latter, because a model with no human training data will not be able to read any human language for the same reason that no human alive can read Linear A… but also, our content includes the games we play, and AI trained on the rules of Go with zero examples of our games beat human Go players at the game of Go, which definitely seems to me to be doing things on "human content". > My test is simple to understand and conceive. We just don't have any valid candidate to put through it. My requirements disqualify LLMs. Where did you write this "simple to understand and conceive" test? Because what you've given here is not even a test. If you can't come up with a test, there's nothing to distinguish you personally from the AI that you're dismissing.
  - alganet 2y ago
    > If you can't agree on our brains being chemical in nature, you're engaged in magical thinking and we have no more to discuss. I agree that our brains have a lot of chemistry. I can't deny that. Science shows we can mess up brains by messing up their chemistry. So far, though, we still don't understand how it works. We can't use chemistry to improve it, just to break it or fix it. That alone should be a huge red flag for the reductionist point of view. It's our best hypothesis, but is not good enough to travel disciplines to IT. We don't know enough to compare the two, hence the skepticism. > What does it mean to "reason on human content"? Pass the turing test should be enough. I'm just changing the requirements (no tiny humans inside, complete or otherwise). > no human alive can read Linear A If we had a living human fluent in Linear A, I believe we could decipher it. Machines have us, living humans speaking a living language, to learn. > If you can't come up with a test, there's nothing to distinguish you personally from the AI that you're dismissing. I can. Open the machine and prove me that there are no tiny humans or human pieces inside.