6 ms·

Humans Who Are Not Concentrating Are Not General Intelligences

76 points by andreyk 7y ago

xtiansimon 7y ago
Now if GPT-2 could write yet another beginner Python lists and tuples blog post, I prolly wouldn’t notice. If it could write a description which helped me to get my head around my client’s Swagger API I would be thrilled. No one really has the time nor patience to explain it to me in a way that clicks.
lostmsu 7y ago
Dupe https://news.ycombinator.com/item?id=19251755 https://news.ycombinator.com/item?id=19251755 Also a copy-paste of https://srconstantin.wordpress.com/2019/02/25/humans-who-are-not-concentrating-are-not-general-intelligences/ https://srconstantin.wordpress.com/2019/02/25/humans-who-are...
- andreyk 7y ago
  For some context, as noted at top of article we re-posted it with permission since the release of the larger GPT2 model led to a new slate of fearful articles on the topic and this has remained one of the best takes on it. Was not aware it was already upvoted a bunch on HN before though, cool to see.
  - lostmsu 7y ago
    The guideline on HN is to use original link whenever possible. The link should be changed.
    - andreyk 7y ago
      ah, whoops, was not aware. Seems I can't change the link, maybe mods can or can just delete.
jerf 7y ago
If you'd like to innoculate yourself, and have a bit of fun in the meantime, consider reading https://www.reddit.com/r/SubSimulatorGPT2/ https://www.reddit.com/r/SubSimulatorGPT2/ . It's not just for fun, you can get a good sense of the algorithm. One of the things it is somewhat prone to is some weird looping, like this: https://www.reddit.com/r/SubSimulatorGPT2/comments/d1nwdg/if_the_president_of_the_united_states_was_your/ezo3sce/ https://www.reddit.com/r/SubSimulatorGPT2/comments/d1nwdg/if... in which the algorithm generates the sentence "Toss some leeches around and wait 'til we get there." (no, it does not make any more sense in context), and then repeats that sentence nearly (but not quite!) exactly 23 more times. (I expect this is a consequence of the way it is tracking some internal state; I assume these sentences are strange attractors in some sort of state that is getting iteratively modified.) You can also see that while it picks up some deep structure, a check of anything trained on /r/jokes (https://www.reddit.com/r/SubSimulatorGPT2/comments/d055mt/a_guy_is_having_a_hard_time_with_his_wifes/?utm_source=share&utm_medium=web2x https://www.reddit.com/r/SubSimulatorGPT2/comments/d055mt/a_... ) or /r/math (https://www.reddit.com/r/SubSimulatorGPT2/comments/d1yz1e/how_exactly_does_the_number_1_not_equal_2/ https://www.reddit.com/r/SubSimulatorGPT2/comments/d1yz1e/ho... ) the algorithm is definitely unable to deal with deeper structure right now. The /r/jokes bot is humorous in its complete lack of humor, I mean, well beyond any sarcastic snark about how unfunny /r/jokes may be. It has the structure of jokes. There was one recent one that even asked "What's a pirate's favorite letter?", and the bot had noticed the answer was being given in the form of letters, but I don't think a single instance of the bot proposed "r". But it does not understand humor in the slightest. Of the several dozen attempts at jokes I've at least skimmed, I believe it only achieved something that was at least recognizable as an attempt at humor once, and it still wasn't that funny. Likewise math. It's got a good idea there's these "prime number" things and they're pretty important, but I've seen at least half-a-dozen wrong definitions of what one is. It's a very interesting algorithm. It's a great babbler. But on its own, it's not a great solution to generating text. Although it may very well be able to generate text that can pass a casual skim text, as the article suggests. Still, it takes human curation to get that far. Any human that can read is going to guess something's inhuman about repeating "Toss some leeches around and wait 'til we get there." 24 times in a row.
- jborichevskiy 7y ago
bitL 7y ago
I am thinking about using GPT-2 or better to write homeworks for my MBA... Nobody would spot a difference (I am worried/relieved).
- julienreszka 7y ago
  Do it
Fnoord 7y ago
Can someone explain the title? It doesn't make sense to me.
- visarga 7y ago
  I think an even stronger version applies. Humans are not general intelligences. We are just good at keeping ourselves alive and making more of us on this planet (and in this ecosystem). All the rest - language, science, culture, economy - are just our current solution to solving the constraints of life. In order for an intelligence to be called general it would be necessary to be effective in all situations. Humans only perceive the world through our five limited senses and then filters it through the coloured glass of our concepts. We can't escape the limitations of our senses and mental models taken together. Humans can't even keep in the working memory more than 7 objects at once, some people can handle a few more but not on the order of hundreds or thousands. What if the real ultimate theory of physics required a working memory of 1000 objects? We'd be forever blocked from grasping it like an ant vs. the stock-market. Programmers live at the edge of this grasping power and know the horror of not being able to take it all in at once, and it's so easy to get into such a situation. It is possible that there is no general intelligence anywhere. It's always an intelligence of a specific environment, solving specific types of problems. A general intelligence would need a much more varied and challenging environment in order to reach that level of intelligence. The more complex the environment, the higher the intelligence of its agents. So there is always going to be an upper limit to intelligence, and the environment has a lot to do with it. No intelligence is truly general.
  - deleted 7y ago
    [deleted]
  - partyboat1586 7y ago
    A true general intelligence would need to handle any environment. The only such intelligence would be universal intelligence which has experienced every possible environment. Unfortunately this kind of intelligence is cheating because it's not really an agent and since it encapsulates everything there are no problems for it to solve.
humble_engineer 7y ago
When I visited a famous AI researcher in San Francisco everyone wanted to ask him when the machines would become sentient, but I asked him when are humans going to realize they are machines?
JackFr 7y ago
Makes me think of "Hook" by Blues Traveler: It doesn't matter what I say So long as I sing with inflection That makes you feel I'll convey Some inner truth or vast reflection But I've said nothing so far And I can keep it up for as long as it takes And it don't matter who you are If I'm doing my job then it's your resolve that breaks
EchoAce 7y ago
> “I’ve taught public school teachers, who were incredibly bad at formal mathematical reasoning (I know, because I graded their tests), to the point that I had not realized humans could be that bad at math” Is it just me or does this come off as incredibly rude? It’s one thing to say people are bad at math but the phrasing of this seems insulting without reason, in my opinion.
- Chlorus 7y ago
  It just reeks of the same lazy garbage of that whole "separating the programming goats from the sheep" meme that went around a few years back. "It's not that my pedagogy has faults, people are just inherently bad and there's no point trying to teach them"
  - Sniffnoy 7y ago
    She doesn't claim that she couldn't teach them. She claimed that they were bad at formal reasoning.
- _bxg1 7y ago
  I mean, she didn't name names. I don't think there's anything wrong with being blunt if it explicitly directed at individuals.
Geee 7y ago
I've certainly noticed that most people speak on auto-pilot, basically just repeating what they've heard or read. I wonder if just a few % of people are actually generally intelligent and everyone else is following along. Or is it just a random process, in which we are just repeating each other and making mistakes leads to new ideas.
- GaryNumanVevo 7y ago
  It's an energy problem. Take the average mental energy level and divide it up among all the facets of daily life. Most of that is probably going to go into working, eating, relaxing. Developing original opinions in every single field isn't realistic, so we'll find someone with authority and repeat what they said.
fallingfrog 7y ago
I’ve definitely had the experience of reading a human written paper, and just skimming it because it didn’t really seem to have a point. Then I sighed and decided to really give it my attention and quickly realized that the reason it was hard to read was because it was chock full of lazy thinking, bad analogies, unexamined assumptions, and non sequiturs to begin with.
Shorel 7y ago
Now, make an hybrid system merging this text generator with the classic symbolic AI called CYC from Doug Lenat. It could be able to generate all our news articles in whatever style we prefer. Bye bye freelancer writers.

76 points by andreyk 7y ago

Now if GPT-2 could write yet another beginner Python lists and tuples blog post, I prolly wouldn’t notice. If it could write a description which helped me to get my head around my client’s Swagger API I would be thrilled. No one really has the time nor patience to explain it to me in a way that clicks.

lostmsu 7y ago

Dupe https://news.ycombinator.com/item?id=19251755 https://news.ycombinator.com/item?id=19251755 Also a copy-paste of https://srconstantin.wordpress.com/2019/02/25/humans-who-are-not-concentrating-are-not-general-intelligences/ https://srconstantin.wordpress.com/2019/02/25/humans-who-are...

andreyk 7y ago
For some context, as noted at top of article we re-posted it with permission since the release of the larger GPT2 model led to a new slate of fearful articles on the topic and this has remained one of the best takes on it. Was not aware it was already upvoted a bunch on HN before though, cool to see.
- lostmsu 7y ago
  The guideline on HN is to use original link whenever possible. The link should be changed.
  - andreyk 7y ago
    ah, whoops, was not aware. Seems I can't change the link, maybe mods can or can just delete.

jerf 7y ago

If you'd like to innoculate yourself, and have a bit of fun in the meantime, consider reading https://www.reddit.com/r/SubSimulatorGPT2/ https://www.reddit.com/r/SubSimulatorGPT2/ . It's not just for fun, you can get a good sense of the algorithm. One of the things it is somewhat prone to is some weird looping, like this: https://www.reddit.com/r/SubSimulatorGPT2/comments/d1nwdg/if_the_president_of_the_united_states_was_your/ezo3sce/ https://www.reddit.com/r/SubSimulatorGPT2/comments/d1nwdg/if... in which the algorithm generates the sentence "Toss some leeches around and wait 'til we get there." (no, it does not make any more sense in context), and then repeats that sentence nearly (but not quite!) exactly 23 more times. (I expect this is a consequence of the way it is tracking some internal state; I assume these sentences are strange attractors in some sort of state that is getting iteratively modified.) You can also see that while it picks up some deep structure, a check of anything trained on /r/jokes (https://www.reddit.com/r/SubSimulatorGPT2/comments/d055mt/a_guy_is_having_a_hard_time_with_his_wifes/?utm_source=share&utm_medium=web2x https://www.reddit.com/r/SubSimulatorGPT2/comments/d055mt/a_... ) or /r/math (https://www.reddit.com/r/SubSimulatorGPT2/comments/d1yz1e/how_exactly_does_the_number_1_not_equal_2/ https://www.reddit.com/r/SubSimulatorGPT2/comments/d1yz1e/ho... ) the algorithm is definitely unable to deal with deeper structure right now. The /r/jokes bot is humorous in its complete lack of humor, I mean, well beyond any sarcastic snark about how unfunny /r/jokes may be. It has the structure of jokes. There was one recent one that even asked "What's a pirate's favorite letter?", and the bot had noticed the answer was being given in the form of letters, but I don't think a single instance of the bot proposed "r". But it does not understand humor in the slightest. Of the several dozen attempts at jokes I've at least skimmed, I believe it only achieved something that was at least recognizable as an attempt at humor once, and it still wasn't that funny. Likewise math. It's got a good idea there's these "prime number" things and they're pretty important, but I've seen at least half-a-dozen wrong definitions of what one is. It's a very interesting algorithm. It's a great babbler. But on its own, it's not a great solution to generating text. Although it may very well be able to generate text that can pass a casual skim text, as the article suggests. Still, it takes human curation to get that far. Any human that can read is going to guess something's inhuman about repeating "Toss some leeches around and wait 'til we get there." 24 times in a row.

jborichevskiy 7y ago

bitL 7y ago

I am thinking about using GPT-2 or better to write homeworks for my MBA... Nobody would spot a difference (I am worried/relieved).

julienreszka 7y ago
Do it

Fnoord 7y ago

Can someone explain the title? It doesn't make sense to me.

visarga 7y ago
I think an even stronger version applies. Humans are not general intelligences. We are just good at keeping ourselves alive and making more of us on this planet (and in this ecosystem). All the rest - language, science, culture, economy - are just our current solution to solving the constraints of life. In order for an intelligence to be called general it would be necessary to be effective in all situations. Humans only perceive the world through our five limited senses and then filters it through the coloured glass of our concepts. We can't escape the limitations of our senses and mental models taken together. Humans can't even keep in the working memory more than 7 objects at once, some people can handle a few more but not on the order of hundreds or thousands. What if the real ultimate theory of physics required a working memory of 1000 objects? We'd be forever blocked from grasping it like an ant vs. the stock-market. Programmers live at the edge of this grasping power and know the horror of not being able to take it all in at once, and it's so easy to get into such a situation. It is possible that there is no general intelligence anywhere. It's always an intelligence of a specific environment, solving specific types of problems. A general intelligence would need a much more varied and challenging environment in order to reach that level of intelligence. The more complex the environment, the higher the intelligence of its agents. So there is always going to be an upper limit to intelligence, and the environment has a lot to do with it. No intelligence is truly general.
- deleted 7y ago
  [deleted]
- partyboat1586 7y ago
  A true general intelligence would need to handle any environment. The only such intelligence would be universal intelligence which has experienced every possible environment. Unfortunately this kind of intelligence is cheating because it's not really an agent and since it encapsulates everything there are no problems for it to solve.

humble_engineer 7y ago

When I visited a famous AI researcher in San Francisco everyone wanted to ask him when the machines would become sentient, but I asked him when are humans going to realize they are machines?

JackFr 7y ago

Makes me think of "Hook" by Blues Traveler: It doesn't matter what I say So long as I sing with inflection That makes you feel I'll convey Some inner truth or vast reflection But I've said nothing so far And I can keep it up for as long as it takes And it don't matter who you are If I'm doing my job then it's your resolve that breaks

EchoAce 7y ago

> “I’ve taught public school teachers, who were incredibly bad at formal mathematical reasoning (I know, because I graded their tests), to the point that I had not realized humans could be that bad at math” Is it just me or does this come off as incredibly rude? It’s one thing to say people are bad at math but the phrasing of this seems insulting without reason, in my opinion.

Chlorus 7y ago
It just reeks of the same lazy garbage of that whole "separating the programming goats from the sheep" meme that went around a few years back. "It's not that my pedagogy has faults, people are just inherently bad and there's no point trying to teach them"
- Sniffnoy 7y ago
  She doesn't claim that she couldn't teach them. She claimed that they were bad at formal reasoning.
_bxg1 7y ago
I mean, she didn't name names. I don't think there's anything wrong with being blunt if it explicitly directed at individuals.

Geee 7y ago

I've certainly noticed that most people speak on auto-pilot, basically just repeating what they've heard or read. I wonder if just a few % of people are actually generally intelligent and everyone else is following along. Or is it just a random process, in which we are just repeating each other and making mistakes leads to new ideas.

GaryNumanVevo 7y ago
It's an energy problem. Take the average mental energy level and divide it up among all the facets of daily life. Most of that is probably going to go into working, eating, relaxing. Developing original opinions in every single field isn't realistic, so we'll find someone with authority and repeat what they said.

fallingfrog 7y ago

I’ve definitely had the experience of reading a human written paper, and just skimming it because it didn’t really seem to have a point. Then I sighed and decided to really give it my attention and quickly realized that the reason it was hard to read was because it was chock full of lazy thinking, bad analogies, unexamined assumptions, and non sequiturs to begin with.

Shorel 7y ago

Now, make an hybrid system merging this text generator with the classic symbolic AI called CYC from Doug Lenat. It could be able to generate all our news articles in whatever style we prefer. Bye bye freelancer writers.