5 ms·

Ask any purported “AGI” this simple IQ test question: What is the shortest python program you can come up with that outputs: 000000000100010000110010000101001

Ask any purported “AGI” this simple IQ test question: What is the shortest python program you can come up with that outputs: 0000000001000100001100100001010011000111010000100101010010110110001101011100111110000100011001010011101001010110110101111100011001110101101111100111011111011111 For background on this kind of question see Shane Legg's (now ancient) lecture on measures of machine intelligence: https://youtu.be/0ghzG14dT-w?t=890 https://youtu.be/0ghzG14dT-w?t=890 It's amazing after all this time that people are _still_ trying to discover what Solomonoff proved over a half century ago.

vuln 3y ago
chatGPT-4 Result: Sure, you can use the following Python program to output the string you provided: ```python print("0000000001000100001100100001010011000111010000100101010010110110001101011100111110000100011001010011101001010110110101111100011001110101101111100111011111011111") ``` This is the simplest and most direct method to output the string. If you have a more complex task in mind, like generating this string according to a certain pattern, please provide more details.
- jabowery 3y ago
  The "more complex task in mind" was, of course, to generate the "shortest" program. GPT-4, by asking for a "certain pattern" is attempting to have you do the intellectual heavy lifting for it -- although in this case the intellectual lifting is quite light.
  - blowski 3y ago
    I really don't understand your requirements.
- deleted 3y ago
  [deleted]
- arp242 3y ago
  This is shorter for starters: print(bin(0x443214c74254b635cf84653a56d7c675be77df)[2:]) May be possible to shave off a few bytes with f'..' strings, or see if there are any repeating patterns, I'm not the sort who enjoys "code golfing", but "use base-16 to represent a base-2 number more compactly" seems fairly obvious to me.
  - jabowery 3y ago
    Wrong output. What you call "code golf" is the essence of the natural sciences: Inducing natural laws from the data generated by those natural laws. In this case, the universe to be modeled was generated by: print(‘’.join([f’{xint:0{5}b}’ for xint in range(32)]))
    - arp242 3y ago
      Oh right, you need the leading zeroes won't get printed; need a formatting string with a specific width for that. I don't do much Python so I don't recall the exact syntax off-hand, but the point was: there is an obvious way to compact the number that can be done without any analysis of the number itself (or even looking at it, for that matter). While print(literal) is "cheating" if you ask for "create a program that generates ...", it is a very obvious thing to do if you want to go down that route.
wilg 3y ago
I claim that there are no purported AGIs.
- jabowery 3y ago
  There are plenty of those who purport AGIs threaten us and conflate "existence" with "potential". This is aimed at those driven to hysterics by such.
  - notJim 3y ago
    I think the argument is that current and future AI advancements could lead to AGI. The people I've seen like Yudkowsky who are concerned about AGI don't claim that Chat-GPT is an AGI AFAIK. BTW, I disagree with Yud, but there's no reason to misconstrue his statements.
    - jabowery 3y ago
      Yud is doing more than his share of generating misconstrual of his own statements as evidenced by the laws and regulations being enacted by people who are convinced that AGI is upon is. Ironically, they're right in the sense that the global economy is an unfriendly AGI causing the demographic transition to extinction levels of total fertility rate in exact proportion to the degree it has turned its human components into sterile worker mechanical Turks -- most exemplified by the very people who are misconstruing Yud's statements.
  - deleted 3y ago
    [deleted]
  - nvy 3y ago
    >There are plenty of those who purport AGIs threaten us and conflate "existence" with "potential". This is aimed at those driven to hysterics by such. I'd hazard a guess that the Venn diagrams of "those who purport AGIs threaten us and conflate 'existence' with 'potential'" and of "people who grok binary and can solve esoteric brain teasers using it" have very little overlap. You might have more success with an example that's a little more accessible to "normies".
psyklic 3y ago
What is the answer supposed to be? Doesn't seem like a simple IQ question to me. print(f'{0x110c8531d0952d8:066b}') EDIT: A browser extension hid most of the number from my view, so this answer is incorrect.
- jabowery 3y ago
  It doesn't take much to check the output of that and see it isn't off by a large amount. As for the answer, look at it in groups of 5 bits.
  - psyklic 3y ago
    I don't see how arbitrary questions like this substantially show AGI. If there is a common solution, it could simply look up the solution. Also, AGI could be present just not in this very niche problem (that 99.9% of humans can't solve).
    - jabowery 3y ago
      The point of this "IQ Test" is to set a relatively low-bar for passing the IQ test question so that even intellectually lazy people can get an intuitive feel for the limitation of Transformer models. This limitation has been pointed out formally by the DeepMind paper "Neural Networks and the Chomsky Hierarchy". https://arxiv.org/abs/2207.02098 https://arxiv.org/abs/2207.02098 The general principle may be understood in terms of the approximation of Solomonoff Induction by natural intelligence during the activity known as "data driven science" aka "The Unreasonable Effectiveness of Mathematics In the Natural Sciences". Basically, if your learning model is incapable of at least context sensitive grammars in the Chomsky hierarchy, it isn't capable of inducing dynamical algorithmic models of the world. If it can't do that, then it can't model causality and is therefore going to go astray when it comes to understanding what "is" and therefore can't be relied upon when it comes to alignment of what it "ought" to be doing. PS: You never bothered to say whether the program you provided was from an LLM or from yourself. Why not?
poopbutt7 3y ago
If you clarify the prompt a little (I used Write a python script that can produce this pattern:), GPT-4 gave me this: To generate the given pattern, we need to identify the structure. After analyzing, it seems that the pattern follows a certain structure where the binary representation of each number from 1 to 15 is preceded by a certain number of zeroes. Here's a python script to generate such a pattern: def generate_pattern(): pattern = "" for i in range(1, 16): # Convert i to binary and remove the "0b" prefix. binary = bin(i)[2:] # Number of zeroes to add before each binary. zeroes = 5 - len(binary) # Add binary to pattern with zeroes. pattern += "0" \* zeroes + binary return pattern print(generate_pattern()) The script starts by iterating through numbers from 1 to 15. For each number, it converts the number to a binary string and removes the "0b" prefix that Python adds by default. It calculates the number of zeroes to add before the binary string to maintain a fixed width of 5 characters, and then adds the binary string to the pattern with the preceding zeroes. Finally, it returns the complete pattern. Which outputs: 000010001000011001000010100110001110100001001010100101101100011010111001111 It's missing the first set of zero's, but matches otherwise. So, I guess it's AGI then.
- arp242 3y ago
  The answer is wrong though (not just because it's missing leading zeros, but perhaps you didn't copy the right input?) and it's certainly not the shortest way to output that.
  - poopbutt7 3y ago
    Not sure I follow- the answer matches minus the first leading zeros. Change the range from 0-32, and it matches exactly. So it pretty clearly recognized the pattern and produced working code. This question is a pretty obscure benchmark. Another commenter has it just printing the string, as suggested. If there's some weird math trick to get an optimal implementation, it's probably beyond the grasp of nearly all actual people.
    - arp242 3y ago
vorticalbox 3y ago
Asking gpt3 this and adding "with out printing the string directly" it comes up with this print(''.join(['0' * 10, '1', '0' * 3, '1', '0' * 7, '1', '0' * 3, '1', '0' * 9, '1', '0' * 10, '1', '0' * 13, '1', '0' * 2, '1', '0' * 6, '1', '0' * 5, '1', '0' * 8, '1', '0' * 9, '1', '0' * 11, '1', '0' * 9]))
letmevoteplease 3y ago
If 99% of humans would fail your intelligence test, it is not a good test for the presence of intelligence.
- jabowery 3y ago
  I would venture to guess most college graduates familiar with Python would be able to write a shorter program even if restricted from using hexidecimal representation. Agreed, that may be the 99th percentile of the general population, but this isn't meant to be a Turing test. The Turing test isn't really about intelligence.

vuln 3y ago

chatGPT-4 Result: Sure, you can use the following Python program to output the string you provided: ```python print("0000000001000100001100100001010011000111010000100101010010110110001101011100111110000100011001010011101001010110110101111100011001110101101111100111011111011111") ``` This is the simplest and most direct method to output the string. If you have a more complex task in mind, like generating this string according to a certain pattern, please provide more details.

jabowery 3y ago
The "more complex task in mind" was, of course, to generate the "shortest" program. GPT-4, by asking for a "certain pattern" is attempting to have you do the intellectual heavy lifting for it -- although in this case the intellectual lifting is quite light.
- blowski 3y ago
  I really don't understand your requirements.
deleted 3y ago
[deleted]
arp242 3y ago
This is shorter for starters: print(bin(0x443214c74254b635cf84653a56d7c675be77df)[2:]) May be possible to shave off a few bytes with f'..' strings, or see if there are any repeating patterns, I'm not the sort who enjoys "code golfing", but "use base-16 to represent a base-2 number more compactly" seems fairly obvious to me.
- jabowery 3y ago
  Wrong output. What you call "code golf" is the essence of the natural sciences: Inducing natural laws from the data generated by those natural laws. In this case, the universe to be modeled was generated by: print(‘’.join([f’{xint:0{5}b}’ for xint in range(32)]))
  - arp242 3y ago
    Oh right, you need the leading zeroes won't get printed; need a formatting string with a specific width for that. I don't do much Python so I don't recall the exact syntax off-hand, but the point was: there is an obvious way to compact the number that can be done without any analysis of the number itself (or even looking at it, for that matter). While print(literal) is "cheating" if you ask for "create a program that generates ...", it is a very obvious thing to do if you want to go down that route.

wilg 3y ago

I claim that there are no purported AGIs.

jabowery 3y ago
There are plenty of those who purport AGIs threaten us and conflate "existence" with "potential". This is aimed at those driven to hysterics by such.
- notJim 3y ago
  I think the argument is that current and future AI advancements could lead to AGI. The people I've seen like Yudkowsky who are concerned about AGI don't claim that Chat-GPT is an AGI AFAIK. BTW, I disagree with Yud, but there's no reason to misconstrue his statements.
  - jabowery 3y ago
    Yud is doing more than his share of generating misconstrual of his own statements as evidenced by the laws and regulations being enacted by people who are convinced that AGI is upon is. Ironically, they're right in the sense that the global economy is an unfriendly AGI causing the demographic transition to extinction levels of total fertility rate in exact proportion to the degree it has turned its human components into sterile worker mechanical Turks -- most exemplified by the very people who are misconstruing Yud's statements.
- deleted 3y ago
  [deleted]
- nvy 3y ago
  >There are plenty of those who purport AGIs threaten us and conflate "existence" with "potential". This is aimed at those driven to hysterics by such. I'd hazard a guess that the Venn diagrams of "those who purport AGIs threaten us and conflate 'existence' with 'potential'" and of "people who grok binary and can solve esoteric brain teasers using it" have very little overlap. You might have more success with an example that's a little more accessible to "normies".

psyklic 3y ago

What is the answer supposed to be? Doesn't seem like a simple IQ question to me. print(f'{0x110c8531d0952d8:066b}') EDIT: A browser extension hid most of the number from my view, so this answer is incorrect.

jabowery 3y ago
It doesn't take much to check the output of that and see it isn't off by a large amount. As for the answer, look at it in groups of 5 bits.
- psyklic 3y ago
  I don't see how arbitrary questions like this substantially show AGI. If there is a common solution, it could simply look up the solution. Also, AGI could be present just not in this very niche problem (that 99.9% of humans can't solve).
  - jabowery 3y ago
    The point of this "IQ Test" is to set a relatively low-bar for passing the IQ test question so that even intellectually lazy people can get an intuitive feel for the limitation of Transformer models. This limitation has been pointed out formally by the DeepMind paper "Neural Networks and the Chomsky Hierarchy". https://arxiv.org/abs/2207.02098 https://arxiv.org/abs/2207.02098 The general principle may be understood in terms of the approximation of Solomonoff Induction by natural intelligence during the activity known as "data driven science" aka "The Unreasonable Effectiveness of Mathematics In the Natural Sciences". Basically, if your learning model is incapable of at least context sensitive grammars in the Chomsky hierarchy, it isn't capable of inducing dynamical algorithmic models of the world. If it can't do that, then it can't model causality and is therefore going to go astray when it comes to understanding what "is" and therefore can't be relied upon when it comes to alignment of what it "ought" to be doing. PS: You never bothered to say whether the program you provided was from an LLM or from yourself. Why not?

poopbutt7 3y ago

If you clarify the prompt a little (I used Write a python script that can produce this pattern:), GPT-4 gave me this: To generate the given pattern, we need to identify the structure. After analyzing, it seems that the pattern follows a certain structure where the binary representation of each number from 1 to 15 is preceded by a certain number of zeroes. Here's a python script to generate such a pattern: def generate_pattern(): pattern = "" for i in range(1, 16): # Convert i to binary and remove the "0b" prefix. binary = bin(i)[2:] # Number of zeroes to add before each binary. zeroes = 5 - len(binary) # Add binary to pattern with zeroes. pattern += "0" \* zeroes + binary return pattern print(generate_pattern()) The script starts by iterating through numbers from 1 to 15. For each number, it converts the number to a binary string and removes the "0b" prefix that Python adds by default. It calculates the number of zeroes to add before the binary string to maintain a fixed width of 5 characters, and then adds the binary string to the pattern with the preceding zeroes. Finally, it returns the complete pattern. Which outputs: 000010001000011001000010100110001110100001001010100101101100011010111001111 It's missing the first set of zero's, but matches otherwise. So, I guess it's AGI then.

arp242 3y ago
The answer is wrong though (not just because it's missing leading zeros, but perhaps you didn't copy the right input?) and it's certainly not the shortest way to output that.
- poopbutt7 3y ago
  Not sure I follow- the answer matches minus the first leading zeros. Change the range from 0-32, and it matches exactly. So it pretty clearly recognized the pattern and produced working code. This question is a pretty obscure benchmark. Another commenter has it just printing the string, as suggested. If there's some weird math trick to get an optimal implementation, it's probably beyond the grasp of nearly all actual people.
  - arp242 3y ago

vorticalbox 3y ago

Asking gpt3 this and adding "with out printing the string directly" it comes up with this print(''.join(['0' * 10, '1', '0' * 3, '1', '0' * 7, '1', '0' * 3, '1', '0' * 9, '1', '0' * 10, '1', '0' * 13, '1', '0' * 2, '1', '0' * 6, '1', '0' * 5, '1', '0' * 8, '1', '0' * 9, '1', '0' * 11, '1', '0' * 9]))

letmevoteplease 3y ago

If 99% of humans would fail your intelligence test, it is not a good test for the presence of intelligence.

jabowery 3y ago
I would venture to guess most college graduates familiar with Python would be able to write a shorter program even if restricted from using hexidecimal representation. Agreed, that may be the 99th percentile of the general population, but this isn't meant to be a Turing test. The Turing test isn't really about intelligence.