4 ms·

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human: design a Turing complete programming la

by norir 2y ago

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human: design a Turing complete programming language that is both human and machine readable and implement a self hosted compiler in this language that self compiles on existing hardware faster than any known language implementation that also self compiles. Additionally, for any syntactically or semantically invalid program, the compiler must provide an error message that points exactly to the source location of the first error that occurs in the program. I will get excited for/scared of LLMs when they can tackle this kind of problem. But I don't believe they can because of the fundamental nature of their design, which is both backward looking (thus not better than the human state of the art) and lacks human intuition and self awareness. Or perhaps rather I believe that the prompt that would be required to get an LLM to produce such a program is a problem of at least equivalent complexity to implementing the program without an LLM.

Xenoamorphous 2y ago
> Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human That’s possible for a highly intelligent, extensively trained, very small subset of humans.
- hatefulmoron 2y ago
  If you took the intersection of every human's abilities you'd be left with a very unimpressive set. That also ignores the fact that the small set of humans capable of building programming languages and compilers is a consequence of specialization and lack of interest. There are plenty of humans that are capable of learning how to do it. LLMs, on the other hand, are both specialized for the task and aren't lazy or uninterested.
- luckydata 2y ago
  does it mean people that can build languages and compilers are not humans? What is the point you're trying to make?
  - fragmede 2y ago
    It means that's a really high bar for intelligence, human or otherwise. If AGI is "as good as a human, and the test is a trick task that most humans would fail at (especially considering the weasel requirement that it additionally has to be faster), why is that considered a reasonable bar for human-grade intelligence.
jedberg 2y ago
I will get excited when an LLM (or whatever technology is next) can solve tasks that 80%+ of adult humans can solve. Heck let's even say 80% of college graduates to make it harder. Things like drive a car, fold laundry, run an errand, do some basic math. You'll notice that two of those require some form of robot or mobility. I think that is key -- you can't have AGI without the ability to interact with the world in a way similar to most humans.
- ata_aman 2y ago
  So embodied cognition right?
bob1029 2y ago
This sounds like something more up the alley of linear genetic programming. There are some very interesting experiments out there that utilize UTMs (BrainFuck, Forth, et. al.) [0,1,2]. I've personally had some mild success getting these UTM variants to output their own children in a meta programming arrangement. The base program only has access to the valid instruction set of ~12 instructions per byte, while the task program has access to the full range of instructions and data per byte (256). By only training the base program, we reduce the search space by a very substantial factor. I think this would be similar to the idea of a self-hosted compiler, etc. I don't think there would be too much of a stretch to give it access to x86 instructions and a full VM once a certain amount of bootstrapping has been achieved. [0]: https://arxiv.org/abs/2406.19108 https://arxiv.org/abs/2406.19108 [1]: https://github.com/kurtjd/brainfuck-evolved https://github.com/kurtjd/brainfuck-evolved [2]: https://news.ycombinator.com/item?id=36120286 https://news.ycombinator.com/item?id=36120286
Vampiero 2y ago
Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for an average human: designing a functional trivia app. There, you don't need to invoke Turing or compiler bootstrapping. You just need one example of a use case where the accuracy of responses is mission critical
- alainx277 2y ago
  o1-preview managed to complete this in one attempt: https://chatgpt.com/share/67373737-04a8-800d-bc57-de74a415e2c9 https://chatgpt.com/share/67373737-04a8-800d-bc57-de74a415e2... I think the parent comment's challenge is more appropriate.
  - Vampiero 2y ago
    Have you personally verified that the answers are not hallucinations and that they are indeed factually true? Oh, you just asked it to make a trivia app that feeds on JSON. Cute, but that's not what I meant. The web is full of tutorials for basic stuff like that. To be clear I meant that LLMs can't write trivia questions and answers, thus proving that they can't produce trustworthy outputs. And a trivia app is a toy (one might even say... a trivial example), but it's a useful demonstration of why you wouldn't put an LLM into a system on which lives depend on, let alone invest billions on it. If you don't trust my words just go back to fiddling with your models and ask them to write a trivia quiz about a topic that you know very well. Like a TV show.

by norir 2y ago

Xenoamorphous 2y ago

> Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human That’s possible for a highly intelligent, extensively trained, very small subset of humans.

hatefulmoron 2y ago
If you took the intersection of every human's abilities you'd be left with a very unimpressive set. That also ignores the fact that the small set of humans capable of building programming languages and compilers is a consequence of specialization and lack of interest. There are plenty of humans that are capable of learning how to do it. LLMs, on the other hand, are both specialized for the task and aren't lazy or uninterested.
luckydata 2y ago
does it mean people that can build languages and compilers are not humans? What is the point you're trying to make?
- fragmede 2y ago
  It means that's a really high bar for intelligence, human or otherwise. If AGI is "as good as a human, and the test is a trick task that most humans would fail at (especially considering the weasel requirement that it additionally has to be faster), why is that considered a reasonable bar for human-grade intelligence.

jedberg 2y ago

I will get excited when an LLM (or whatever technology is next) can solve tasks that 80%+ of adult humans can solve. Heck let's even say 80% of college graduates to make it harder. Things like drive a car, fold laundry, run an errand, do some basic math. You'll notice that two of those require some form of robot or mobility. I think that is key -- you can't have AGI without the ability to interact with the world in a way similar to most humans.

ata_aman 2y ago
So embodied cognition right?

bob1029 2y ago

This sounds like something more up the alley of linear genetic programming. There are some very interesting experiments out there that utilize UTMs (BrainFuck, Forth, et. al.) [0,1,2]. I've personally had some mild success getting these UTM variants to output their own children in a meta programming arrangement. The base program only has access to the valid instruction set of ~12 instructions per byte, while the task program has access to the full range of instructions and data per byte (256). By only training the base program, we reduce the search space by a very substantial factor. I think this would be similar to the idea of a self-hosted compiler, etc. I don't think there would be too much of a stretch to give it access to x86 instructions and a full VM once a certain amount of bootstrapping has been achieved. [0]: https://arxiv.org/abs/2406.19108 https://arxiv.org/abs/2406.19108 [1]: https://github.com/kurtjd/brainfuck-evolved https://github.com/kurtjd/brainfuck-evolved [2]: https://news.ycombinator.com/item?id=36120286 https://news.ycombinator.com/item?id=36120286

Vampiero 2y ago

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for an average human: designing a functional trivia app. There, you don't need to invoke Turing or compiler bootstrapping. You just need one example of a use case where the accuracy of responses is mission critical

alainx277 2y ago
o1-preview managed to complete this in one attempt: https://chatgpt.com/share/67373737-04a8-800d-bc57-de74a415e2c9 https://chatgpt.com/share/67373737-04a8-800d-bc57-de74a415e2... I think the parent comment's challenge is more appropriate.
- Vampiero 2y ago
  Have you personally verified that the answers are not hallucinations and that they are indeed factually true? Oh, you just asked it to make a trivia app that feeds on JSON. Cute, but that's not what I meant. The web is full of tutorials for basic stuff like that. To be clear I meant that LLMs can't write trivia questions and answers, thus proving that they can't produce trustworthy outputs. And a trivia app is a toy (one might even say... a trivial example), but it's a useful demonstration of why you wouldn't put an LLM into a system on which lives depend on, let alone invest billions on it. If you don't trust my words just go back to fiddling with your models and ask them to write a trivia quiz about a topic that you know very well. Like a TV show.