9 ms·
The technological singularity is approaching fast. It's pretty clear that, given terminal access and an appropriate outer loop, GPT models can iteratively crea
by c7b 3y ago
The technological singularity is approaching fast.
It's pretty clear that, given terminal access and an appropriate outer loop, GPT models can iteratively create new GPT models (either by writing and executing Pyhton code, or later versions trained on LLM weights may even be able to output new weights directly). If the inner workings of the loop are sufficiently obfuscated (in the code-based version), it wouldn't necessarily be clear to us what had changed, and the model weights/architecture on their own are not interpretable. That's very close to a singularity definition (machines that self-improve faster than what our understanding can keep up with), the only open question is whether the new versions would actually be improvements. But that sounds solvable too (except that it wouldn't be easy for us to tell for sure).
- gwoolhurme 3y agoCould you explain in detail how that causes the singularity? I’m lost. I saw a cool tool used to make another cool tool. You saw the singularity. Where is this logic coming from? Why would terminal access which I’m fairly certain I’ve seen in autogpt, change much.
- jimbokun 3y agoThe cool tool makes another cool tool, which in turn makes another cool tool, faster and faster, until we really don't understand at all what the latest cool tool is doing. But it just keeps getting smarter/faster/more effective/whatever it's optimizing for. That's the basic definition of "the singularity".
- gwoolhurme 3y agoI don’t see that in this specific case for one thing or how we are closer from before. Code generation and auto correction getting closer is very cool, the singularity needs to be spelled out for me how a tool making a tool will get us there. How does that solve all of the hurdles that go into AGI or the singularity. How does it change a large language model.
- monkpit 3y agoWould there be some limit that it approaches over a large number of iterations though?
- generic92034 3y agoWhat never was so clear to me in that vision is, how the version n actually makes sure the version n+1 is actually faster and better. Initially there might be the easy option of using more tokens/memory/<whatever easily measurable and beneficial factor you can imagine>. But when that becomes impractical, how will the "dumber AI" select between x generated "smarter AIs"? How will it ensure that the newly generated versions are better at all (if there are no easily measurable parameters to be increased)?
- travisjungroth 3y agoYou have it optimize towards reasoning tests, not tokens or memory. It’s like if you were optimizing a race car. You make the explicit goal time around the track, not horsepower or weight.
- generic92034 3y agoThat could end up in getting better and better special-purpose expert systems. How do you create better and better general-purpose AIs this way? What is more, when the AIs are more advanced, it might be challenging to create meaningful tests for them (outside of very specialized domains).
- travisjungroth 3y agoMy point with the race analogy was you score it on the outcome (time) not a factor (horsepower). For your concern, just make the outcome broader. Someone will still say it’s not AGI as it does 100 different remote jobs, but just plan on ignoring them. Something that scares me is people don’t want to believe this is where we’re headed or it’s even possible. I say this because I think your concerns are easy enough to address, it makes me think you didn’t try answering your own questions. The idea in this scenario is it’s self optimizing. No reason it can’t make it’s own more specific tests to the general test of “become as smart as possible”. And people can make tests that they’re not smart enough to pass. You just make them as a team and take more time. It could also discover new things and then verify if they’re true, which is easier.
- c7b 3y agoThe cycle could look something like this: LLMv1 > prompt "get me the code for a better LLM" > paste output to terminal, execute > LLMv2 > repeat. There are lots of different singularity definitions on wikipedia, most focus on 'intelligence' (eg "an upgradable intelligent agent will eventually enter a 'runaway reaction' of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly"). I think focusing on what 'intelligence' really means, or whether any model is 'truly' intelligent can be be a bit of a distraction. So I just emphasized capabilities in a rather generic sense instead. It's clear that LLMs can compete with humans on many tasks (coding, creative writing, medical diagnosis, psychotherapy,...), and it is conceivable that they may surpass most humans on those tasks some day. If we have models that can outperform most or all humans on important professional and everyday tasks, and also produce new models that perform even better, maybe at an accelerating rate, through means that are ultimately not interpretable for us, I'd say that's pretty close to a singularity, regardless of whether they're 'truly' intelligent. Even more so if they pass a duck test (walks like a duck, quacks like a duck,...).
- ZephyrBlu 3y agoYou can already do this in theory with tools like AutoGPT and BabyAGI. Why hasn't the singularity happened already?
- gwoolhurme 3y agoExactly AutoGPT is exactly what he is describing but I’m still lost as to what the verbiage is then. Is that the singularity?
- KaoruAoiShiho 3y agoYes AutoGPT would be the singularity if it had more brain-like subsystems like long and short term memory and is hooked up to real world instruments.
- ZephyrBlu 3y ago
- choeger 3y agoCalm down. It is a language model. People have figured out how to predict the next word for a given prefix. That's very cool and it will definitely have a significant impact on software. But it is not artificial intelligence.
- mlboss 3y agoWhat does it has to do that will qualify it as artificial intelligence ? In my opinion, all the ingredients are there for artificial intelligence. Somebody just need to stitch everything up. It can understand text, reason about it, identity next steps, can write code to execute the steps, understand error messages and fix the code. That feels like AI.
- bonzini 3y agoBe able to say "I don't know".
- JKCalhoun 3y ago(You've just excluded a few people I know.)
- deleted 3y ago[deleted]
- moffkalast 3y agoGPT 4 tells me that on a daily basis because of the 2021 data cutoff.
- bonzini 3y agoSo it never hallucinates APIs, theorems, biographies or places?
- moffkalast 3y ago
- avidphantasm 3y agoUnless this singularity is able to construct for itself sensors and actuators that can refine existing knowledge and acquire new knowledge, it will forever be hamstrung by the incomplete and static knowledge of the universe that was used to train the initial model. This is where regulation is desperately needed. We must be very careful with the kinds of sensors and actuators that are integrated with these systems. Perhaps none should be allowed.
- c7b 3y agoWell, if the terminal has internet access, it can download whatever new data has been created and train a new model on that. In theory, it could also hack into a 3D printing workshop, or get some funds (eg steal credit card data, or perhaps even legally) and just issue regular orders. This is clearly very speculative thinking, but I think we are making a similar point actually.
- cwmoore 3y agoThere is no singularity if we get there first.
- hoangbao0201 3y ago[flagged]
- comfypotato 3y agoYou’re clear in your use of a singularity definition. I’ve always taken the (I think more popular?) stance that the singularity is better defined by AGI surpassing average or best human intelligence. This definition is still far off. The lacking aspects of end-to-end functionality in LLMs may or may not ever see completion. There are paradigms shifts necessary for many applications.
- robbintt 3y agoIt seems like there is some sort of "viral singularity" that is quite likely. Still a "paperclip maximizer" where paperclip is something unexpected. In a sense life, and humanity, are types of paperclip maximizers.
- bionhoward 3y agoThe interpretability of LLMs comes in chain of thought. When a human explains a process, they talk through it. You don’t measure their neurons. The interpretability of neuron weights is a sub-symbolic red herring. Causal emergence says, the agent is what matters, not the individual synapse. Think like a manager with AI, not a cellular neurobiologist