7 ms·
Toward AGI – What Is Missing?
- nh23423fefe 3y agoAGI wouldn't forget bullet point 3. how could an "intelligent" human make such an obvious mistake. /s
- jasfi 3y agoRight now I'd say what's missing is sufficient time to build complex systems with LLMs, that will be AGIs with various features and components. We'll need to benchmark them as is currently done with LLMs. It's just a matter of time.
- robot 3y agowhat is missing in 2 words: "problem solving"
- ImHereToVote 3y agoLet's focus on alignment before focusing on AGI. Let's not be goobers and extinct ourselves.
- goatlover 3y agoNah, we need another Cold War experiment to see if we can make it out a second time, not they're we're technically out of a nuclear war scenario
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- ilaksh 3y agoI don't think "AGI" is used in a consistent way. It seems like people want it to mean not just general utility but a full simulation of a person (which then sometimes automatically becomes godlike). But they don't say that. Anyway, decent article. Reminds me of what LeCun has been talking about. https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/ https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
- robotresearcher 3y agoThe article says we need: > Online non-greedy planning technologies that can operate in the presence of a priori unseen tasks, states, and actions. > A world model technology that robustly predicts state-action-state transitions for all states and actions that might be encountered while performing any task in any environment. People use greedy strategies all the time, and the intractability of considering 'all states and actions' is the Frame Problem. It is lack of generality that avoids the Frame Problem, and makes planning tractable for animals. In my view, AGI is an anti-goal. Specialization is what makes intelligent systems feasible. On the other hand, there are technologies like DNNs, gradient descent strategies, even computation itself, that are generally useful for application to specialized intelligent systems. By analogy, all animals are made of cells, and developed by evolution. But there's no General Animal that can do what all the others can, even if they share core technology. edit: I wish I hadn't given the analogy. Quibbling about it seems to be irresistible, and has has distracted from my point here.
- goatlover 3y ago> But there's no General Animal that can do what all the others can, even if they share core technology. Humans can artificially travel fast, fly, swim deep, drill deep, move heavy things, kill very efficiently, etc.
- TheRoque 3y agoIt's still not general. Besides, it relies on a heavy use of energy that might not be around forever.
- goatlover 3y agoThen your definition of general is too general to be meaningful.
- robotresearcher 3y agoUnless that 'etc' is working very hard, I don't think you are seriously claiming that humans and their tech can do everything another animal can do. And if you were, we would only have to consider energy efficiency to save the analogy.
- rafaelero 3y agoWhy are you using GPT3.5? Red flag.
- vouaobrasil 3y ago> There are benefits and downsides to AGI. On the upside, AGIs can do most of the labor that consume a vast amount of humanity’s time and energy. AGI can herald a utopia where no one has wants that cannot be fulfilled. This is not a utopia, but a juvenile and unthinking definition of utopia, or a dystopia in disguise. Most people thrive on some sort of challenge to at least do basic tasks for themselves. A world where AGI fulfils every need is more like being in a senior's home where everyone does everything for you. It sounds good at first but is actually a nightmare where your autonomy is stripped by the lack of goals to work towards. Immense surplus and easy access to everything all the time means human beings will no longer need each other much either, so it means a world full of people uncaring towards each other. Fulfilling any want without hard work to get it means becoming effectively like a bacteria, consuming every resource in sight, transforming us into even more of a plague than we already are. I certainly would not want to live in such a world.
- dingnuts 3y agoDoesn't it bother you to describe humanity as a plague? The overall point is fine but these are people you're talking about, and NOT bacteria. It's just so anti-human to think of people that way. Souls. But a plague on the Earth, I guess. It's just a depressing viewpoint imo
- vouaobrasil 3y agoWell, if you spend a lot of time looking at the damage we cause every day, studying reports and seeing innocent creatures like sharks being killed by the hundred millions, thousands of species being rendered extinct, endless garbage being dumped left right and center, habitats being destroyed endlessly by the millions of square kilometers, yeah, we do look like a plague. And now we're trying to use AI to become even more resource-intensive, burning up endless energy to be even more powerful consumers...give me a break. It's sick. As individuals there's nothing seriously wrong with most people but en masse with modern technology, that is exactly what we are, a plague.
- robotresearcher 3y ago
- canvascritic 3y agoback when we were scaling a saas offering in the late 2000s, we were told by some sales 'engineer' at a certain company well known in those days with lots of hype marketing (that died explosively a year later, big shock), that ML would solve all our engagement problems. It was hyped up, just like AGI is now. the reality? it has made a difference, no doubt, mainly in the form of incremental improvements at best. this feels eerily similar. I've sat in countless pitches and meetings with bright-eyed founders touting the imminent rise of AGI. There's a palpable excitement around the topic, but from a purely business perspective, there's an equally strong air of naiveté. I often wonder if proponents of AGI have truly considered its commercial implications. let's indulge the idea for a moment and say that it becomes a reality. how do you price it? how do you convince industries to adopt and trust something that mirrors human cognition, with all its opacity, when we still grapple with the biases and imperfections of simpler AI models? add to this the astronomical R&D costs. the sheer capex required is a massive barrier. even with vc backing, the path to profitability seems tenuous at best. and in our current startup landscape and inflation reality, such ventures are money holes. to me, AGI feels like childish pursuit of the shadows of sci-fi dreams. there's merit in pushing the boundaries of technology, but mistaking theoretical marvels for immediate commercial viability is a classic pitfall. for now, I'll reserve my enthusiasm until someone can convincingly bridge the chasm between agi's promise and its pragmatic business value.
- DennisP 3y ago> Because non-greedy online planning can be slow, offline planning can be used to generate a policy that maps any state to the best action to take. The resulting policy can be run very fast at execution time; no matter what state the agent ends up in after executing an action, the next action to execute is available as quickly as π can be queried. The article claims this is insufficient for AGI, but to me it sounds a lot like Kahneman's distinction between fast and slow thinking. Slow thinking is the offline planning, the trained network does the fast thinking. If humans work that way, then a human-level AGI probably can too.
- Animats 3y agoThe article has a flavor of trying to find a role for good old fashioned AI, with planning and search, in this new world. It points out that to some extent large language models can plan, but only for things where the steps required are roughly described on the Internet. It's basically splicing together bits of known sequences of steps. That can handle many cases. As usual, the problem is lack of evaluation - totally bogus plans can emerge unnoticed. This is the lack of a model problem. The big lack remains manipulation in unstructured situations. Although that Google project mentioned a few days ago has some promise. They train on simple manipulation tasks with a force feedback teleoperator, and then use an LLM for the higher level task sequencing. But the simple manipulation tasks remain rather simple. Too simple to, say, assemble flat-pack furniture, let alone repair a car. LLMs just don't seem to be relevant to such non-language tasks.
- ranie93 3y agoIf a time traveler from a future where AGI was discovered brought back the information (program?, algorithm(s)?) necessary to create AGI: 1. How much space would it take up? GBs? 2. Would we have the computing power to run "it", say on a top of the line consumer laptop?
- bob1029 3y agoI say the biggest missing link remains a complete lack of modeling whatsoever for some time domain. Representation of time delay between neurons and other elements can encode a lot of information. As it currently stands, everything in popular AI architectures is evaluated in a completely timeless manner. I believe very deeply that spiking neural networks are the path to AGI. I also believe that the amount of compute ultimately required to realize practical SNNs is going to turn out to be shockingly small.
- l33t7332273 3y agoI thought there were NN architectures where some layers’ outputs are sent to layers not directly in front of them. Wouldn’t that essentially capture some simulation of a time delay?
- bob1029 3y agoIs there any inherent delay modeled between any of the layers in your hypothetical NN architecture (assuming non-SNN)? If there are no actual time delays modeled within the network, then whatever it evaluates is representative of the outside state at some instant in time.
- cglan 3y agoI think the problem is that SNNs aren't suited to standard von neuman architecture. I'm in total agreement. That amount of power and compute thrown at language models, and ML in general is shocking relative to the capabilities. For GPT4, it's staggering. Probably on the order of KWHs in power alone for basic tasks. I think a properly modeled architecture would look something like the actor model represented in hardware, where each actor is basically a small extremely power efficient cpu but unfortunately you wouldn't be able to pull that off the shelf.
- bob1029 3y ago> I think the problem is that SNNs aren't suited to standard von neuman architecture. This is where we begin to disagree. If you are content with moving from actual real time to synthetic time, all of the potential frustrations fall away again. Think about online vs offline rendering in computer graphics. Theoretically, you could render all of Toy Story on a RasPi4. Most of us would be willing to wait days for answers, assuming the batch size is acceptable and the answers are correct.
- korse 3y agoPhilosophy of mind. It is impossible to create an artificial version of something that we cannot yet define. People need to dial back the hype and take some more time to consider why we think and what constitutes a 'sense of self'. Spending some more time figuring out how biological brains 'store data' wouldn't hurt either, but impressive MRI party tricks (with monetization paths) seem to win the day every time. Side note is that language models are extremely inefficient for the benefit they provide. Most humans don't need to ingest terabytes of domain specific data in order to engage in basic reading comprehension and reasoning. I'd argue they excel at things like designing molecules, where we don't have the mental capacity or speed to ingest as much information as we would need to reason quickly, but the amount of information and processing necessary to replace a secretary is laughable.
- robotresearcher 3y ago> Most humans don't need to ingest terabytes of domain specific data in order to engage in basic reading comprehension and reasoning. How much data do you think a 6 year old has processed by the time they read a book? Eyes and ears are pretty high bitrate input devices.
- russdill 3y agoNot only that, the built-in connection network that developed over hundreds of millions of years from untold trillions of organisms.
- bluetomcat 3y agoA child doesn’t process “data” in the form of a sequence of tokens. A child associates visual objects with similar traits to pronounced words, heard from their parents. It also acquires a basic feeling of temporal sequentiality and physical causality by throwing toys, for example. It learns the rules of the language grammar by subconsciously noticing the basic structure of sentences with objects, verbs and qualities.
- xboxnolifes 3y ago
- Vicinity9635 3y ago[dead]
- antisthenes 3y agoAny attempt to arrive at AGI without acknowledging that the system must have multiple precise sensory inputs is doomed to fail. You cannot build an accurate world model without sensory input and the ability to gather data-driven descriptors of the physical world.