6 ms·
I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models
by voxleone 1y ago
I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations.
But as impressive as this is, it’s easy to lose sight of the bigger picture: we’ve only scratched the surface of what artificial intelligence could be — because we’ve only scaled two modalities: text and images.
That’s like saying we’ve modeled human intelligence by mastering reading and eyesight, while ignoring touch, taste, smell, motion, memory, emotion, and everything else that makes our cognition rich, embodied, and contextual.
Human intelligence is multimodal. We make sense of the world through:
Touch (the texture of a surface, the feedback of pressure, the warmth of skin0; Smell and taste (deeply tied to memory, danger, pleasure, and even creativity); Proprioception (the sense of where your body is in space — how you move and balance); Emotional and internal states (hunger, pain, comfort, fear, motivation).
None of these are captured by current LLMs or vision transformers. Not even close. And yet, our cognitive lives depend on them.
Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence.
The real frontier of AI lies in the messy, rich, sensory world where people live. We’ll need new hardware (sensors), new data representations (beyond tokens), and new ways to train models that grow understanding from experience, not just patterns.
- Swizec 1y ago> The real frontier of AI lies in the messy, rich, sensory world where people live. We’ll need new hardware (sensors), new data representations (beyond tokens), and new ways to train models that grow understanding from experience, not just patterns. Like Dr. Who said: DALEKs aren't brains in a machine, they are the machine! Same is true for humans. We really are the whole body, we're not just driving it around.
- nomel 1y agoThere are many people who mentally developed while paralyzed that literally drive around their bodies via motorized wheelchair. I don't think there's any evidence that a brain couldn't exist or develop in a jar, given only the inputs modern AI now has (text, video, audio).
- Swizec 1y ago> any evidence that a brain couldn't exist or develop in a jar The brain could. Of course it could. It's just a signals processing machine. But would it be missing anything we consider core to the way humans think? Would it struggle with parts of cognition? For example: experiments were done with cats growing up in environments with vertical lines only. They were then put in a normal room and had a hard time understanding flat surfaces. https://computervisionblog.wordpress.com/2013/06/01/cats-and-vision-is-vision-acquired-or-innate/ https://computervisionblog.wordpress.com/2013/06/01/cats-and...
- deleted 1y ago[deleted]
- nomel 1y agoThis isn't remotely a hypothetical, so I imagine there are some examples out there, especially from back when polio was a problem. Although, for practical reasons, they might have had limited exposure to novelty, which could have negative consequences.
- Swizec 1y agoI agree it’s not hypothetical and also as a layperson I don’t know how much impact on cognition has been studied. Would be cool if it has! I do know of studies that showed blind people start using their visual cortex to process sounds. That is pretty cool imo
- skydhash 1y agoYeah, but are there new ideas or only wishes?
- jdgoesmarching 1y agoIt’s pure magical thinking that would be correctly dismissed if it didn’t have AI attached to it. Imagine talking this way about anything else. “We’ve barely scratched the surface with Rust, so far we’re only focused on code and haven’t even explored building mansions or ending world hunger”
- tim333 1y agoAI has some real possibilities of building mansions and ending hunger in a way that Rust doesn't.
- drw85 1y agoHow is this ending anyones hunger? As long as humans are steering the ship, the commodity will be limited to those that control it and they will make all the money. If anything, it has a big potential to cause more hunger.
- dinfinity 1y ago> Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence. I respectfully disagree. Touch gives pretty cool skills, but language, video and audio are all that are needed for all online interactions. We use touch for typing and pointing, but that is only because we don't have a more efficient and effective interface. Now I'm not saying that all other senses are uninteresting. Integrating touch, extensive proprioception, and olfaction is going to unlock a lot of 'real world' behavior, but your comment was specifically about intelligence. Compare humans to apes and other animals and the thing that sets us apart is definitely not in the 'remaining' senses, but firmly in the realm of audio, video and language.
- voxleone 1y ago> Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence. I probably made a mistake when i asserted that -- should have thought it over. Vision is evolutionarily older and more “primitive”, while language is uniquely human [or maybe, more broadly, primate, cetacean, cephalopod, avian...] symbolic, and abstract — arguably a different order of cognition altogether. But i maintain that each and every sense is important as far as human cognition -- and its replication -- is concerned.
- wizzwizz4 1y agoPeople who lack one of those senses, or even two of them, tend to do just fine.
- oasisaimlessly 1y agoMostly thanks to other humans helping them. If all humans lacked vision, the human race would definitely not do just fine.
- actionfromafar 1y agoI think we need to think about vision and world modelling somewhat separately. We could construct an artificial (tech enhanced) society where sight was not available. People would still "model the world in their minds" with the "abstract model" part of the vision/world system.
- chasd00 1y ago> Language and vision are just the beginning.. Based on the architectures we have they may also be the ending. There’s been a lot of news in the past couple years about LLMs but has there been any breakthroughs making headlines anywhere else in AI?
- dragonwriter 1y ago> There’s been a lot of news in the past couple years about LLMs but has there been any breakthroughs making headlines anywhere else in AI? Yeah, lots of stuff tied to robotics, for instance; this overlaps with vision, but the advances go beyond vision. Audio has seen quite a bit. And I imagine there is stuff happening in niche areas that just aren't as publicly interesting as language, vision/imagery, audio, and robotics.
- nomel 1y agoTwo Nobel prizes in chemistry: https://www.nature.com/articles/s41746-024-01345-9 https://www.nature.com/articles/s41746-024-01345-9
- edanm 1y agoSure. In physics, math, chemistry, biology. To name a few.
- mr_world 1y agoOrganic adaption and persistence of memory I would say are the two major advancements that need to happen. Human neural networks are dynamic, they change and rearrange, grow and sever. An LLM is fixed and relies on context, if you give it the right answer it won't "learn" that is the correct answer unless it is fed back into the system and trained over months. What if it's only the right answer for a limited period of time? To build an intelligent machine, it must be able train itself in real time and remember.
- specialist 1y agoYes and: and forget.
- 8n4vidtmkvmk 1y agoWhy is forgetting important? Things can either have an end time where they are no longer applicable or things we thought were true turn out to be false but it's still useful to see where we went wrong. I imagine humans are limited by the # of synapses we have so it's useful to forget but maybe machines can move the useless stuff to deep storage until it's dug out, in the same way certain things can trigger a deep memory in humans.
- specialist 1y agoDo you remove dead code? Get rid of clutter? Ever try to change a habit?
- 8n4vidtmkvmk 1y agoYeah, I delete dead code but it's important to remember why I wrote it that way in the first place and why I'm deleting it now. Doomed to repeat past mistakes and all that.
- lupire 1y agoWhich is cheaper, among these two logically equivalent things: reducing one weight, or increasing every other weight?
- deleted 1y ago[deleted]
- timewizard 1y ago> has made jaw-dropping progress They took 1970s dead tech and deployed it on machines 1 million times more powerful. I'm not sure I'd qualify this as progress. I'd also need an explanation as to what systemic improvements in models and computations that give an exponential growth in performance are planned. I don't see anything.
- ekunazanu 1y agoWinning two Nobel prizes wasn't enough progress?
- timewizard 1y agoIs progress measured in nobel prizes? My understanding is those are put to a vote by institutional committee. Putting that aside. The shared prize in 2024 was given for work done in the 1970s and 1980s. Was this meant to be a confirmation of my point? You've done so beautifully. In 2022 they saw fit to award Ben Bernanke. Yep. That one. For, I kid you not, work on the impacts of financial crises. Ironically also work originally done in the 1970s and 80s.
- ekunazanu 1y agoAlphaFold uses transformers. That is definitely not from the 70s and 80s. Progress for me includes both small iterative refinements and big leaps. It also includes trying old techniques in new domains with new technology. So I think we just have differing definitions for progress.
- petesergeant 1y ago> They took 1970s dead tech and deployed it on machines 1 million times more powerful. I’m not sure I’d qualify this as progress If this isn’t meant to be sarcasm or irony, you’ve got some really exciting research and learning ahead of you! At the moment it reads very “computers are just addition and multiplication and we’ve had that for thousands of years!”
- 1y ago
- slashdave 1y ago> modeled human intelligence That's not what these models do
- TimByte 1y agoIn a way until AI systems can feel the weight of a cup or flinch from heat, we're not close to modeling anything like embodied cognition
- tsimionescu 1y agoI think one counterpoint to this idea is the compute cost. To a great extent, it's not AI research that is the primary driver behind the huge advances in AI, either in terms of techniques (transformers) or data sets. Instead, the biggest single factor responsible for this huge boost are advances in compute hardware and compute power in general. Even if we had known about the Transformer architecture 20 years earlier, and we had had the datasets that OpenAI and Google amassed 20 years earlier, we still would not have been able to get anywhere close to training an LLM on hardware from 20 years ago. And given this, and given that LLMs have already pushed this compute power to the limit, it's very possible that we'll stagnate at more or less the current level unless and until a new 10x or even 100x boost in compute power happens. It's very unlikely that you could train a model on 100x as much data as you get today without that, which is what you would likely require to add multiple modalities and then combine them.
- lupire 1y agoTick-tock model. Research switches between adding new ideas and pouring more power into good ideas.
- antithesizer 1y agoThe big horizon isn't just incorporating another sensory modality, it's what Heidegger called being-in-the-world, living among us as a human-like social being. That advancement depends on robotics to provide emboddied experience.