7 ms·

I’m not sure how to word my excitement about the progress we see in AI research in the last years. If you haven’t read it, give Tim Urbans classic piece a slice

I’m not sure how to word my excitement about the progress we see in AI research in the last years. If you haven’t read it, give Tim Urbans classic piece a slice of your attention: https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol... It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.

f38zf5vdt 4y ago
That human intelligence might just be token prediction evolving from successive small bit-width float matrix transformations is depressing to me.
- goatlover 4y ago
  Is that what biologists or neuroscientists think the nervous system is actually doing?
- triceratops 4y ago
  > That human intelligence might just be token prediction I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)
  - londons_explore 4y ago
    Eating salad is good for your health. Not only word salad, but green salad and egg salad.
  - mdp2021 4y ago
    This trains in seeing what intelligence is not, not the opposite!
- xixixao 4y ago
  It’s most fascinating (or very obvious) - look at Conway’s Game of Life, then scale it up - a lot. Unlimited complexity can arise from very simple rules and initial conditions. Now consciousness on the other hand is unfathomable and (in its finitude) extremely depressing for me.
- Der_Einzige 4y ago
  Dear god I hope that we are using something more complicated than sampling with top_p, top_k, and a set temperature as our decoder!
- chriswarbo 4y ago
  That's a poor usage of "just": discovering that "X is just Y" doesn't diminish X; it tells us that Y is a much more complex and amazing topic than we might have previously thought. For example: "Life is just chemistry", "Earth is just a pile of atoms", "Behaviours are just Turing Machines", etc.
- 0xBABAD00C 4y ago
  Wait till you find out all of physics is just linear operators & complex numbers
  - goatlover 4y ago
    Unless nature is mathematical, the linear operators & complex numbers are just useful tools for making predictive models about nature. The map isn't the territory.
- mdp2021 4y ago
  Stop being depressed because it simply, clearly, certainly, is not. I just wrote a few paragraphs about it in an immediately previous post. This confirms that this phase is getting some people fooled on basics.
gurkendoktor 4y ago
Not you specifically, but I honestly don't understand how positive many in this community (or really anyone at all) can be about these news. Tim Urban's article explicitly touches on the risk of human extinction, not to mention all the smaller-scale risks from weaponized AI. Have we made any progress on preventing this? Or is HN mostly happy with deprecating humanity because our replacement has more teraflops? Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.
- tim333 4y ago
  >happy with deprecating humanity because our replacement has more teraflops? For me immortality a bigger thing than the teraflops. Also I don't think regular humanity would be got rid of but continue in parallel.
- JohnPrine 4y ago
  Agreed. People think of the best case scenario without seriously considering everything that can go wrong. If we stay on this path the most likely outcome is human extinction. Full stop
  - JoeAltmaier 4y ago
    Says a random internet post. It takes a little more evidence or argument to be convincing, besides hyperbole.
  - alm1 4y ago
    Mechanized factories failed to kill humanity two hundreds ago and the Luddite movement against them seems comical today. What makes you think extinction is most likely?
  - stefs 4y ago
    this path will indeed lead to human extinction, but the path is climate change. AI is one of the biggest last hopes for reversing it. from my perspective, if it does kill us all, well, it's most likely still a less painful death.
edouard-harris 4y ago
That Tim Urban piece is great. It's also an interesting time capsule in terms of which AI problems were and were not considered hard in 2015 (when the post was written). From the post: > Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it. The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem. Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as examples, so laughably out of reach they seemed in the heathen dark ages of... 2015. And as for recognizing a cat or a dog — that's a problem so trivial today that it isn't even worth using as the very first example in an introductory AI course. [9] If someone re-wrote this post today, I wonder what sorts of things would go into the "hard for a computer" bucket? And how many of those would be left standing in 2029? [1] https://arxiv.org/abs/2204.14198 https://arxiv.org/abs/2204.14198 [2] https://arxiv.org/abs/2004.10934 https://arxiv.org/abs/2004.10934 [3] https://www.youtube.com/watch?v=tF4DML7FIWk https://www.youtube.com/watch?v=tF4DML7FIWk [4] https://openai.com/blog/solving-rubiks-cube/ https://openai.com/blog/solving-rubiks-cube/ [5] https://say-can.github.io/ https://say-can.github.io/ [6] https://www.deepmind.com/open-source/perceiver-io https://www.deepmind.com/open-source/perceiver-io [7] https://arxiv.org/abs/2201.08239v2 https://arxiv.org/abs/2201.08239v2 [8] https://openai.com/dall-e-2/ https://openai.com/dall-e-2/ [9] https://www.fast.ai/ https://www.fast.ai/
- mdp2021 4y ago
  > And as for recognizing a cat or a dog — that's a problem so trivial today Last time I checked - though it's been a long while I could not check thoroughly owing to other commitments - "«recognizing»" there was "consistently successfully guessing", not "critically defining". It may be that the problem was solved in the latest years, I cannot exclude it - but I have not seen around in the brief "news checking" exercise the signals required for the solution. The real deal is far from trivial. A clock can tell the time but does not know it.
sinenomine 4y ago
Excitement alone won't help us. We should ask our compute overlords to perform their experiments in as open environment as possible, just because we, the public, should have the power to oversee the exact direction this AI revolution is taking us. If you think about it, AI safetyism is a red herring compared to a very real scenario of powerful AGIs working safely as intended, just not in our common interest. The safety of AGI owners' mindset seems like a more pressing concern compared to a hypothetical unsafety of a pile of tensors knit together via gradient descent over internet pictures.

f38zf5vdt 4y ago

That human intelligence might just be token prediction evolving from successive small bit-width float matrix transformations is depressing to me.

goatlover 4y ago
Is that what biologists or neuroscientists think the nervous system is actually doing?
triceratops 4y ago
> That human intelligence might just be token prediction I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)
- londons_explore 4y ago
  Eating salad is good for your health. Not only word salad, but green salad and egg salad.
- mdp2021 4y ago
  This trains in seeing what intelligence is not, not the opposite!
xixixao 4y ago
It’s most fascinating (or very obvious) - look at Conway’s Game of Life, then scale it up - a lot. Unlimited complexity can arise from very simple rules and initial conditions. Now consciousness on the other hand is unfathomable and (in its finitude) extremely depressing for me.
Der_Einzige 4y ago
Dear god I hope that we are using something more complicated than sampling with top_p, top_k, and a set temperature as our decoder!
chriswarbo 4y ago
That's a poor usage of "just": discovering that "X is just Y" doesn't diminish X; it tells us that Y is a much more complex and amazing topic than we might have previously thought. For example: "Life is just chemistry", "Earth is just a pile of atoms", "Behaviours are just Turing Machines", etc.
0xBABAD00C 4y ago
Wait till you find out all of physics is just linear operators & complex numbers
- goatlover 4y ago
  Unless nature is mathematical, the linear operators & complex numbers are just useful tools for making predictive models about nature. The map isn't the territory.
mdp2021 4y ago
Stop being depressed because it simply, clearly, certainly, is not. I just wrote a few paragraphs about it in an immediately previous post. This confirms that this phase is getting some people fooled on basics.

gurkendoktor 4y ago

Not you specifically, but I honestly don't understand how positive many in this community (or really anyone at all) can be about these news. Tim Urban's article explicitly touches on the risk of human extinction, not to mention all the smaller-scale risks from weaponized AI. Have we made any progress on preventing this? Or is HN mostly happy with deprecating humanity because our replacement has more teraflops? Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.

tim333 4y ago
>happy with deprecating humanity because our replacement has more teraflops? For me immortality a bigger thing than the teraflops. Also I don't think regular humanity would be got rid of but continue in parallel.
JohnPrine 4y ago
Agreed. People think of the best case scenario without seriously considering everything that can go wrong. If we stay on this path the most likely outcome is human extinction. Full stop
- JoeAltmaier 4y ago
  Says a random internet post. It takes a little more evidence or argument to be convincing, besides hyperbole.
- alm1 4y ago
  Mechanized factories failed to kill humanity two hundreds ago and the Luddite movement against them seems comical today. What makes you think extinction is most likely?
- stefs 4y ago
  this path will indeed lead to human extinction, but the path is climate change. AI is one of the biggest last hopes for reversing it. from my perspective, if it does kill us all, well, it's most likely still a less painful death.

edouard-harris 4y ago

That Tim Urban piece is great. It's also an interesting time capsule in terms of which AI problems were and were not considered hard in 2015 (when the post was written). From the post: > Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it. The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem. Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as examples, so laughably out of reach they seemed in the heathen dark ages of... 2015. And as for recognizing a cat or a dog — that's a problem so trivial today that it isn't even worth using as the very first example in an introductory AI course. [9] If someone re-wrote this post today, I wonder what sorts of things would go into the "hard for a computer" bucket? And how many of those would be left standing in 2029? [1] https://arxiv.org/abs/2204.14198 https://arxiv.org/abs/2204.14198 [2] https://arxiv.org/abs/2004.10934 https://arxiv.org/abs/2004.10934 [3] https://www.youtube.com/watch?v=tF4DML7FIWk https://www.youtube.com/watch?v=tF4DML7FIWk [4] https://openai.com/blog/solving-rubiks-cube/ https://openai.com/blog/solving-rubiks-cube/ [5] https://say-can.github.io/ https://say-can.github.io/ [6] https://www.deepmind.com/open-source/perceiver-io https://www.deepmind.com/open-source/perceiver-io [7] https://arxiv.org/abs/2201.08239v2 https://arxiv.org/abs/2201.08239v2 [8] https://openai.com/dall-e-2/ https://openai.com/dall-e-2/ [9] https://www.fast.ai/ https://www.fast.ai/

mdp2021 4y ago
> And as for recognizing a cat or a dog — that's a problem so trivial today Last time I checked - though it's been a long while I could not check thoroughly owing to other commitments - "«recognizing»" there was "consistently successfully guessing", not "critically defining". It may be that the problem was solved in the latest years, I cannot exclude it - but I have not seen around in the brief "news checking" exercise the signals required for the solution. The real deal is far from trivial. A clock can tell the time but does not know it.

sinenomine 4y ago

Excitement alone won't help us. We should ask our compute overlords to perform their experiments in as open environment as possible, just because we, the public, should have the power to oversee the exact direction this AI revolution is taking us. If you think about it, AI safetyism is a red herring compared to a very real scenario of powerful AGIs working safely as intended, just not in our common interest. The safety of AGI owners' mindset seems like a more pressing concern compared to a hypothetical unsafety of a pile of tensors knit together via gradient descent over internet pictures.