6 ms·

Lots of people here hate Marcus, and he can lay it on thick, but I quite like him. He has actually studied human cognition and intelligence outside of programmi

Lots of people here hate Marcus, and he can lay it on thick, but I quite like him. He has actually studied human cognition and intelligence outside of programming – something that can't be said about a great many AI pundits. I'm doing an interdisciplinary PhD in CS and Music right, so both music cognition and AI is in my program. From what I can see (which is not a lot of data, I admit) no one who has actually studied music cognition would think LLMs are going to lead to AGI by scaling. Linguistic thinking is only a small part of human intelligence, and certainly not the most complex. LLMs are so, so far from being able to the thinking that goes in a real-time musical improvisation context it's laughable.

klipt 2y ago
Well music is arguably its own language. You wouldn't expect a monolingual English speaker to speak fluent Chinese. In the same way you can't expect a musically untrained person (or model) to "speak music". But if you took an LLM size neural network and trained it on all the music in the world - I dare say you may get some interesting results.
- seunosewa 2y ago
  It's been done. We have AI music websites
  - perfmode 2y ago
    such as?
    - captaindiego 2y ago
      Give suno a try: https://suno.com/home https://suno.com/home
    - williamcotton 2y ago
      https://www.riffusion.com https://www.riffusion.com
    - dumpsterdiver 2y ago
      Suno! https://suno.com/song/394fa73c-ba78-4caa-94c1-4da9c964846c https://suno.com/song/394fa73c-ba78-4caa-94c1-4da9c964846c
mountainriver 2y ago
We aren't very far away from that, if you think we are you should look closer
nyolfen 2y ago
> LLMs are so, so far from being able to the thinking that goes in a real-time musical improvisation context it's laughable. have you actually tried any of the commercial AI music generation tools from the last year, eg suno? not an LLM but rather (probably) diffusion, it made my jaw drop the first time i played with it. but it turns out you can also use diffusion for language models https://www.inceptionlabs.ai/ https://www.inceptionlabs.ai/
- iainctduncan 2y ago
  This is exactly my point. The idea that generating music is the sum total of all the cognition that goes on in real time is totally off base. Can we make crappy simulacrums of recorded music? sure. Can we make a program that could: work the physical instrument, react in milliseconds to other players, create new work off input it's never heard before, react to things in the room, and do a myriad of other things a performer does on stage in real time? Not even remotely close. THAT is what would be required to call it AGI – thinking (ALL the thinking) on par with a highly trained human. Pretending anything else is AGI is nonsense. Is it impressive? sure. Frighteningly so even. But it's not AGI and the claims that it is are pure huckersterism. They just round the term down to whatever the hell is convenient for the pitch.
  - nyolfen 2y ago
    yes, i absolutely believe we are less than two years from what you describe (aside from physically manipulating an instrument -- but robotics seems to be quickly picking up pace too). what you are imagining is only a difference in speed, not kind. this is the 'god of the gaps' argument, over and over, every time some insurmountable previous benchmark is shattered -- well, it will never be able to do my special thing > thinking (ALL the thinking) on par with a highly trained human you are mistaking means for ends. "an automobile must be able to perform dressage on par with a fine thoroughbred!"
    - kranke155 2y ago
      We are likely more than 10 year away. The algorithms haven’t been found yet I think.
JohnKemeny 2y ago
I, for one, agree with you, and my area is cognitive science and artificial intelligence. It's staggering to see the hive-mind of Sam Altman followers chanting about AGI in a Markov chain. Yes, we can today make natural-sounding hyper-average text, images, and perhaps even music. But that's not what writing and creating is about.
- kranke155 2y ago
  As a writer, people thinking about LLms writing a book-length story don’t realise it’s in the wrong problem space. LLMs can solve equations and write code, like make a React web page. So they assume “a story is just an equation, we just need to get the LLM to write an equation”. Wrong imo. Storytelling is above language in the problem space. Storytelling can be done in multiple methods - visual (images), auditory (music), characters (letters and language) and gestural (using your body, such as a clown or a mime, Im thinking of Chaplin or Jacques Tati). That means storytelling exists above language, not as a function of it. If you agree with the Noah Harari vision of things, which I do, then storytelling is actually deeply connected somehow to our level of awareness and it seems to emerge with consciousness. Which means that thinking that an LLM that can understand story because it understands language is… foolish. Storytelling lives in a problem space above language. Thinking you can get an LLM to tell a story because you can get it to write a sentence is a misunderstanding of what problem space stories are in. It’s the same as thinking that if an LLM can write a React web app it can invent brand new and groundbreaking computer science. It can’t. Not for now anyway.
  - namaria 2y ago
    I agree. Language manipulation is a projection of something higher order. LLMs can shuffle the projection but they clearly cannot go over the gap to the higher order thinking.
- fooker 2y ago
  > AGI in a Markov chain Do you understand why modern LLMs are different from Markov chains?
j_bum 2y ago
What a wonderful blend of fields for your PhD. I tend to agree with you, but I would be cautious in stating that an understanding of music is required for AGI. Plenty of humans are tone deaf and can’t understand music, yet are cognitively capable. I’d love to hear about your research. I have my PhD in neuroscience and a undergrad degree in piano performance. Not quite the same path as you, but similar!
llamajams 2y ago
Hello, I am interested in your area of research. Would you mind linking to some of your work?

klipt 2y ago

Well music is arguably its own language. You wouldn't expect a monolingual English speaker to speak fluent Chinese. In the same way you can't expect a musically untrained person (or model) to "speak music". But if you took an LLM size neural network and trained it on all the music in the world - I dare say you may get some interesting results.

seunosewa 2y ago
It's been done. We have AI music websites
- perfmode 2y ago
  such as?
  - captaindiego 2y ago
    Give suno a try: https://suno.com/home https://suno.com/home
  - williamcotton 2y ago
    https://www.riffusion.com https://www.riffusion.com
  - dumpsterdiver 2y ago
    Suno! https://suno.com/song/394fa73c-ba78-4caa-94c1-4da9c964846c https://suno.com/song/394fa73c-ba78-4caa-94c1-4da9c964846c

mountainriver 2y ago

We aren't very far away from that, if you think we are you should look closer

nyolfen 2y ago

> LLMs are so, so far from being able to the thinking that goes in a real-time musical improvisation context it's laughable. have you actually tried any of the commercial AI music generation tools from the last year, eg suno? not an LLM but rather (probably) diffusion, it made my jaw drop the first time i played with it. but it turns out you can also use diffusion for language models https://www.inceptionlabs.ai/ https://www.inceptionlabs.ai/

iainctduncan 2y ago
This is exactly my point. The idea that generating music is the sum total of all the cognition that goes on in real time is totally off base. Can we make crappy simulacrums of recorded music? sure. Can we make a program that could: work the physical instrument, react in milliseconds to other players, create new work off input it's never heard before, react to things in the room, and do a myriad of other things a performer does on stage in real time? Not even remotely close. THAT is what would be required to call it AGI – thinking (ALL the thinking) on par with a highly trained human. Pretending anything else is AGI is nonsense. Is it impressive? sure. Frighteningly so even. But it's not AGI and the claims that it is are pure huckersterism. They just round the term down to whatever the hell is convenient for the pitch.
- nyolfen 2y ago
  yes, i absolutely believe we are less than two years from what you describe (aside from physically manipulating an instrument -- but robotics seems to be quickly picking up pace too). what you are imagining is only a difference in speed, not kind. this is the 'god of the gaps' argument, over and over, every time some insurmountable previous benchmark is shattered -- well, it will never be able to do my special thing > thinking (ALL the thinking) on par with a highly trained human you are mistaking means for ends. "an automobile must be able to perform dressage on par with a fine thoroughbred!"
  - kranke155 2y ago
    We are likely more than 10 year away. The algorithms haven’t been found yet I think.

JohnKemeny 2y ago

I, for one, agree with you, and my area is cognitive science and artificial intelligence. It's staggering to see the hive-mind of Sam Altman followers chanting about AGI in a Markov chain. Yes, we can today make natural-sounding hyper-average text, images, and perhaps even music. But that's not what writing and creating is about.

kranke155 2y ago
As a writer, people thinking about LLms writing a book-length story don’t realise it’s in the wrong problem space. LLMs can solve equations and write code, like make a React web page. So they assume “a story is just an equation, we just need to get the LLM to write an equation”. Wrong imo. Storytelling is above language in the problem space. Storytelling can be done in multiple methods - visual (images), auditory (music), characters (letters and language) and gestural (using your body, such as a clown or a mime, Im thinking of Chaplin or Jacques Tati). That means storytelling exists above language, not as a function of it. If you agree with the Noah Harari vision of things, which I do, then storytelling is actually deeply connected somehow to our level of awareness and it seems to emerge with consciousness. Which means that thinking that an LLM that can understand story because it understands language is… foolish. Storytelling lives in a problem space above language. Thinking you can get an LLM to tell a story because you can get it to write a sentence is a misunderstanding of what problem space stories are in. It’s the same as thinking that if an LLM can write a React web app it can invent brand new and groundbreaking computer science. It can’t. Not for now anyway.
- namaria 2y ago
  I agree. Language manipulation is a projection of something higher order. LLMs can shuffle the projection but they clearly cannot go over the gap to the higher order thinking.
fooker 2y ago
> AGI in a Markov chain Do you understand why modern LLMs are different from Markov chains?

j_bum 2y ago

What a wonderful blend of fields for your PhD. I tend to agree with you, but I would be cautious in stating that an understanding of music is required for AGI. Plenty of humans are tone deaf and can’t understand music, yet are cognitively capable. I’d love to hear about your research. I have my PhD in neuroscience and a undergrad degree in piano performance. Not quite the same path as you, but similar!

llamajams 2y ago

Hello, I am interested in your area of research. Would you mind linking to some of your work?