5 ms·

These kinds of discussions always leaves me wondering if people consider how actual humans learn their craft, constantly studying and mimicking others. Inspirat

These kinds of discussions always leaves me wondering if people consider how actual humans learn their craft, constantly studying and mimicking others. Inspiration is to use existing experiences however mixed together, while originality comes from an input or an experience that others have yet to use. "Write a sad song about the MIT license" is certainly such new input, and if I was commissioned to write the song it would be based on inspiration (i.e., "use training on") music I have heard or studied. And yes, none of the musicians I have listened to or have studied will benefit from the endless money fountain I'd acquire from composing such song.

notahacker 3y ago
Even the most derivative of singer songwriters tend to use their own voices rather than a weighted average of the voices of other singers in their genre...
- bobajeff 3y ago
  Is that why so many people sound so much like Adele or some other popular artist?
- arghwhat 3y ago
  Using the skills they presumably developed listening to and copying other singers and studying music, with an instrument built from roughly the same instructions as everyone else. That a person can't sound like the weighted average is human limitation (although with modern pop people do get quite close!), not because new singers aren't trying to. That of course adds variation that we appreciate, but doesn't change the underlying similarity in how acquired skill is mimicry of those who acquired it before us - with very rare exceptions.
  - notahacker 3y ago
    No, sounding like the genre-weighted average of Spotify simply isn't what singers try to do. They haven't listened to that much music, they have actual preferences, they have natural qualities to their voice which they're complimented on or asked to mask, and they're trying to hit notes based on their aural perception of harmony and related theoretical principles not based on the waveforms of other songs involving singer songwriters. The fact that they literally couldn't do what NNs do even if they wanted to also seems quite relevant to the fact that they don't do what NNs do. What next, are we going to argue that what programmers creating new programs are really trying to do is generate a prompt-weighted average of the bytecode of every program they've ever downloaded, and all that business analysis and functional spec and use of high level programming languages and expressed preferences for coding standards is irrelevant?
    - arghwhat 3y ago
      > they have actual preferences That's just a bias. > natural qualities to their voice That's the physical limitations I referred to, which isn't something humans tend to be happy about but can sometimes end up being a differentiating benefit. > What next, are we going to argue that what programmers creating new programs are really trying to do is generate a prompt-weighted average of the bytecode of every program they've ever downloaded That's a horrible strawman. Do you as programmer often read and write bytecode directly?
      - notahacker 3y ago
        > That's just a bias. I'm beginning to assume you're an LLM, because I'm not convinced a human would honestly try to argue that their emotional reaction to their favourite songs is basically equivalent to flipping the values of some bits to ensure that they generate music more similarly to them. > That's a horrible strawman. Do you as programmer often read and write bytecode directly? As an improvising guitarist (even a very mediocre one) my creative process is even further removed from an LLM parsing and generating sound files directly....
        arghwhat 3y ago
        I suspect the issue here is just the assumption that LLMs are "just flipping some bits", while simultaneously putting humanity on some unreachable pedestal. We are all nothing but a horde of molecular machines. Your "you" is just individual neurons reacting to input in accordance to their current chemical state and active connections. All your experiences, unique personality treats, and creativity you add to the process is solely the result of the current state of your network of neurons in response to a particular input. But while an LLM is trained once and then has its state fixed in place regardless of input, we "train" continously, and while an LLM might have experience of an inhuman corpus for a certain subject, we have many "irrelevant" experiences to mix things up. Your "prompt" is also messy, including the current sound of your own heartbeat, the remaining taste in your mouth from your last meal, the feeling of a breeze through your hair as it tickles your neck, while the LLM has just one, maybe two half-assed sentences. This mix of messy experiences and noisy input fuels "creativity". You don't think "I need to copy XYZ", but neither does the AI. You both just react. In some regards our chaos is better, in others it is worse. But while the machinery of an LLM still does not even remotely approach a brain, we should not forget that we are nothing but more a cluster of small machines, assembled from roughly 750 MB worth of blueprint.
yoyohello13 3y ago
In the case of a human studying, a person puts in effort and gets rewarded for their efforts. In the case of AI, a person puts in minimal effort to generate something that devalues the work of all the people who did put in effort.
- fho 3y ago
  Stable Diffusion did cost 500k to train ... I wouldn't call that "minimal effort". (And that is only the computation cost.)
  - deleted 3y ago
    [deleted]
- arghwhat 3y ago
  > In the case of a human studying, a person puts in effort and gets rewarded for their efforts. When someone needs something composed, they don't learn how to write music. They pay someone else the bare minimum, e.g. a few bucks on fiverr. The person will spend the least possible amount of effort to try to make their life go around with the little money they got. When you then use an AI model, the work done for those five bucks is replaced by work done for almost free. Neither the person you would hire or the AI credited those who created the material they trained on.
  - throwaway290 3y ago
    > When you then use an AI model, the work done for those five bucks is replaced by work done for almost free. In other words, you pay a few cents to big tech for a generator that only exists thanks to work real composers, singers etc who now get the grand total of 0$.
    - arghwhat 3y ago
      As opposed to paying $5 to a singular composer that only exists because of other real composers, singers, etc., as they studied the craft. On the other hand, the easy access may cause new types of artists to appear by lowering the bar of entry, or just make custom music more generally available and more widely used as the price makes it a commodity. We also stopped hiring computers (the occupation) and instead pay big tech companies which made computers (the device) available to everyone. And we stopped hiring people to do dangerous manual labor as companies started selling machinery to automate it. Markets change.

notahacker 3y ago

Even the most derivative of singer songwriters tend to use their own voices rather than a weighted average of the voices of other singers in their genre...

bobajeff 3y ago
Is that why so many people sound so much like Adele or some other popular artist?
arghwhat 3y ago
Using the skills they presumably developed listening to and copying other singers and studying music, with an instrument built from roughly the same instructions as everyone else. That a person can't sound like the weighted average is human limitation (although with modern pop people do get quite close!), not because new singers aren't trying to. That of course adds variation that we appreciate, but doesn't change the underlying similarity in how acquired skill is mimicry of those who acquired it before us - with very rare exceptions.
- notahacker 3y ago
  No, sounding like the genre-weighted average of Spotify simply isn't what singers try to do. They haven't listened to that much music, they have actual preferences, they have natural qualities to their voice which they're complimented on or asked to mask, and they're trying to hit notes based on their aural perception of harmony and related theoretical principles not based on the waveforms of other songs involving singer songwriters. The fact that they literally couldn't do what NNs do even if they wanted to also seems quite relevant to the fact that they don't do what NNs do. What next, are we going to argue that what programmers creating new programs are really trying to do is generate a prompt-weighted average of the bytecode of every program they've ever downloaded, and all that business analysis and functional spec and use of high level programming languages and expressed preferences for coding standards is irrelevant?
  - arghwhat 3y ago
    > they have actual preferences That's just a bias. > natural qualities to their voice That's the physical limitations I referred to, which isn't something humans tend to be happy about but can sometimes end up being a differentiating benefit. > What next, are we going to argue that what programmers creating new programs are really trying to do is generate a prompt-weighted average of the bytecode of every program they've ever downloaded That's a horrible strawman. Do you as programmer often read and write bytecode directly?
    - notahacker 3y ago
      > That's just a bias. I'm beginning to assume you're an LLM, because I'm not convinced a human would honestly try to argue that their emotional reaction to their favourite songs is basically equivalent to flipping the values of some bits to ensure that they generate music more similarly to them. > That's a horrible strawman. Do you as programmer often read and write bytecode directly? As an improvising guitarist (even a very mediocre one) my creative process is even further removed from an LLM parsing and generating sound files directly....
      - arghwhat 3y ago
        I suspect the issue here is just the assumption that LLMs are "just flipping some bits", while simultaneously putting humanity on some unreachable pedestal. We are all nothing but a horde of molecular machines. Your "you" is just individual neurons reacting to input in accordance to their current chemical state and active connections. All your experiences, unique personality treats, and creativity you add to the process is solely the result of the current state of your network of neurons in response to a particular input. But while an LLM is trained once and then has its state fixed in place regardless of input, we "train" continously, and while an LLM might have experience of an inhuman corpus for a certain subject, we have many "irrelevant" experiences to mix things up. Your "prompt" is also messy, including the current sound of your own heartbeat, the remaining taste in your mouth from your last meal, the feeling of a breeze through your hair as it tickles your neck, while the LLM has just one, maybe two half-assed sentences. This mix of messy experiences and noisy input fuels "creativity". You don't think "I need to copy XYZ", but neither does the AI. You both just react. In some regards our chaos is better, in others it is worse. But while the machinery of an LLM still does not even remotely approach a brain, we should not forget that we are nothing but more a cluster of small machines, assembled from roughly 750 MB worth of blueprint.

yoyohello13 3y ago

In the case of a human studying, a person puts in effort and gets rewarded for their efforts. In the case of AI, a person puts in minimal effort to generate something that devalues the work of all the people who did put in effort.

fho 3y ago
Stable Diffusion did cost 500k to train ... I wouldn't call that "minimal effort". (And that is only the computation cost.)
- deleted 3y ago
  [deleted]
arghwhat 3y ago
> In the case of a human studying, a person puts in effort and gets rewarded for their efforts. When someone needs something composed, they don't learn how to write music. They pay someone else the bare minimum, e.g. a few bucks on fiverr. The person will spend the least possible amount of effort to try to make their life go around with the little money they got. When you then use an AI model, the work done for those five bucks is replaced by work done for almost free. Neither the person you would hire or the AI credited those who created the material they trained on.
- throwaway290 3y ago
  > When you then use an AI model, the work done for those five bucks is replaced by work done for almost free. In other words, you pay a few cents to big tech for a generator that only exists thanks to work real composers, singers etc who now get the grand total of 0$.
  - arghwhat 3y ago
    As opposed to paying $5 to a singular composer that only exists because of other real composers, singers, etc., as they studied the craft. On the other hand, the easy access may cause new types of artists to appear by lowering the bar of entry, or just make custom music more generally available and more widely used as the price makes it a commodity. We also stopped hiring computers (the occupation) and instead pay big tech companies which made computers (the device) available to everyone. And we stopped hiring people to do dangerous manual labor as companies started selling machinery to automate it. Markets change.