3 ms·

The big difference is that you are a human with a full life and an unimaginable amount of experiences that have been accruing for decades. You have learned in s

The big difference is that you are a human with a full life and an unimaginable amount of experiences that have been accruing for decades. You have learned in school; played outside; read books and blog posts; listened to music; written code; had deep conversations with interesting people; fallen in love; gotten hurt. You have seen and smelled and touched and thought an almost infinite variety of things, you have experienced ecstasy and tragedy and everything in between, and all that comes together to make you an incredibly unique and special person. This colors your every experience, every idea, every sensation. GitHub Copilot is a machine and the sum total of everything it knows is code from GitHub. Literally its entire universe! There is absolutely no "experience" it has ever had that is not code from GitHub. There is no other fount of knowledge or inspiration from which it can draw. It is incapable of doing anything other than rearranging the Lego blocks in its brain — in fact, it doesn't even have a notion of what it's doing! Neural networks are a very complex math functions that take in some numbers, multiply them and spit out some other numbers. That's it.

horsawlarway 3y ago
I think this is a pretty reductive way to look at what's happening. My personal take - it's more like the infinite monkey theorem. --- a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type any given text, such as the complete works of William Shakespeare. In fact, the monkey would almost surely type every possible finite text an infinite number of times. --- except for each letter, we're jostling the monkey's hand a little based on the letters he's already hit, and the training data. Ex: There is a total problem space of "Valid Python" or "Valid JSON" and these machines are designed to explore parts of that problem space directed by the user and their training data. And just like it only takes 52 cards to make any given shuffle essentially an entirely new & never before seen section of the problem space "decks of cards" (see: https://www.mcgill.ca/oss/article/did-you-know-infographics/there-are-more-ways-arrange-deck-cards-there-are-atoms-earth#:~:text=No%20one%20has%2C%20and%20will,Just%20think%20about%20that https://www.mcgill.ca/oss/article/did-you-know-infographics/......) I don't really know of a clear way to distinguish what it's doing from "human inspiration & creativity" because those activities also tend to recombine previous experiences and data to create new but related works. Basically - we also tend to explore related problem spaces based on what is currently happening around us and our life experiences (This is why the arts have period specific themes and styles). This thing just does it a whole lot faster than we do. Whether that's good or bad is a different question.
- jakelazaroff 3y ago
  > I don't really know of a clear way to distinguish what it's doing from "human inspiration & creativity" because those activities also tend to recombine previous experiences and data to create new but related works. This is core of your argument — "why isn't it the same?". I hear it so often and it's just utterly unconvincing to me. Neural networks are "function approximators" — that is, given a function f, you train a network on a subset of its domain and corresponding known values in its range, and you hope that when you feed in other x values you get accurate predictions for the y values. I want to see an affirmative argument that this is how our brains work. Where is the evidence that we can be modeled as functions in this way? Why is your default hypothesis that neural networks are analogous to human brains?
  - horsawlarway 3y ago
    > I want to see an affirmative argument that this is how our brains work. I don't know why you think this matters. I don't care at all if this is how our brains work - I care that the result is obviously analogous to what we call creativity - there are only so many ways you can arrange a 1kx1k pixel grid. Human digital artists explore that space, Machines can explore that space. We happen to like some parts of that space more than others, and so we guide the machines to explore those parts more heavily. So... > Why is your default hypothesis that neural networks are analogous to human brains? Because I can see that they are obviously capable of a similar skill. Does it matter that your earth-mover is using hydraulics and not human muscles? Fuck no. It's digging. Does it matter that this AI is using pre-trained weights and not human brain flesh? Fuck no. It's digging. ---- Now - entirely outside of that discussion: I also happen to think modern copyright is one of the absolute worst abuses in human history (and it's getting significantly worse right now, with forced rentals, digital oversight, locked down devices, DMCA, etc) Our cultural heritage is OURS: collectively. I am not thrilled, but accept, the original intent of 7 years copyright with 7 years renewal. I am absolutely, morally, ethically, opposed to modern copyright of "Life of author + 70 years." It means that as adults, we cannot tell the stories we heard as children in our own voices. We cannot bridge our culture between generations. We have sold our soul to fucking mickey fucking mouse. Fuck that bullshit. I'd rather steal it all than play that game.

horsawlarway 3y ago

I think this is a pretty reductive way to look at what's happening. My personal take - it's more like the infinite monkey theorem. --- a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type any given text, such as the complete works of William Shakespeare. In fact, the monkey would almost surely type every possible finite text an infinite number of times. --- except for each letter, we're jostling the monkey's hand a little based on the letters he's already hit, and the training data. Ex: There is a total problem space of "Valid Python" or "Valid JSON" and these machines are designed to explore parts of that problem space directed by the user and their training data. And just like it only takes 52 cards to make any given shuffle essentially an entirely new & never before seen section of the problem space "decks of cards" (see: https://www.mcgill.ca/oss/article/did-you-know-infographics/there-are-more-ways-arrange-deck-cards-there-are-atoms-earth#:~:text=No%20one%20has%2C%20and%20will,Just%20think%20about%20that https://www.mcgill.ca/oss/article/did-you-know-infographics/......) I don't really know of a clear way to distinguish what it's doing from "human inspiration & creativity" because those activities also tend to recombine previous experiences and data to create new but related works. Basically - we also tend to explore related problem spaces based on what is currently happening around us and our life experiences (This is why the arts have period specific themes and styles). This thing just does it a whole lot faster than we do. Whether that's good or bad is a different question.

jakelazaroff 3y ago
> I don't really know of a clear way to distinguish what it's doing from "human inspiration & creativity" because those activities also tend to recombine previous experiences and data to create new but related works. This is core of your argument — "why isn't it the same?". I hear it so often and it's just utterly unconvincing to me. Neural networks are "function approximators" — that is, given a function f, you train a network on a subset of its domain and corresponding known values in its range, and you hope that when you feed in other x values you get accurate predictions for the y values. I want to see an affirmative argument that this is how our brains work. Where is the evidence that we can be modeled as functions in this way? Why is your default hypothesis that neural networks are analogous to human brains?
- horsawlarway 3y ago
  > I want to see an affirmative argument that this is how our brains work. I don't know why you think this matters. I don't care at all if this is how our brains work - I care that the result is obviously analogous to what we call creativity - there are only so many ways you can arrange a 1kx1k pixel grid. Human digital artists explore that space, Machines can explore that space. We happen to like some parts of that space more than others, and so we guide the machines to explore those parts more heavily. So... > Why is your default hypothesis that neural networks are analogous to human brains? Because I can see that they are obviously capable of a similar skill. Does it matter that your earth-mover is using hydraulics and not human muscles? Fuck no. It's digging. Does it matter that this AI is using pre-trained weights and not human brain flesh? Fuck no. It's digging. ---- Now - entirely outside of that discussion: I also happen to think modern copyright is one of the absolute worst abuses in human history (and it's getting significantly worse right now, with forced rentals, digital oversight, locked down devices, DMCA, etc) Our cultural heritage is OURS: collectively. I am not thrilled, but accept, the original intent of 7 years copyright with 7 years renewal. I am absolutely, morally, ethically, opposed to modern copyright of "Life of author + 70 years." It means that as adults, we cannot tell the stories we heard as children in our own voices. We cannot bridge our culture between generations. We have sold our soul to fucking mickey fucking mouse. Fuck that bullshit. I'd rather steal it all than play that game.