3 ms·

Nothing you say is credible unless you tell me you have tried copilot. Without you having tried copilot and seen its capabilities we have no common ground. So g

by _7bxa 4y ago

Nothing you say is credible unless you tell me you have tried copilot. Without you having tried copilot and seen its capabilities we have no common ground. So get back to me once you have used it. I claim copilot is capable of creating. When copilot writes an amazing lodash one liner in my code --that isn't regurgitating an existing lodash snippet, it's creating something new to fit my use case. This is undeniable and there is nothing to argue here. Copilot regularly looks at my code and writes new code that is highly specific to my existing code. It's honestly better than me at languages I don't know well (like C). And yes, 0.1% of the time copilot is spitting out existing code verbatim; the other 99.9% of the time copilot is not overfitting and is synthesizing new code. Luckily most companies don't needs clean room implementations. Copilot writes code that is adapted to my existing code base. This is quite obviously and undeniably not just regurgitation because my codebase is unique to me. Lastly copilot is a better programmer (locally) than many of my peers. It can write better lodash one liners, amongst other things, and while that's embarrassing it's true.

BeefWellington 4y ago
> Nothing you say is credible unless you tell me you have tried copilot. Without you having tried copilot and seen its capabilities we have no common ground. So get back to me once you have used it. I posted elsewhere in this thread talking about my experiences (both old and recent) using it. Dismissing a reply you don't like simply because of a (terrible, given the tool is very available) assumption is just a poor quality response. > I claim copilot is capable of creating. When copilot writes an amazing lodash one liner in my code --that isn't regurgitating an existing lodash snippet, it's creating something new to fit my use case. This is undeniable and there is nothing to argue here. Copilot regularly looks at my code and writes new code that is highly specific to my existing code. It's honestly better than me at languages I don't know well (like C). > And yes, 0.1% of the time copilot is spitting out existing code verbatim; the other 99.9% of the time copilot is not overfitting and is synthesizing new code. Luckily most companies don't needs clean room implementations. Where do you get this figure of 0.1%? I'm not aware of anyone having studied it, and absent that the figure seems entirely fabricated and could be higher or lower. Indeed, the way it works via prompting suggests that an overall percentage is irrelevant if 100% of the time you ask it for specific things it generates copyrighted or strictly licensed code. If you have references though, I'm interested. > Copilot writes code that is adapted to my existing code base. This is quite obviously and undeniably not just regurgitation because my codebase is unique to me. Using your code to show you code you might likely write isn't "creative" and is exactly regurgitating what it's seen. Is a Markov chain "Creative"? That's essentially what you're describing here. > Lastly copilot is a better programmer (locally) than many of my peers. It can write better lodash one liners, amongst other things, and while that's embarrassing it's true. You've used lodash one-liners as an example a couple of times now. Why? Why is that a litmus test for a good programmer? What makes the copilot generated ones superior? Do you have examples? My experiences with copilot as I've shared elsewhere are that it produces about 90% of the time code that needs to be debugged, doesn't quite fit coding conventions we use, and often doesn't do exactly what I'm looking for. It's fine if you're using it to stub in specific kinds of boilerplate or using it to generate a function to do some very standard math thing that exists or should exist in a library somewhere.
- marmada 4y ago
  The 0.1% figure comes from OpenAI: https://twitter.com/eevee/status/1410037309848752128 https://twitter.com/eevee/status/1410037309848752128 (person disagrees w/ me, the image is what's relevant). I talk about Lodash one-liners because it makes it pretty obvious that Copilot is not just copy-pasting code (which would be copyright infringement). It's quite unlikely (even if we consider all variables to have the same name), that Copilot is copy-pasting an exact copy of some other snippet, given that the snippet is very specific to my problems. (I'm not asking it to write a generic math function, I'm asking it to use lodash on data structures in my codebase to accomplish a very specific outcome). By quite unlikely I mean (1 / (2 ^ 32)), if we consider the one-liner to be composed of 32 different AST nodes. > Is a Markov chain "Creative"? That's essentially what you're describing here. Have you read the paper that (eventually) inspired Copilot? "Attention is All You Need". It's not a Markov chain. There's many, many, many different steps / layers. I think if you know & understand the fundamental building block that is responsible for it working (the "transformer"), then a lot of the worries around plagiarism go away. Also. What is coding if not a search problem over a very large space? I'm searching for the next N lines to write over the space of all possible lines. To guide my search I use things like my prior knowledge. In my head, this knowledge is encoded using neurons. In Copilot, the knowledge is encoded in parameters. Abstract concepts in my head are encoded in neurons. In Copilot's "head", abstract concepts are encoded using "embedding vectors" of 512 bytes (maybe more / less, not sure). Yeah, maybe I use more than 512 bytes to encode a concept, but still, I don't see a huge difference. If I'm not plagiarizing, then I can't imagine Copilot is (except for in the 0.1% of cases where it's over-fitting)

by _7bxa 4y ago

BeefWellington 4y ago

> Nothing you say is credible unless you tell me you have tried copilot. Without you having tried copilot and seen its capabilities we have no common ground. So get back to me once you have used it. I posted elsewhere in this thread talking about my experiences (both old and recent) using it. Dismissing a reply you don't like simply because of a (terrible, given the tool is very available) assumption is just a poor quality response. > I claim copilot is capable of creating. When copilot writes an amazing lodash one liner in my code --that isn't regurgitating an existing lodash snippet, it's creating something new to fit my use case. This is undeniable and there is nothing to argue here. Copilot regularly looks at my code and writes new code that is highly specific to my existing code. It's honestly better than me at languages I don't know well (like C). > And yes, 0.1% of the time copilot is spitting out existing code verbatim; the other 99.9% of the time copilot is not overfitting and is synthesizing new code. Luckily most companies don't needs clean room implementations. Where do you get this figure of 0.1%? I'm not aware of anyone having studied it, and absent that the figure seems entirely fabricated and could be higher or lower. Indeed, the way it works via prompting suggests that an overall percentage is irrelevant if 100% of the time you ask it for specific things it generates copyrighted or strictly licensed code. If you have references though, I'm interested. > Copilot writes code that is adapted to my existing code base. This is quite obviously and undeniably not just regurgitation because my codebase is unique to me. Using your code to show you code you might likely write isn't "creative" and is exactly regurgitating what it's seen. Is a Markov chain "Creative"? That's essentially what you're describing here. > Lastly copilot is a better programmer (locally) than many of my peers. It can write better lodash one liners, amongst other things, and while that's embarrassing it's true. You've used lodash one-liners as an example a couple of times now. Why? Why is that a litmus test for a good programmer? What makes the copilot generated ones superior? Do you have examples? My experiences with copilot as I've shared elsewhere are that it produces about 90% of the time code that needs to be debugged, doesn't quite fit coding conventions we use, and often doesn't do exactly what I'm looking for. It's fine if you're using it to stub in specific kinds of boilerplate or using it to generate a function to do some very standard math thing that exists or should exist in a library somewhere.

marmada 4y ago
The 0.1% figure comes from OpenAI: https://twitter.com/eevee/status/1410037309848752128 https://twitter.com/eevee/status/1410037309848752128 (person disagrees w/ me, the image is what's relevant). I talk about Lodash one-liners because it makes it pretty obvious that Copilot is not just copy-pasting code (which would be copyright infringement). It's quite unlikely (even if we consider all variables to have the same name), that Copilot is copy-pasting an exact copy of some other snippet, given that the snippet is very specific to my problems. (I'm not asking it to write a generic math function, I'm asking it to use lodash on data structures in my codebase to accomplish a very specific outcome). By quite unlikely I mean (1 / (2 ^ 32)), if we consider the one-liner to be composed of 32 different AST nodes. > Is a Markov chain "Creative"? That's essentially what you're describing here. Have you read the paper that (eventually) inspired Copilot? "Attention is All You Need". It's not a Markov chain. There's many, many, many different steps / layers. I think if you know & understand the fundamental building block that is responsible for it working (the "transformer"), then a lot of the worries around plagiarism go away. Also. What is coding if not a search problem over a very large space? I'm searching for the next N lines to write over the space of all possible lines. To guide my search I use things like my prior knowledge. In my head, this knowledge is encoded using neurons. In Copilot, the knowledge is encoded in parameters. Abstract concepts in my head are encoded in neurons. In Copilot's "head", abstract concepts are encoded using "embedding vectors" of 512 bytes (maybe more / less, not sure). Yeah, maybe I use more than 512 bytes to encode a concept, but still, I don't see a huge difference. If I'm not plagiarizing, then I can't imagine Copilot is (except for in the 0.1% of cases where it's over-fitting)