3 ms·

I can't help but think that whenever copilot is able to help with a solution, it's because the language itself doesn't allow you to express solutions concisely.

I can't help but think that whenever copilot is able to help with a solution, it's because the language itself doesn't allow you to express solutions concisely. I looked at the solutions in the article, and very little of the code is actually part of solving the problem. There are a few bytes of relevant information in each line. I usually use Common Lisp, and whenever I find myself typing boilerplate of a type that copilot would be able to help with, it's because I haven't used the right abstractions. As an extreme example of this, I solved day 1 in KAP (my APL dialect) and for the second problem, my solution was 8 characters long. There was no real need to have anything type that out for me.

jakear 5y ago
Agreed, even with trivial JS constructs parts 1 and 2 can be solved (including parsing) in 58 chars (and only O(n), not O(n*w) like Copilot's solution). The copilot prompt alone for the just the logic part of part one was 98 chars. Surprise, programming languages are more efficient at encoding logic than English.
thomasahle 5y ago
It's a question of whether copilot can be "creative". I'm mit sure, but it did manage to solve some of the harder leetcode questions here https://youtu.be/FHwnrYm0mNc https://youtu.be/FHwnrYm0mNc and I'm curious how it will fare on the later advent of code problems.
- callmekit 5y ago
  The exact answers to leetcode questions are probably included many times in the Copilot dataset, in many programming languages. Fresh Advent of code problems are much more interesting.
  - YeGoblynQueenne 5y ago
    Copilot is trained on examples of programs, so it will not be able to produce programs unlike any it has seen before. This will not be easy to see in any kind of coding exercise. It is very hard to come up with coding problems with solutions that have never been written before, at least for a coding competition like Advent of Code. For example, we don't know of polynomial time solutions to the Travelling Salesman problem but it wouldn't make sense to have that as a coding challenge in Advent of Code, or Leetcode, or anywhere else. Since nobody has ever produced a program solving such a problem, Copilot will not be able to produce a solution either, but it's not designed to do that anyway.
8fingerlouie 5y ago
It could be that my use case was simple enough, but for a class i'm taking, we were tasked with creating some code for a web application, as well as unit tests for it, and while copilot wasn't of much use for coding the actual application, it pretty much nailed autogenerating the test cases. Now, as i said, this was for a class, so it was simple. Sadly work policies at my employer forbids the use of copilot (or anything transmitting code out of the company), so i can't really test it with "real code",
jvictor118 5y ago
I'm fascinated by APL but man is it hard to get used to. I keep waiting for someone to build something like APL but sufficiently modernized as to actually be useful to me :)
- lokedhs 5y ago
  That is what I'm trying to do, actually. However, the core syntax for fundamental array operations are still the same, but there is a layer of regular imperative structures around it, so you can write code that looks like JS if you want. It's just not as efficient. It's not ready for actual use yet though, but any suggestions would of course be appreciated.

jakear 5y ago

Agreed, even with trivial JS constructs parts 1 and 2 can be solved (including parsing) in 58 chars (and only O(n), not O(n*w) like Copilot's solution). The copilot prompt alone for the just the logic part of part one was 98 chars. Surprise, programming languages are more efficient at encoding logic than English.

thomasahle 5y ago

It's a question of whether copilot can be "creative". I'm mit sure, but it did manage to solve some of the harder leetcode questions here https://youtu.be/FHwnrYm0mNc https://youtu.be/FHwnrYm0mNc and I'm curious how it will fare on the later advent of code problems.

callmekit 5y ago
The exact answers to leetcode questions are probably included many times in the Copilot dataset, in many programming languages. Fresh Advent of code problems are much more interesting.
- YeGoblynQueenne 5y ago
  Copilot is trained on examples of programs, so it will not be able to produce programs unlike any it has seen before. This will not be easy to see in any kind of coding exercise. It is very hard to come up with coding problems with solutions that have never been written before, at least for a coding competition like Advent of Code. For example, we don't know of polynomial time solutions to the Travelling Salesman problem but it wouldn't make sense to have that as a coding challenge in Advent of Code, or Leetcode, or anywhere else. Since nobody has ever produced a program solving such a problem, Copilot will not be able to produce a solution either, but it's not designed to do that anyway.

8fingerlouie 5y ago

It could be that my use case was simple enough, but for a class i'm taking, we were tasked with creating some code for a web application, as well as unit tests for it, and while copilot wasn't of much use for coding the actual application, it pretty much nailed autogenerating the test cases. Now, as i said, this was for a class, so it was simple. Sadly work policies at my employer forbids the use of copilot (or anything transmitting code out of the company), so i can't really test it with "real code",

jvictor118 5y ago

I'm fascinated by APL but man is it hard to get used to. I keep waiting for someone to build something like APL but sufficiently modernized as to actually be useful to me :)

lokedhs 5y ago
That is what I'm trying to do, actually. However, the core syntax for fundamental array operations are still the same, but there is a layer of regular imperative structures around it, so you can write code that looks like JS if you want. It's just not as efficient. It's not ready for actual use yet though, but any suggestions would of course be appreciated.