28 ms·

Coping with Copilot

• lakomen 4y ago
  1. Entertaining read The unavoidable question is, will our and their kids be able to function without their machine overlords? In the 80s I had to come up with my own solutions because no internet and no books, being 11. With the internet I searched it with the error messages. Pre 2000 I got useful and accurate results, post 2k I got more and opinions than results, which had to do with the amount of mediocre intellect on the net and Google trying to provide a larger list of results, but mostly the stupid made it onto the net. Om github you already have the stupid I'd imagine since it's such a popular platform, although many have left it when Microsoft tool over. But many also let their work be abused by Microsoft to sell the Copilot service, so I'm guessing the stupid is very present on Github. The evil of taking someone's work, that they shared with good intent to help other people, run it through a machine network and sell it to the "other man" again. And now it's even corrupting our children and our children's children. Pure evil Microsoft.
• peppermint_tea 4y ago
  grade on explaning the code and not the code itself? Someone who can explain some code understood it (or memorized the explanations) from an academic point of view, it is not that bad. even for the people in the "memorized" category, they memorized the explanation... brings them quite close to understanding IMO
• footlose_3815 4y ago
  [dead]
• GrumpyYoungMan 4y ago
  Straight up assembly language, preferably Motorola MC680x0 or some other contemporary that was meant to be coded by hand; it's the only way, man! Plus students will gain some appreciation for how microprocessors actually work. In the same vein, just have them write code out on paper during exams and coding competitions like the way we did it back in my day. It's not as if they're not going to need that skill anyway when they get whiteboarded during interview loops when even the most lenient employers get exasperated at "CoPilot-only" graduates. </tongue-in-cheek>
• jeffbee 4y ago
  I guess CS programs are going to need to grow up and teach real topics like ethics, technical communications, effective testing, and large-scale systems design instead of how to write simple loops, which is something I learned at the age of 4 from “Gortek and the Microchips”.
  • eru 4y ago
    What makes you think GPT-3 (or similar) won't write your ethics essay for you?
    • jeffbee 4y ago
      That’s very easy to deal with. Oral exams or blue-book exams.
      • eru 4y ago
        What's a blue-book exam? You can test loop-writing in an oral-exam, too. Thus your proposed solution for GPT-3 writing ethics essays is so good, that it fixes the original problem.
        • jeffbee 4y ago
          You're right, that a pretty good approach anyway. Teach your whole course in a fake language or on a fake machine like MMIX.
          • eru 4y ago
            > Teach your whole course in a fake language or on a fake machine like MMIX. Sorry, what's the point of that? I don't think it would make any difference for cheating?
            • jeffbee 4y ago
              There isn't any MMIX machine code in Copilot's training corpus.
• zitterbewegung 4y ago
  Just add a requirement where the student has to explain how the code works. To be honest there should be a class called language model querying because for simple tasks it does well but eventually you will have to debug the output (not just copilot but GPT3 and dall-e).
  • emeryberger 4y ago
    Article author here. Turns out language models already can explain how code works, pretty convincingly. I gave a prompt to OpenAI, got the solution, and then asked it to explain the solution (possibly easier to read in my tweet: everything in green below is from OpenAI - https://twitter.com/emeryberger/status/1560618825208320001 https://twitter.com/emeryberger/status/1560618825208320001). I wrote the comment asking a question ("# How did you solve this problem? Explain how you got each digit from each number.") and Tab completed the rest: # I solved this problem by using the % operator to get each digit from the number. I declared and initialized a sum variable to keep track of the sum of the digits of the number. I set a temp variable equal to the input number. I then used a while loop to loop through each digit of the number. In the loop, I used the % operator to get the remainder of temp divided by 10. This gave me the last digit of the number. I added this digit to the sum variable. I then divided temp by 10 to remove the last digit of the number. I repeated these steps until all of the digits of the number were added to the sum variable. Finally, I printed the value of the sum variable.
  • deleted 4y ago
    [deleted]
  • wiradikusuma 4y ago
    > Just add a requirement where the student has to explain how the code works. There's an AI for that!
  • jefftk 4y ago
    > there should be a class called language model querying Do you think the skills taught in that class would be relevant for very long?
• niek_pas 4y ago
  The scrolljacking on this site is awful. Why do people keep doing this?
  • foepys 4y ago
    I'm using reader mode in Firefox more and more often because websites just cannot keep their hands off my inputs nowadays. It's infuriating.
  • tazjin 4y ago
    Only way to use the web is worth uBlock disabling all JS by default. Works on mobile, too.
  • stevebmark 4y ago
    This is why I love the disable javascript plugin for Chrome. One click and terrible sites like these are tamed. uBlock origin is also useful to block sticky headers.
• fancyfredbot 4y ago
  I get that copilot is AI and pretty cool, but students could look up a Fibonacci program on Google before it existed. What's more these algorithms are written in books. If students wanted to cheat on your "write depth first search" assignment then they already were.
  • jefftk 4y ago
    I think the big change here is how much easier it is? Students, like everyone else, are lazy. Previously just doing the assignment was typically less work than cheating, and Copilot makes that much less common.
    • niek_pas 4y ago
      > Previously just doing the assignment was typically less work than cheating I don't think this is generally true, especially for algorithms and smaller programs: Googling 'fibonacci python' takes 3 seconds.
    • moffkalast 4y ago
      Type the assignment into google and press enter vs type it as a comment and press tab? Seems roughly the same amount of effort, and since Copilot is not free anymore most students won't bother paying for access on any kind of scale that matters. Whoever was going to cheat will cheat regardless. Besides it's one of those "you won't have a calculator with you everyday" fallacies. If you can solve problems with Copilot in class you can also do it at your job later on.
      • sliken 4y ago
        Not really, for small snippets, sure. But jobs require getting something done. Sure you might be able to tab complete 10 pieces, but at some point you have to tie them together to get a working program and those 10 pieces will have different assumptions and require some real understanding to integrate.
        • 4y ago
• deleted 4y ago
  [deleted]
• muzani 4y ago
  This reminds me of when my school teacher insisted that real engineers didn't use calculators. They'd calculate their sines and cosines from a table. 3 years later, the education ministry decided that was BS and let everyone bring calculators to math exams. Surprisingly, math scores didn't go up much.
  • eru 4y ago
    We never did much calculation in math exams. That's what physics class was for.
    • muzani 4y ago
      It was 8th or 9th grade for me when the switch happened. We were still doing areas and perimeters. A calculator actually helped me understand physics a lot better, especially things like slopes.
      • eru 4y ago
        Makes sense!
  • TakeBlaster16 4y ago
    I had to deal with this too. "You're not going to go through life with a calculator in your pocket!"
    • Invictus0 4y ago
      I've seen people take this to the extreme and literally be unable to do basic fractions and percentages either in their head or on paper.
  • lancebeet 4y ago
    I wasn't allowed to have a calculator until I was around 10 or 12 years old. I didn't like it at the time but now I'm quite happy about it.
• spcebar 4y ago
  Cheating is getting easier by the day, but the bottom line is that if the student is cheating, they're probably not absorbing the coursework (or if they are cheating and absorbing the coursework, there might be something wrong with the coursework). The most effective policy might not be to try and mechanically prevent cheating, but to explain that, this is information that you're going to need if you actually want to go into the field and if you're using copilot here you are damaging yourself in the long run. Maybe it's not fair to give as good grades to students who are cheating on assignments as those who are actually doing the assignments, but at the end of the day is the purpose of the course to measure the student or to teach the student?
  • GrumpyYoungMan 4y ago
    > "...this is information that you're going to need if you actually want to go into the field..." That has never, ever stopped anyone motivated to cheat. Even here on highly-educated HN, you get posters with open disdain for what they learned in college. > "... at the end of the day is the purpose of the course to measure the student or to teach the student?" Both. Otherwise, the college diploma really does become the meaningless scrap of sheepskin that its detractors claim it is.
  • zeta0134 4y ago
    There's more to this: it used to be that a college degree carried with it something of a soft guarantee that the student could achieve a baseline level of work on their own. This acted as a useful filter for employers. If cheating runs rampant that filter becomes meaningless, and once the student has successfully cheated their way into the work force, their incompetence can (depending on field) do very real damage to the lives of others. Because of this, there will always be considerable pressure to detect cheating and remove offending parties from the program, be that the students copying answers from the internet, or the institutions failing to detect the problem before handing out a degree.
    • buscoquadnary 4y ago
      A college degree used to signal competence. Now it signals you have the ability to show up and avoid doing something to get kicked out for 4 years. I feel we'd probably get the same value to society if we just made grades 9-12 optional. It would remove the people that don't want to be there, improve the quality of instruction, and college would actually be a meaningful achievement so we no longer have requirements for entry level jobs to require a Master's degree.
• i_am_toaster 4y ago
  “Coping with google… students can just search for the answers by typing in ‘depth first search code python’. Buckle up and hunker down for one slippery slope of an article!”
• chiefalchemist 4y ago
  The good students will take the time to learn and understand (even if they didn't write the code). Other students, as they always have, will look for short cuts. Not everyone who take CS1* courses are CS (or similar eng) majors. We keep crying "Everyone should learn to code" and now that's made easier than ever some of continue to complain. Huh? Full disclosure: I started to use CoPilot a couple of months ago. It does not remove me from the process. It does allow me to shift more brain cycles from the mundane to the more difficult. My general feeling is, I have yet to get close to taking full advantage of what it can offer me.
• refulgentis 4y ago
  I love the choice of an AI-generated photo for the header, and the hidden message of its poor quality if you look at it longer than a glance. There's an emerging divide between those with experience with AI tools and people who tried them a few times and got hyped and blogged a maximalist view of consequences.
• driverdan 4y ago
  The solution doesn't seem that hard to me. Put less weight on homework grades and more weight on projects and tests. If they cheat the homework they'll bomb on tasks that can't be solved with Copilot.
  • emeryberger 4y ago
    Yes, I mention this in the original article ("Well, how about we just weigh grades on exams more, and have students take their tests either using pen and paper or locked-down computers?").
• throwaway4good 4y ago
  What is this? Computer science Luddites?! Just pose better questions and more open ended assignments.
• tazjin 4y ago
  These examples of producing known algorithms in response to function signatures make it seem to me like Copilot is an intelligent, vendoring "package manager", not something that solves problems by itself. The perceived productivity boost that people get from it could be interpreted as a condemnation of how verbose a lot of coding still is even in known areas. Lots of brain cycles wasted on doing things suboptimally that we can already do optimally.
  • vbezhenar 4y ago
    The difference is that package management hides packages somewhere deep and you're not supposed to care about code quality. If it popular and works, good enough. Copilot puts terrible code snippets right before your eyes and you'll spend more time making it better and fixing bugs rather than writing that code from scratch. I tried copilot and for me it's not ready yet. I guess my issue is that I don't like average code and copilot was trained on that code, so it's as good as average coder and that bar is not high. Or I'm too picky and high-minded, doesn't matter, the result is the same.
  • deleted 4y ago
    [deleted]
• adrian17 4y ago
  Extra nitpick regarding generated code: the `quicksort_random_pivot` implementation isn't what people commonly mean by "quicksort", as in: it's not in-place. This means that some properties like space complexity are also going to be different than what's usually expected of quicksort.
• Jamie9912 4y ago
  Had to disable Javascript on your site because of the awful hijacked scroll smoothing
• meltyness 4y ago
  My proudest moment was in an embedded systems course, the teacher asked for a function to generate fibonacci linearly in assembly, there was a reaalllllly long pause, and I barked the answer across the classroom. I vividly remember some stunned, disgusted glances.
• hospadar 4y ago
  I've always thought that the ability to communicate to other humans about programs and to be able to synthesize a conceptual understanding from looking at code are highly valuable and sometimes-overlooked in the workplace. Assignments to write a function don't really demonstrate either skill. Maybe more assignments like: - "Here is a function that is supposed to generate a fibonacci sequence, but it is not correct, explain why" - Here is a sort function. What is the name of the algorithm implemented by this sort function? The reality of any career in CS is going to involve a lot of maintenance/understanding of existing code any systems. Copilot's failure to replace all human programmers isn't because of it's inability to cough up complex code, it's because it can't communicate with the product owner, debug a program amidst a time-sensitive incident, or explain its work.
  • dwohnitmok 4y ago
    Those two questions are definitely within the reach of current AI. Not within 100%, but probably better than 70%.
    • emeryberger 4y ago
      Agreed. I just tried this by writing three sort functions named "some_sort", "another_sort", and "yet_another_sort". I started comments after each with "# The algorithm used in some_sort is known as: " and Copilot correctly completed them in all three cases (bubble, insertion, and merge).
• norwalkbear 4y ago
  So algos take a back seat to imperative code
• Lev1a 4y ago
  In computer class back in 9th grade there were one or two exams about Excel, where we were to use excel to compute some of the answers for questions. The fact that "memorization questions" (if that's the right English term?) about several functions and concepts were on the same exam sheet and that the Excel help function was right there was something that seemingly slipped the teacher's mind. Although he was an older physics teacher who only really did computer class ("Informatik") because there was nobody else available or (more) qualified to do the job. When later on there were classes on programming, he made us use TurboPascal instead of something less cluttered and actually useful in modern economy like Python. That same teacher later on performed some show experiments about induction and electromagnetic forces in physics class where he let a magnet fall through a glass tube with several copper coils around it. So far so good, but he then tried to readjust one of the coils one the glass tube, which predictably shattered in his hands, nearly severing some fingers and actually severing some tendons, which IIRC were later replaced with other tendons from his legs or something. Needless to say, not the sharpest tool in the shed. My other physics teacher during 10th-12th grade physics classes was a severely esoteric nutjob, preferring to hold long rants about "science" being able to diagnose diseases in people by somehow shining a laser at a drop of their blood and interpreting the reflection/refraction because there was a "connection" between the person and their blood (or something like that, I honestly tuned out his ravings after a while). Apparently his wife is/was some kind of homeopathic "healer". All that is to say, computer and physics class teachers for me were "very fun" and "useful", since one did not have a clue that their students were "cheating" and both were not very in tune with modern and/or scientific methods.
• stew-j 4y ago
  Guess how I learned query/replace in Emacs in 1985? My friend who copied my Pascal programming assignment is now a director at a $1B hedge fund. I'm the founder of a series of scrappy, ramen noodle startups. I guess it's the "Senator Blutarsky" effect: https://en.wikipedia.org/wiki/John_%22Bluto%22_Blutarsky https://en.wikipedia.org/wiki/John_%22Bluto%22_Blutarsky
  • SoftTalker 4y ago
    The skill of getting others to do work for you is highly valued in the Director and C-suites.
    • stew-j 4y ago
      I think my friend might have known that in 1985. I just figured it out this morning, and by now all the good hedge fund manager jobs are taken.
• Spivak 4y ago
  I love their take at the end which is to not expand the already crazy anti-cheating apparatus and just give students more interesting problems and let them use all the information at their disposal. I wish more professors took this attitude. Having every student implement the exact same famous algorithms is really silly when you take a step back. You don’t gain any appreciation for them because they’re unmotivated and you lack all the foundational knowledge. The proofs/explanations are all totally bespoke and don’t generalize. You really win as a professor when you design a project that naturally causes you to reach for the knowledge you’re trying to teach. One of the best was my networking class where the project had us secretly rediscover tcp by trying to invent reliable transmission over UDP and then once we sufficiently cut our teeth the second half of the course was implementing a real working version on top of the professor provided userspace ip stack.
• Aeolun 4y ago
  This is no different from finding the solutions on wikipedia, or any of a thousand other websites. People that want to cheat will do so, and they might even be correct in a completely selfish sense.
• wiseowise 4y ago
  Are comments supposed to be funny?
• shakow 4y ago
  Could you just use another teaching language? Does copilot work as well with e.g. Racket, Raku or Pascal, which all seem like decent CS101 languages to me?
  • dragonwriter 4y ago
    If it doesn't now, it will quickly if that solution is adopted.
  • SoftTalker 4y ago
    Students don't like learning languages that they are not going to use professionally. They want to learn Python, Javascript, maybe Java or C#. This is another clear indicator that most students today see college as a higher class trade school, not as a place of higher learning.
    • hirihoto 4y ago
      Difficult to fault students for not wanting to be able to pay off the massive debt they're forced to buy into just to get entry level career jobs. College may have been a place of higher learning at some time before the last thirty years but I doubt it. Javascript may be the only one of those four worth teaching as it has different patterns and different ways of solving the same problem.
  • infinityio 4y ago
    To quote the article: > Here’s an approach that’ll work for sure: use some, let’s call them alternative, programming languages that Copilot doesn’t really know. (...) Sadly, I have news: Copilot’s love for programming languages knows no bounds! Racket! Haskell! ML! (...) Copilot is a ravenous beast: if any code in any language found its way into a GitHub repo, it’s already swallowed it up and is hungry for more, nom nom nom. Not sure how true this is in practice - I only used Copilot once, with Python - but if it is that might invalidate this concept
  • emeryberger 4y ago
    Hi all - article author here. A reader of my article sent me this note, pointing me to some relevant hot-off-the-presses work (reproduced with permission): ----- A group recently evaluated the performance of Codex (and another model) on 18 programming languages: https://nuprl.github.io/MultiPL-E/ https://nuprl.github.io/MultiPL-E/ The high-order bits are: 1. The latest and greatest Codex is now twice as good on its own benchmark suite than the original version published a year ago. 2. It's just as good on Python as it is on JS, Scala, C++, Swift, TypeScript... and other languages are not too far behind. It's not bad at bash of all things. Paper here: https://arxiv.org/abs/2208.08227 https://arxiv.org/abs/2208.08227
• BiteCode_dev 4y ago
  While it is true code pilot is quite amazing and can produce some very decent snippets, if the exercice you give is specific enought (using a special database schema, file format, or API), with objectives that are not common (ask the user specific questions, and draw conclusions out of it or react to events), then it cannot produce a fully working program. I use code pilot regularly now, and it does speed up my work, but I can't do anything with it without understanding the problem at hand. There is nothing wrong with the student assemblings and fixing parts of code that exist. Doing that requires understanding those parts, how they interacts and what you can do with them. It's not killing tests. Plus as professionals, we all do that anyway. Heck, I learned doing that. I'm paid doing that. However, there is something wrong with academic exercices that I see in the wild. Things that are very abstracts, requiring way more than programming knowledge, or stuff that are asked out of context completely. Students are both understimulated, and given too hard bites to chew. It's a terrible way to learn, and to be tested IMO. Tests nowaday don't tests for knowledge, they test either for compliance, or act as a filter instead of a feedback loop. This is not education. In fact, I've yet to see school IT course that I didn't find deeply flawed, and I completely understand the students for cheating at them to that they can suffer as little as possible. They are not given a chance to prove themself or progress, they are not respected. They are forced fed junk and asked to spit it back all shiny. I remember when I was a student, and I went to 11 different schools because of my twisted life. They all sucked. And I say that as I spent half my student life at the top of my class grade ladder. I regret having cheated so rarely now.
  • wobbleblob 4y ago
    Github Copilot won't give students anything they couldn't already google. It seems to be a natural part of aging, to start to complain about education and kids these days because it's not the same as when we were in school. I try not to do that, because I never forgot how my parents generation said the exact same about us when we were in school, and I guarantee my grandparents generation complained when my parents were in school.
  • NikolaNovak 4y ago
    As I moved as well, I experienced three different Canadian universities' Computer Science Program. There was only one professor and course that I still remember 20 years later: CS408, Software Engineering, Professor Wortman, UofToronto. Class project was in four phases, cumulative (basic functionality for the application in phase 1, progressively additional functionality in other phases, frequently strongly interacting with previous phase code). Here's the kicker: After each phase, you had to swap your code with another team. So you had to pick up somebody else's code, figure it out, and then build up on it. Few of us that had real-world working experience loved the course and flourished in it. This is what we are training for! This is what programming is like! You are taking real code and building a real thing with it! About 250 other students signed a petition to the Dean on how this is unfair and awful and they will not put up with it. They were just too used to / spoiled by 16 years of 5 assignments per semester with abstract, entirely separate questions of 1.a), 1.b), 1.c), etc. All I could think of - if you did not like this course, you are about to not enjoy next 4-5 decades of your life :D Other than this one course, I can say that I'm a prolific, enthusiastic, life-long learner, and my university experience was the absolute dumps - it was far less about learning, and far more about bureaucratic hoops and artificial constraints and restrictions. I was top of the class in some hard courses (generative compilers etc), mid-pack in the some of meh courses, but in retrospective, my life opened up when I was done with academia and could work and learn in the 'real world'.
• brundolf 4y ago
  Eh, it's always been super easy to cheat on programming assignments. Often they're variants on toy problems that have been used in the past, or worst-case, you could crowdsource an answer on stackoverflow or Reddit Just make sure Copilot is explicitly against the rules, there's not much more you can do than that
• twodave 4y ago
  Programming and, even more so, software engineering are _trade skills_. There will ALWAYS be “cheaters” but you will know them by the end product of their work. Yes, it’s important to understand the fundamentals. An unwillingness to learn these things is a great indicator that you will be a failure as a professional. As another commenter mentioned, the focus on grades at all makes the problem worse. More important than even fundamentals, though, is modeling a successful software development mindset and problem solving technique. Too many professors have not _had_ any success in software development, which is sad. I can count dozens of examples of my own personal growth that could have been handled in college, that I instead ended up inflicting on my first employer.
• lfkdev 4y ago
  With that logic using a calculator is also cheating.
  • tgv 4y ago
    On an arithmetic test, yes, it is.
• t_mann 4y ago
  > we teach in programming languages that don’t even exist I remember reading about a framework that actually sort of does that - create randomized programming languages. It was meant as a security feature against code injections iirc, but technically it should work as a solution to OP's problem. Although I guess some of the other suggestions are better and easier to implement - ask students to explain their code (select one person randomly per problem, it's quite common anyway) and give more weight to tests.
  • O__________O 4y ago
    Only matter of time before CoPilot like systems (if they do not already) just reverse engineer the compilers to produce and link a dynamic syntax to know instances in plain-text source code of the “foreign” programming languages and render them within a language the user already understands; yes, there might be gaps in idioms, assuming a natively written language, but using example you provided, there in theory would not; ...and if need, return a final version in the target “foreign” programming language.
    • int_19h 4y ago
      GPT-3 can already "interpret" Python pseudocode.
    • t_mann 4y ago
      such systems might spring up, but the market is far more limited than for generalist CoPilot-like systems, and they will thus likely be less sophisticated and thus easier spot (by other automated solutions)
      • O__________O 4y ago
        Might? This is the future of systems like CoPilot, reading existing source code only goes so far and it is only matter of time before the state of the art systems dynamically map compilers, even foreign ones, to source code. Once mapped, system will use its universal transcoder to remap the code to a know target language and even dynamically rewrite the compiler of the foreign language if appropriate to fill idiomatic gaps. Ultimate end user for systems like these are not humans.
• mediumcat 4y ago
  Copilot still works quite poorly with problems that are not “Stackoverflowable”. Perhaps the best way to fight this is to create assignments that cannot be googled in 5 seconds.
• charcircuit 4y ago
  Copilot is limited to short snippets. You can't just ask it to write a HTTP server. Also I don't see how having copilot generate the code for listening on a server socket is any better than when I copied how to do it out of a man page. Copilot doesn't add anything new, you've always been able to look up the documentation or look up how other people approach the problem. The fibonacci example doesn't make much sense either due to how trivial it is. Without copilot you are just copying from the definition of the function and with it copilot copies it for you. The other algorithms that follow are both online and probably in their textbook.
  • johndough 4y ago
    Copilot can almost write an HTTP server. Here is how I did it: 1. Start a new python file with the line "# This is an implementation of an HTTP server" 2. Press autocomplete a bunch of times and stop somewhere when "enough" libraries are imported 3. Press "#" and let it autocomplete the comment about what comes next (global variables for PORT and BUFFER_SIZE in my case) 4. Again, Press "#" and let it autocomplete a bunch of comments and functions ("get_file_extension", "get_file_content_type", "get_file_size" in my case) 5. At this step, I had to cheat a bit since it was generating too many "get_*" functions, so I started a comment with "# Initialize" and let it autocomplete again to get "init_server_socket" 6. From here on, it generated handle_request, parse_request, get_file_path and send_response automatically. 7. Lastly, I wrote "def main" and let it autocomplete again. This produced an HTTP server which almost worked. I just had to fix a small issue with "file_path", since it expected that files were stored in the root directory, but I wanted it to load files from the local directory. The code is not great since it does not handle most errors gracefully and is vulnerable to directory traversal, but I didn't even have to think about the HTTP protocol while writing it, so it is still quite impressive. A few good points: It generated a huge selection of MIME types which I probably would have had to look up by hand. The generated server is multi-threaded! It automatically reuses the socket, so I do not have to wait a minute every time I restart the server. Here is the final code: https://bpa.st/CTRA https://bpa.st/CTRA
• criloz2 4y ago
  I have been using copilot for two months and I find it more annoying than useful, am the only one that feel like this?, I also used tabnine, and it gives a lot of syntax errors in rust, so I could barely use it for a week. I definitely can agree that copilot is better than tabnine, but I feel like writing code is the easy part of programming, and copilot doesn't offer a crazy help to justify the monthly payment and that my private code is uploaded to Microsoft servers. Not only that, but I find that my IDE suggestion are good enough most of the time.
  • nprateem 4y ago
    Yeah I was going to post this. I did some coding the other day and it kept spitting out code that looked right but was calling functions that didn't exist. It also competes with IntelliJ's auto-complete making things doubly irritating. I leave it disabled more often than it's enabled, unless I'm doing something simple like wanting to copy a file, which in Go is stupidly obtuse.
  • thefourthchime 4y ago
    I use it with python and vscode, it's great there. Sure, half the stuff it makes isn't correct, or nearly correct, but it's very close, and I just need to change a little here or there. It also does a fantastic job of filling in method parameters from my local variables, again not 100% of the time, but 90% of the time it nails it.
  • deleted 4y ago
    [deleted]
  • stew-j 4y ago
    I'm not sure how it would be done, there might not even be enough existing code to train it on, but I would like to try out Copilot for code-based CAD models in something like OpenSCAD. I've never tried it at all, but thanks for sharing your experience, all.
• joak 4y ago
  Grading students is probably an archaic way of teaching. You want them engaged not cheating. But ok, institutions want grades... What about this solution?: https://theconversation.com/i-no-longer-grade-my-students-work-and-i-wish-i-had-stopped-sooner-179617 https://theconversation.com/i-no-longer-grade-my-students-wo...
  • joak 4y ago
    Previously discussed here: https://news.ycombinator.com/item?id=30842587 https://news.ycombinator.com/item?id=30842587
• cmrdporcupine 4y ago
  I understand the problem here and I sympathize with professors in these circumstances. Learning the fundamentals will remain important. But as others have pointed out: if students are unwilling to learn, or are taking shortcuts, it will ultimately hurt them in the long run. One thing tho is that things like Copilot put the lie to the hypothesis (propagated mostly by the Google-style job interview) that intensely coding clever for-loops to perform algorithmic magic (for things usually already in the standard library) is the best measure of the competence of a software developer. If it can literally be done by a machine, maybe we should be measuring based on something else. Especially since this kind of thing is best done on the job by looking in a good textbook or using the standard library. Probably grading at the university level needs to consider this also. I haven't used Copilot. I doubt I'm its intended audience. After 30 years, actually writing code is perhaps the easiest part of my job. The mechanics is the easy part. The big picture thinking and figuring how to get it all together into a system is the hard part (and honestly there are plenty of people far better at it than I am). Now, if we get a Copilot for that ... then our profession is in trouble.
  • eloff 4y ago
    I was skeptical of copilot, but since trying it for two months, I'm really impressed. It's a smarter auto complete. For $10/month it pays for itself in one day. You have to be careful with the output it produces, as it often looks sensible but is wrong. Especially if it contains numeric constants. But on the whole it speeds things up, the same way good code auto complete does.
  • thefourthchime 4y ago
    I've been writing code for nearly as long and man, you have to try it. It's more like autocomplete on roids than something that generates Fibonacci methods automagically.
    • ALittleLight 4y ago
      I've used Copilot for a little while. I'm away from my computer at the moment but I'm very confident it would generate Fibonacci methods.
• O__________O 4y ago
  Curious, has anyone has seen a “cheaters” guide to programming — that basically teaches a predictable barebones process that assumes unmoderated access to internet, computer, etc - and using Google, StackOverflow, CoPilot, etc. to solve 80% of programming issues average programmer faces?
• T3RMINATED 4y ago
• pydry 4y ago
  A professor could put up an almost correct but slightly wrong answer on github or even a right answer with a copyright trap. So it might be more like bringing a tennis racket to a grenade fight.
• yosito 4y ago
  As someone who's used machine translation as a tool to learn 2 human languages, which I eventually gained fluency in, I'm really tired of AI being considered "cheating". Yes, you can use it to approximate answers to questions and be fairly certain those answers are correct. But you can also use it as a tool in your learning process to help you fill in the gaps until you're all the way there. In language, it's quite obvious that using the language will increase your abilities over time. In programming, that may not be so intuitive, but it's also true. The more software you build, the more you will understand about building software well, even if you use Copilot to help write your algorithms. I think the "problem" here is that tools like this make it difficult to enforce that others learn something. But for self-directed people, who are committed to learning and understanding what they are doing, AI is more of a useful tool than a cheat.
• superb-owl 4y ago
  While this will definitely be an issue in the short term, it's part of a long standing series of panics where a new technology develops, and everyone worries that we'll lose the skills it abstracts away. And to some degree we will! I certainly can't do long division anymore, and I struggle to navigate my own neighborhood without GPS. This is a bit sad, but I think something of an inevitability. If history is any guide, academics should embrace the new tech and try to teach students bigger things that copilot can't yet handle, with the assumption that they'll use copilot to fill in some details--the same way we now teach students to use graphing calculators to solve problems that would have been out of reach for their grandparents. There are so many things I never learned as an undergrad because we were too focused on algorithms. Database design, testing strategies, different architectures, programming language design, etc. Maybe these things can come to the forefront of the curriculum if we don't need to drill basic algorithms.
  • svnpenn 4y ago
    > I certainly can't do long division anymore, and I struggle to navigate my own neighborhood without GPS These are not the same thing at all. Yes calculators and such have largely made long division pointless, but it's still quite useful being able to get around without GPS. especially around construction, GPS is awful. You need to know the limits of the tools you're using. If you just say "I'm always going to use GPS, because it's always better" you are really doing yourself a disservice.
• social_quotient 4y ago
  How does medical school, law school, and non-CS engineering resolve this? Seems like they would have had to solve the “how do I know what you know you know issue”.
  • tzs 4y ago
    In law school there generally isn't any homework in classes about the law. The only classes with homework will usually be classes in legal research or in classes that cover non-legal subjects from a legal perspective. As an example of the later, at my school there was a "Quantitative Methods in Law" class that basically was an introductory statistics class focused on applications involving law. For the classes that are about the law (contracts, torts, criminal procedure, etc) it pretty much all comes down to exams. I only had one take-home exam [1]. All the others were in-class. The exam questions were mostly essay questions. There would be a couple paragraphs or so describing some situation, and you would be asked what the legal outcome should be. Your essay needed to identify the legal issues involved, cite the relevant cases and/or statutes, and argue how those applied to the facts of the case to support the result you were trying to argue. [1] Note: I am not a lawyer. I went to law school when I got burned out with programming. By the end of law school I was no longer burned out, and decided I'd rather be a programmer who knows a good bit about law than a lawyer who knows a good bit about programming.
• brimstedt 4y ago
  I don't think using copilot is much worse than autocomplete. It's an improvement of our toolbox, students SHOULD learn to use it. It's not much worse than using c instead of assembler, or java instead of c, or ... Throughout time people have invented new tools that makes their predecessors more efficient in their proffesion. Too many programmers these days behaves like the workers did when automatic looms were invented. Or when the chainsaw was invented.. or the computers.. or the...
  • tgv 4y ago
    > Throughout time people have invented new tools that makes their predecessors more efficient in their proffesion. And yet, here we are, learning reading and writing like it's 500BC.
• caust1c 4y ago
  Maybe AI will know what to do with our burning planet...
• highwaylights 4y ago
  If what you’re asking of these students can easily be automated by a tool then the problem is in the assignment. Either don’t request it, knowing it can be automated with ease, or accept that bubble sorts are a long solved problem and move on to teaching other things. Lest anyone think I’m saying don’t teach important concepts or that such things no longer matter: there’s a whole world of principles and fundamentals that can be taught and assessed without fizzbuzzing 100 students who already had access to Google and StackOverflow before Copilot came along. Yes, directly assessing hands-on coding would be much better, but these tiny snippets aren’t really indicative of real-world tasks that will be needed anyway (given that almost anything so foundational will almost certainly be in a standard library), it doesn’t really prepare students for real-world development work, and there’s a general expectation that new graduates are going to be very very green and therefore need a lot of handholding regardless. Even more than that, you’ve no way of detecting plagiarism already in these assignments given that a couple of basic rename refactors is going to stop you from proving the code was lifted from elsewhere to begin with.
  • Merad 4y ago
    > If what you’re asking of these students can easily be automated by a tool then the problem is in the assignment. Either don’t request it, knowing it can be automated with ease, or accept that bubble sorts are a long solved problem and move on to teaching other things. If assignments that are easily automated are a waste of time, do you have any suggestions on how instructors should help students move from "I know literally nothing about writing code" to "I know the basics well enough to solve problems that aren't trivially generated by Copilot"? In my anecdotal experience (albeit years ago) as a graduate assistant, for students who are totally new to programming that phase often lasts throughout the typical intro to CS course and it's not unusual for it to stretch into the second level course.
• deleted 4y ago
  [deleted]
• captainmuon 4y ago
  I don't know, I tutored an introductory course on data science in Python last semester, and yeah copilot would have been able to answer most of the problems. But students would have been able to copypaste them from somewhere else anyway. If you want to test them, have them explain what they did, and why they wrote it this way and not another way. But personally, I believe that people are at the university to learn learning, and to learn independence, and if they really want to cheat then it is their fault. You spend a lot of time (and in some places money) to be here, so you might as well be honest and learn. The degree itself is not as helpful as one might think. It is the stuff you learn for the degree that helps you later on. But on the other hand, I found copilot useful as a tutor. When somebody asked about an API, I didn't have to akwardly search the documentation, I fired up VS code and have copilot suggest something (without the students seeing it of course ;-)).
• analog31 4y ago
  I wonder if copilot could be used against itself, i.e., to find problems that it is absolutely guaranteed to get wrong.
• renewiltord 4y ago
  Yeah, I totally get it. The purpose of education is credentialing and so cheating undermines that. If the purpose of education were solely instruction you would advise students not to use Copilot to finish assignments but those who do would only harm themselves.
• unnouinceput 4y ago
  Quote: "Oh, have I mentioned that Copilot is free for students? Yep, COPILOT IS FREE FOR STUDENTS. It integrates helpfully right in their favorite IDEs." Does it integrate with Lazarus?
• deleted 4y ago
  [deleted]
• WoodenChair 4y ago
  I wrote an essay “Introductory Programming Assessment Must Accommodate Copilot-like Assistants” about this same topic last month and came to the option 3 conclusion (use more exams or in-class live coding assignments): http://www.observationalhazard.com/2022/07/introductory-programming-assessment.html?m=1 http://www.observationalhazard.com/2022/07/introductory-prog...
• muhehe 4y ago
  I've read the title as "copilot with copilot" and thought that copilot leaked some parts of itself :)
• MarcScott 4y ago
  And old colleague of mine used to teach CS, when students had to send off code they'd written to Cambridge University for it to be compiled and run, and the results were then returned, all by post. Then you were able to compile and run code locally on your machine. Then the IDEs came along, with syntax highlighting and then linting. Copilot is just the next evolution in writing code, and saving you the effort. I bet they railed against Timsort and it's inclusion in the Python standard library, but now who honestly writes their own sorting algorithms? This kind of generational gap reminds me so much of the "Kids are spending too much time reading" > "Kids are spending too much time watching TV" > "Kids are spending too much time on their phones" rants.
• yummybear 4y ago
  Doing algorithms and functions are the least of my worries as a web developer. I need a copilot that helps me determine which kind of ui elements i need to help with task. I need it to automate standard issues like make login functionality invludkng two factor or to determine data structures for a novel problem. That’d be great.
• rwky 4y ago
  When I did my BSc in computer science we had to do exams which involved writing code using pen and paper without a computer in sight. Bringing that back would solve this problem ;)
• somenameforme 4y ago
  Do people no longer generally have exams on paper? For lower level courses we had online assignments, and pen and paper exams - which is where the majority of the course grade came from. If you didn't [intimately] know what you were doing on the assignments, you'd fail the classes due to the tests.
• youssefabdelm 4y ago
  Ever heard of just-in-time learning? Copilot (if it gets good enough) is the damn teaching tool! All they need to do is come up with a programming goal/project thats extremely important/relevant/exciting to them, and break the problem down into sub-problems for copilot. This time they'll actually give a shit to understand why copilot wrote a bug or why the code works. No more boring pointless teacher assignments that hold no relevance whatsoever for the students most immediate and important goals and concerns in life. Finally life is fucking good for students for once and they dont need to google every answer for an hour.
• marmada 4y ago
  I think Copilot will make cheating easier. A benefit of this is that employers may look at side projects much more, which are harder to cheat
• coding123 4y ago
  No big deal. If students have access to this, then instead of trying to teach them and grade them on fibonacci - let the next generation have larger assignments. Like building a 3d rendered first person shooter (a basic one with no AI)
• copperx 4y ago
  GitHub/Microsoft know what's up, because they make Copilot free for students, but teachers have to pay for it. The best thing for Copilot is to keep teachers as oblivious as possible.
• dvh1990 4y ago
  It is not the lecturers job to police their students. People that cheat only end up cheating themselves, and nobody but the cheater should do anything about it.
  • sideshowb 4y ago
    Wrong. They undermine the reputation of the qualification their honest course mates are working for.
• emeryberger 4y ago
  Hi everyone, article author here. I was also interviewed by The Register about this topic: https://www.theregister.com/2022/08/19/copilot_github_students/ https://www.theregister.com/2022/08/19/copilot_github_studen....
  • stew-j 4y ago
    I might have missed it today in your articles or comments here--it's been a hectic day--but has there been some study of just how different code would be given that the students are using the same text from questions? Is there randomization intrinsic to Copilot, or is it just because minor variations in textual input causes code to be so different? My wife taught CS, she did catch cheaters pre-Copilot, and my first thought it that she probably would enter test questions and print out a reference sheet for Copilot generated results.
    • emeryberger 4y ago
      I haven't seen a study yet, but yes, Copilot deliberately (AFAICT) incorporates randomness. For longer code fragments, I believe this would thwart most plagiarism detectors. I'll note that defeating plagiarism detectors is easier than many people think. I and one of my students wrote a paper on an automatic technique to defeat plagiarism detectors, and it was highly effective: "Mossad: Defeating Software Plagiarism Detection", Devore-McDonald and Berger, OOPSLA 2020 (talk and paper link here: https://2020.splashcon.org/details/splash-2020-oopsla/14/Mossad-Defeating-Software-Plagiarism-Detection https://2020.splashcon.org/details/splash-2020-oopsla/14/Mos...).
      • stew-j 4y ago
        From the OOPSLA paper abstract: > techniques inspired by genetic programming with domain-specific knowledge to effectively undermine plagiarism detectors. My creaky memory of CS Theory makes me suspect that, ultimately, Mr. Turing's halting problem may make resistance to cheating futile. [1] It's interesting that the paper is in OOPSLA, I used to follow the work from it more, especially when I listened to IEEE's Software Engineering Radio. Thanks for your work in this area and your reply. DieHard and DieHarder are funny, creative names for fault tolerant memory managers. It's also nice to see a fellow humorist. At least we can rest assured that cheaters get what they deserve, like my classmate at Columbia. [2] Regards. [1] > He, for one, welcomes our new AI overlords. Yes, let's suggest a Borg-like partnership with them. I'd be willing to drive "self-driving-car" tow truck--if they haven't invented a self-driving one yet. [2] https://news.ycombinator.com/item?id=32532896 https://news.ycombinator.com/item?id=32532896
• hbn 4y ago
  I don't know if the idea of future generations of programmers squeezing by on having a robot make best guesses to get their work done without understanding it makes me feel confident in terms of job security, or fills me with dread knowing that I'll surely be relying on the software they produce to access my bank, have my cars controls hijacked because it thinks it knows better than me, keep my life support running when I'm old, etc.
• debbiedowner 4y ago
  Just don't grade HW and have handwritten quizzes and tests in class? (Yes, the grad students will suffer when grading... unless you pay them and then they will be lining up)
• withinboredom 4y ago
  Here’s the thing. When you learn to be a carpenter, they teach you how and when to use certain tools. The same can be for any profession. This is a new tool. They shouldn’t be telling students not to use it, but when to use it and what it is suited for.
• Brian_K_White 4y ago
  It sounds like copilot could ultimately be just another progression in the definition of normal or baseline, like using calculators in math classes. Whatever the current state of the art and tools of the trade are, that's what they are, and probably the bulk of courses will just adapt to reflect that state of reality and incorporate that into the courses. Outside of maybe some specialist courses that may still exist but most students wouldn't need to take, it would just be expected that most students use the current tools to accomplish the tasks. Even going so far as to provide free student access to any paid tools to avoid giving the rich kids an unfair advantage. The nature of posed problems, and their teaching purpose (why they are posed and what you get out of them) would just change from what they are now. Whatever the "calculator" can do, simply becomes uninteresting and not required for most people to worry about. My first couple years of school I was taking mechanical engineering, and it was right during a transition where we were doing both manual and cad drafting. There is a lot of geometry tricks both physical drawing and math required to generate accurate views of any shaped objects intersecting with any other shaped objects, viewed from any angle, sliced at any depth. We spent a lot of time and effort on that. That is all done by magic now inside the cad engine. Someone still needs to know it, but 99% of people whos job is to design or otherwise manipulate models of objects do not need to know it. I, who did learn that stuff, don't think an engineer who didn't have to know how to generate an accurate view with nothing but a pencil and a calculator is a less capable less insightful less quality engineer. If anything they may be better for not having to spend as much of their intellect on mechanics & implementation vs ideas. The knowledge & skills I'm talking about don't contribute to their understanding of the real job, they aren't foundations that the later stages build on. It's just labor of no value at all except to the cad engine designer. It had a value in the past because it was a form of literacy. It was simply the only way to document the ideas you were fabricating. Perhaps copilot is like that. I don't know and I'm not saying it is, but it could be.
• bfung 4y ago
  On tests, make students write short programs with pen and paper. Good practice for white boarding interviews.
• ctoth 4y ago
  Setting aside the academic implications for a moment, if you think that Copilot-like models won't be a significant part of programming in the future I'd like to point out this research from Google: > We compare the hybrid semantic ML code completion of 10k+ Googlers (over three months across eight programming languages) to a control group and see a 6% reduction in coding iteration time (time between builds and tests) and a 7% reduction in context switches (i.e., leaving the IDE) when exposed to single-line ML completion. These results demonstrate that the combination of ML and SEs can improve developer productivity. Currently, 3% of new code (measured in characters) is now generated from accepting ML completion suggestions. So 3% of all code at Google is now written by AI autocomplete, and Google developers who use this technology are 6% more efficient than those who don't[0]. Is it bad that students are cheating on homework with this? As bad as it was when students used Stackoverflow to cheat, I reckon. But will working with AI models be an essential skill to learn for new (and existing) programmers? About as much as getting comfortable with Stackoverflow was, I reckon. [0]: https://ai.googleblog.com/2022/07/ml-enhanced-code-completion-improves.html https://ai.googleblog.com/2022/07/ml-enhanced-code-completio...
  • beej71 4y ago
    Sign of the times, I think. How long until AI offers complete solutions to basic business websites? How long until it does even more?
  • Test0129 4y ago
    I don't mean to be pedantic here but 6% improvement in productivity is a rounding error. You can achieve a higher level of productivity by just removing the ping pong table from the break room. This would indicate to me there is a non-significant improvement in productivity. That seems consistent with my having talked to several senior+ engineers who tried copilot and found it simply wrong or unnecessary. There might be AI written code in the future and surely the "no code" shills will latch onto it with religious fervor. But, much like "no code will eat the world", this too will fail. Until general intelligence is capable of producing large scale systems that need to be coddled and orchestrated, bespoke perhaps even brand new approaches to business problems, etc, there will always be many highly paid humans there. The conspiracy theorist in me makes me think that all of this push to shill this product constantly is yet another way the FAANGS are attempting to dilute the pool enough to lower wages. Engineering is highly skilled, highly technical work. The actual work of someone higher than junior level typically evolved way beyond simple CRUD coding exercises. When you spend more time making sure copilot meets standards, the code is correct, etc it's simply not that useful. It feels much more like the fancy test case generators for some IDEs. For very trivial, borderline rudimentary cases, it works wonderfully and saves time. Once things become complicated and messy it just gets in the way.
• ___j___ 4y ago
• omginternets 4y ago
  This seems like a problem for educational institutions, not for industry. We're already inundated with mediocre programmers, many of whom have CS degrees, which is precisely why the industry looks at job experience more than credentials [0]. I don't see how this is going to further reduce the supply of competent candidates. What it will do, however, is make it difficult for CS departments to evaluate students based on homework, and it's not clear to me that this is such a bad thing. My experience with university CS courses has been that assignments are largely auto-graded, with systems akin to unit tests. In the common case, an overworked TA quickly spot-checks the source code to look for obvious signs of academic dishonesty, but that's it. So, universities are left with the following choice: give everyone an A, or grade differently. In the optimistic case, this might even produce a grading strategy that improves the value of a CS degree as a predictor of programming competence. It likely won't, in which case we have a status quo. [0] This is a problem in its own right. I recently referred a grad student as a job candidate in my current company. I'm the most senior backend engineer, and this grad student was a highly competent contributor to my open-source project, which currently plays a strategic role in our backend. Said grad student is unusually bright and productive in a complex specialty (distributed systems), and I had to push our VP Eng very hard to hire him. His initial reaction was along the lines of "he doesn't have any industry experience, so we don't know if he's good". It gives me nausea to think about how many outstanding, 2-sigma engineers are rejected because companies don't know how to evaluate their talents. In the end, we ended up hiring him, but only because I stated that I would take full responsibility for his productivity. I regret nothing. </rant>
  • armchairhacker 4y ago
    Northeastern University is a college which is extremely popular for it's co-op program. What is co-op? It's literally just working at a company for 6 months. Yet students proclaim how it teaches them so much better than classes and helps them so much to get a great job, and I mean, it does. College is the new high-school: everyone goes, only to get a job, because you have to go to college and get a CS degree to get a job. Why? A lot of "compute science" isn't used in software development at all. I think we should replace colleges with "boot camps" which are unpaid internships which teach high schoolers how to work in actual software development, in various fields (let's call them "entry level jobs which don't require 5 years of experience"). Why unpaid? Because companies will love it, and it will still be better than colleges where you pay a full year's salary in tuition. What about people who actually want to pursue higher-ed computer science? Just don't replace all colleges. It seems like most people getting into CS are not doing it for academia, those that are will have a better time taking courses with others who are passionate about the subject instead of simply using it to get a high-paying job.
• Arch-TK 4y ago
  While copilot does great on short simple fundamental problems, it sucks (and requires good attention and experience to find all the bugs in generated solutions) when solving anything more difficult. Although I can totally see how this would make teaching the basics difficult. That being said, I think Microsoft should be fined $100B every time they give one of their products away for free to students. Hopefully that will be sufficient incentive for them to stop so being so overtly anti-competitive.
• cothrow_away 4y ago
  Euclid said that there is no royal road to geometry. People who pick the royal road end up elsewhere. In a society that grades them and acts on these grades, it may very well end up on a king's throne. But not geometry.
• tommyage 4y ago
  Who should fix the bugs in the future? Nerds? Where are they coming from 60 years from now? E(very) S(ingle) W(ord) S(hould) be (no [ACCEPTED] completion here) W(ritten) W(ith) C(onscience). You should be responsible for your code submitted. Are your feeling confident? Won't touch Co-Pilot. Probably you are spending equal time proof reading than _understanding what needs to be written_ in advance. At least that is what I am hoping about. Of course you don't have to proof read, and most things can probably be grasp in a glance. But you rely on a black-box. From a corporation. To me that sounds like agreeing with a dictator. Wouldn't promise my craftmanship for things I didn't grasp in advance. But maybe that's a culture hindrance. I like it though. I want to know the code I am corrected about. I want to archive mastery.
• pavelboyko 4y ago
  This reminds me of a story that an old professor of theoretical physics told me. In the early nineties, he left the former Soviet Union for the United States to teach physics at one of the top universities. There he encountered the fact that American students were fantastically good at solving all his standard problems for integrals. It quickly became clear that the students were using the then-new program for symbolic calculations, Mathematica. As a result, our professor also mastered Mathematica and spent half the night finding such integrals that it still could not calculate for assignments. I use Copilot every day and I can assure you it makes a lot of mistakes. I think that, at least in the short term, CS teachers will still find assignments where it makes mistakes.
• LegoZombieKing 4y ago
  Just wait until students start using GPT-3 for written essays. Not giving anyone ideas or anything, but if you feed it your previous assignments and it learns your writing style it should be hard to detect with minor editing.
• datpuz 4y ago
  Maybe I'm weird, but I do not get the hype. I was using Copilot for a few weeks as a Go backend dev with occasional work in the React/TypeScript frontend, and I thought it actually reduced my productivity. A lot of times, the suggestions were totally wrong, but looked correct-ish, and I would have to pause and basically debug that code, which generally took longer and snapped me out of my flow. Some of the autocomplete was okay in that it saved me some keystrokes at times, but in general, it felt like more of a distraction. I especially hated the suggested comments; these were almost never close to what I intended to write, and it would always give me a pause when they'd pop up, and often forget what I was about to write. There were a few instances where it magically did autocomplete a surprising bit of code, and I enjoyed the novelty when it would happen, but that wore off quick.
  • cageface 4y ago
    I eventually turned it off too. It was good at certain things like filling out repetitive test data but overall I found it more distracting than helpful.
  • LelouBil 4y ago
    You can keep it disabled and enable it with a hotkey when you think it can help. That's my strategy. You need to recognize what it's good for, like generating code similar to what you've written but a little bit different. For example : I had a project in Rust where I would print a sentence describing an error that occured. I had a big match block with my possible error variants. The error enum was declared in the same file. And so, when I added a new error variant, I just had to place my cursor at the end of the match and watch copilot generate exactly or almost what I needed, a similar block of code that the other cases, but with a different message and using the fields of this varient intelligently.
  • zanek 4y ago
    I agree, I just disabled it today. I'd say 3 out of 10 times it produced a good result. Most of the time the structure looked ok but the variable names were wrong, or the types were wrong, or it had logic issues. Surprisingly, I thought it did great with autocompleting the comments. In Visual Studio, it would block function autocomplete which was really annoying also
• eh9 4y ago
  I love copilot when it gets it right, but I’ve stopped using it because it was missing the slightly obvious suggestions my type system was providing.
• ivraatiems 4y ago
  Here's a thought: Replace mandatory homework with proctored, time-limited exams on systems that don't have copilot available (or dramatically reduce homework's percentage of one's grade). If one cares about cheaters (and I'm not convinced we should), doesn't that solve the problem?
  • emeryberger 4y ago
    Yes, I mention this in the article as one of the solutions: "Well, how about we just weigh grades on exams more, and have students take their tests either using pen and paper or locked-down computers?"
• s3p 4y ago
  This article is giving me a headache. I understand the narrator's frustration but wonder why they have to write in such an animated and confusing way. Seems like they think they're making a script for a tiktok instead of an article.
• cosmojg 4y ago
  If the problems you assign can be solved by a total novice armed with Copilot, you probably aren't teaching anything worth paying for.
• acbart 4y ago
  I have a hard time being interested in anything that Emery Berger has to say about Computer Science Education, since he's doing nothing to support the field. https://github.com/emeryberger/CSrankings/issues/11 https://github.com/emeryberger/CSrankings/issues/11
• gfiorav 4y ago
  I agree with the point that there's no point in fighting the advent of AI-aided development, but it's also true that the invention of calculators didn't mean we stopped learning how to add.
• ridiculous_fish 4y ago
  One of the best assignments I had in CS was "here is a compiler, there are nine known bugs in it, find and fix six of them." I don't see how Copilot could help in that case.
  • omnicognate 4y ago
    By giving you tons of practice at spotting bugs in code you didn't write, I suppose :-D
• runald 4y ago
  Ask canned questions, expect canned answers as well. Dumb or lazy incompetent students will look up solutions to problems like fibonacci sequence or sum of numbers without understanding them or solving them first on their own. This has always been the case, even before copilot came along. Copilot only made it even easier and accessible to even dumber students, and made it harder for dumb teachers to lazily give canned programming questions. I haven't used copilot yet, so I don't know how intelligent it is at producing solutions that it hasn't seen before. Assuming it's not that intelligent, one solution would be to just add small, tricky variations or constraints to well-known programming problems. So instead of just asking "find the sum of numbers", ask "find the sum of foo numbers", where foo is some made-up property, like a number contains even number of odd digits. Then require the implementation to count down instead of counting up.
• de6u99er 4y ago
  >Oh, have I mentioned that Copilot is free for students? Yep, COPILOT IS FREE FOR STUDENTS. I guess Microsoft will know which students did their assignments on their own when they apply for a job at Microsoft.
• jeffbr13 4y ago
  I’m not sure I have much sympathy for the problem. Copilot is going to force everyone to move up the abstraction stack. I absolutely disagree with how Copilot has been trained on the intellectual property of others, and I think Microsoft/GitHub should be taken to court over it (and I say that as an IP cynic, but if you play that game, you should stick by the rules), but this technological cat is out of the bag. Students (and everyone else) should be thinking much harder about how they test and verify the behaviour of any given piece of code and how they design systems to stick together. This class of technology is going to automate away many of the jobs where you just pump out code. This is fine. It’s no different to someone inventing a burger-flipping machine, and I’m sure many here would agree it’s better if humans didn’t have to flip burgers for a living (modulo solving employment being necessary for a decent/tolerable existence). It may turn out that the next step in software productivity is not a new generation of cool and highly-abstract declarative programming languages (4GL) but simply automating away the drudgery of writing code in the extant kinda advanced languages (3GL) which are ‘good enough’. Arguably Copilot doesn’t change much about the software development process. You can already get software developed pretty cheap if you farm it out to low-quality contractors or a lot of juniors straight out of school. There’s a whole branch of the industry that will hire anyone straight out of university to manage and front development teams in South Asia. But as we say, quantity has a quality all of its own. And the only way to manage software (not code) quality with this method is to verify and test that the deliverables actually meet your criteria and expectations. It’s currently a great time to focus on TDD/BDD and code specification because our time spent on the D is about to get a lot shorter. There is still a place for artisanal hand-crafted code. But the mechanical loom has arrived.