7 ms·
I tested Copilot in an existing open-source codebase with unit tests [video]
- AYBABTME 5y agoIs there a TL;DR?
- hdjjhhvvhga 5y agoIt's mediocre. Sometimes it's plain wrong, sometimes right but not exact and so on.
- yourapostasy 5y agoWas Copilot's value proposition that it gets better over time as more developers use it over a longer period?
- hdjjhhvvhga 5y agoBut the value for who? Last time I checked the consensus was that using it in any project that is not for private use can get you in legal trouble. So for me it's completely useless, even if technically it could be improved with time (which may or may not happen).
- yourapostasy 5y ago> But the value for who? Presumably value "unlocked" for GitHub investors, by leveraging the ML model that drives some product based upon Copilot like a next-gen IDE. I agree it's not a good risk to assume for the users of Copilot.
- deleted 5y ago[deleted]
- byteface 5y agoHe seems to know the codebase and works out the problem for about 30 mins sometimes fastfowarded. Then at about 28 mins copilot smashes the answer instantly when he types a method name at the bottom of a utils file. he doesn't like it though as he seems to know the api really well and wants the util to use his api instead. so deletes the copilot answer and types out a different answer using the api which he knows. then creats a few unit tests. This was an interesting watch and useful to see other peoples setups.
- deleted 5y ago[deleted]
- psvj 5y agoit would be interesting if copilot could generate the code based on your unit tests
- postalrat 5y agoWould that be more desirable than building your tests from your application? Seems to me that once you can go either way then you just proved the other redundant.
- h0l0cube 5y agoThe central thesis of this diatribe is that in order to have a 'complete' unit^ test, you need something at least as complicated as the original code: https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste... ^ where the 'unit' is basically a function. i.e., to test a function fully, write a test as complicated as the function
- Tcc1 5y agoHe probably spent more time figuring out if the suggestions are usable than he would have needed to just implement those manually.
- deleted 5y ago[deleted]
- byteface 5y agoI guess if he wrote tests first it would have said. no method called x in util. TDD helps take a lot of thinking out of problems.