3 ms·
I think it's too early to tell but my main concern about copilot is code maintainability and security. Copilot is able to barf out helpful expressions that wil
by munk-a 3y ago
I think it's too early to tell but my main concern about copilot is code maintainability and security. Copilot is able to barf out helpful expressions that will reduce the amount of code we need to write by hand - I think it's excellent when it comes to reducing boilerplate... but I think a large amount of boilerplate existing belies a bigger issue with the project. The majority of software engineering isn't writing code - copilot may be beneficial as an accessibility aide for developers that have typing impairments but most developers can type faster than they can think - if the level of boilerplate in your project is reasonable then this should mean you're never prevented from thinking because your fingers are still working on recording your previous thought. However, at the end of the day, if you can help reduce carpal tunnel that's still a win.
The problem I can foresee with copilot is that the scenario change you're agreeing to is that you'll type less but need to read over the code produced more - this is an effort that isn't normally necessary (typos happen but those should take a trivial time to correct) but when copilot is involved you need to proof all the code that is being generated. There is a motivation to skip this step and just accept the code was written correctly and that will inevitably lead to security problems - and there is a motivation to not correct or alter auto-filled command. If there's a multi-dimensional array and you think it semantically makes sense to iterate it over dimension a then dimension b and copilot instead goes with b as the major index then it's more likely to remain in a b major iteration - that may make code less readable or it may cause major issues down the line.
Copilot, IMO, is optimizing the least important part of development right now and it costs us more to correct it then it would to just splat out the correct code _but_ this is a similar argument to longbows vs. crossbows - hand a peasant a crossbow and they can fire a crossbow - train a peasant for 30 years and they can fire a longbow - the longbow is more powerful, but the crossbow is a clear choice in terms of RoI. It may be that today's developers will only benefit from copilot minimally since we've invested the training time in standard development practices but tomorrow's developers will eschew a lot of the algorithmic learning and still be able to deliver the majority of the value.
- Zambyte 3y ago> I think it's excellent when it comes to reducing boilerplate I think the rest of your comment already elaborates this point, but I want to highlight that language models do not reduce boilerplate, they automate it. Reducing boilerplate happens by designing simpler APIs, not by making language models interface with complex APIs for you.
- munk-a 3y agoI agree - a lot of my work is actually meta-programming... I've built the view interfaces, the DB handles, the library functions that makes other developer's lives easier - there are real gains you can achieve by making other developers more expressive.
- javaunsafe2019 3y agoThis is so well summarised. Thank you. I also don’t believe the hype. I tried to migrate spring boot security configuration between major versions and Copilot was just not able to produce a failure free output no matter how long I tried (I tried to hint a lot but it didn’t help). And in the end software engineering isn’t as much about writing code than to build systems for a certain purpose and in the first place you need to understand the needs/purpose and therefore talk with people instead of just writing code. Writing code is not difficult. Understanding you domain might be. Pretty sure this bubble will burst and we will have a second wave in years which might be able to do it all. But not this text generation vodoo we see today.
- currymj 3y agoHere's a scenario: - I have no idea how to use an API, and it is poorly documented, but I am an expert in the general problem it's trying to model - I need to write code using this API in the next hour - That code will only ever run 5-6 times, each by hand with an expert to look at any errors or crashes, and then will be thrown away forever copilot is killer for this scenario. it's the most common scenario when writing machine learning research code. i think probably the people building these tools are enjoying the dogfood, maybe more than the typical end user.
- gv83 3y agothe only thing this llm craze is helping is nvidia/openai/ms war chest, and fueling the illusion that every company can finally have their developers by taking any domain expert or barely knowledgeable person paired with an ai assistant. the mountains of trash produced by these things will end up costing a metric ton.