6 ms·
I'm mostly seeing software developers looking at the textual equivalent, GPT-3, and giving a spectrum of responses from "This is fantastic! Take my money so I c
by ben_w 4y ago
I'm mostly seeing software developers looking at the textual equivalent, GPT-3, and giving a spectrum of responses from "This is fantastic! Take my money so I can use it to help me with my work!" to "Meh, buggy code, worse than dealing with a junior dev."
I think the two biggest differences between art AI and code AI are that (a) code that's only 95% right is just wrong, whereas art can be very wrong before a client even notices [0]; and (b) we've been expecting this for ages already, to the extent that many of us are cynical and jaded about what the newest AI can do.
[0] for example, I was recently in the Cambridge University Press Bookshop, and they sell gift maps of the city. The background of the poster advertising these is pixelated and has JPEG artefacts.
It's highly regarded, and the shop has existed since 1581, and yet they have what I think is an amateur-hour advert on their walls.
- meebob 4y agoI do appreciate that the way in which a piece of code "works" and the way in which an piece of art "works" is in some ways totally different- but, I also think that in many cases, notably automated systems that create reports or dashboards, they aren't so far apart. In the end, the result just has to seem right. Even in computer programming, amateur hour level correctness isn't so uncommon, I would say. I would personally be astonished if any of the distributed systems I've worked on in my career were even close to 95% correct, haha.
- lycopodiopsida 4y agoUnderstanding what you are plotting and displaying in the dashboard is the complicated part, not writing the dashboard. Programmers are not very afraid of AI because it is still just a glorified fronted to stackoverflow, and SO has not destroyed the demand for programmers so far. Also, understanding the subtle logical bugs and errors introduced by such boilerplate AI-tools requires no less expertise than knowing how write the code upfront. Debugging is not a very popular activity among programmers for a reason. It may be that one day AI will also make their creators obsolete. But at that point so many professions will be replaced by it already, that we will live in a massively changed society where talking about the "job" has no meaning anymore.
- azornathogron 4y agoA misleading dashboard is a really really bad. This is absolutely not something where I would be happy to give it to an AI to do just because "no one will notice". The fact that no one will notice errors until it's too late is why dashboards need extra effort by their author to actually test the thing. If you want to give programming work to an AI, give it the things where incorrect behaviour is going to be really obvious, so that it can be fixed. Don't give it the stuff where everyone will just naively trust the computer without thinking about it.
- jimmaswell 4y ago> code that's only 95% right is just wrong It's still worth it on the whole but I have already gotten caught up on subtly wrong Copilot code a few times.
- edanm 4y ago> code that's only 95% right is just wrong, I know what you mean, but thinking about it critically, this is just wrong. All software has bugs in it. Small bugs, big bugs, critical bugs, security bugs, everything. No code is immune. The largest software used by millions every day has bugs. Library code that has existed and been in use for 30 years has bugs. I don't think you were actually thinking of this in your comparison, but I think it's actually a great analogy - code, like art, can be 95% complete, and that's usually enough. (For art, looks good and is what I wanted is enough, for code, does what I want right now, nevermind edge cases is enough.)
- CapmCrackaWaka 4y agoThis depends entirely on _how_ the code is wrong. I asked chatGPT to write me code in python that would calculate SHAP values when given a sklearn model the other day. It returned code that ran, and even _looked_ like it did the right thing at a cursory glance. But I've written SHAP a package before, and there were several manipulations it got wrong. I mean completely wrong. You would never have known the code was wrong unless you knew how to write the code in the first place. To me, code that is 95% correct will either fail catastrophically or give very wrong results. Imagine if the code you wrote was off 5% for every number it was supposed to generate. Code that is 99.99% correct will introduce subtle bugs. * No shade to chatGPT, writing a function that calculates shap values is tough lol, I just wanted to see what it could do. I do think that, given time, it'll be able to write a days worth of high quality code in a few seconds.
- nmfisher 4y agoOver the weekend I tried to tease out a sed command that would fix an uber simple compiler error from ChatGPT [0]. I gave up after 4 or 5 tries - while it got the root cause correct ("." instead of "->" because the property was a pointer), it just couldn't figure out the right sed command. That's such a simple task, its failure doesn't inspire confidence in getting more complicated things correct. This is the main reason I haven't actually incorporated any AI tools into my daily programming yet - I'm mindful that I might end up spending more time tracking down issues in the auto-generated code than I saved using it in the first place. [0] You can see the results here https://twitter.com/NickFisherAU/status/1601838829882986496 https://twitter.com/NickFisherAU/status/1601838829882986496
- edanm 4y agoEDIT: I posted this comment twice by accident! This comment has more details but the other more answers, so please check the other one! > code that's only 95% right is just wrong, I know what you mean, but thinking about it critically, this is just wrong. All software has bugs in it. Small bugs, big bugs, critical bugs, security bugs, everything. No code is immune. The largest software used by millions every day has bugs. Library code that has existed and been in use for 30 years has bugs. I don't think you were actually thinking of this in your comparison, but I think it's actually a great analogy - code, like art, can be 95% complete, and that's usually enough. (For art, looks good and is what I wanted is enough, for code, does what I want right now, nevermind edge cases is enough.) The reason ChatGPT isn't threatening programmers is for other reasons. Firstly, it's code isn't 95% good, it's like 80% good. Secondly, we do a lot more than write one-off pieces of code. We write much, much larger systems, and the connections between different pieces of code, even on a function-to-function level, are very complex.
- yourapostasy 4y ago> The reason ChatGPT isn't threatening programmers is for other reasons. Firstly, it's code isn't 95% good, it's like 80% good. The role that is possibly highly streamlined with a near-future ChatGPT/CoPilot are requirements-gathering business analysts, but developers at Staff level on up sits closer to requiring AGI to even become 30% good. We'll likely see a bifurcation/barbell: Moravec's Paradox on one end, AGI on the other. An LLM that can transcribe a verbal discussion directly with a domain expert for a particular business process with high fidelity, give a precis of domain jargon to a developer in a sidebar, extracts out further jargon created by the conversation, summarize the discussion into documentation, and extract how the how's and why's like a judicious editor might at 80% fidelity, then put out semi-working code at even 50% fidelity, that works 24x7x365 and automatically incorporates everything from GitHub it created for you before and that your team polished into working code and final documentation? I have clients who would pay for an initial deployment of that for an appliance/container head end of that which transits the processing through the vendor SaaS' GPU farm but holds the model data at rest within their network / cloud account boundary. Being able to condense weeks or even months of work by a team into several hours that requires say a team to tighten and polish it up by a handful of developers would be interesting to explore as a new way to work.
- Kye 4y ago>> "I think the two biggest differences between art AI and code AI are that (a) code that's only 95% right is just wrong, whereas art can be very wrong before a client even notices [0];" Art can also be extremely wrong in a way everyone notices and still be highly successful. For example: Rob Liefeld.
- jhbadger 4y agoAnd in the same way as Liefeld has a problem drawing hands! Maybe he was actually ahead of us all and had an AI art tool before the rest of us.
- itronitron 4y ago>> whereas art can be very wrong before a client even notices No actually, that's not how that works. You're demonstrating the lack of empathy that the parent comment brings up as alarming. Regarding programming, code that's only 95% right can just be run through code assist to fix everything.
- ben_w 4y agoArtists are, necessarily, perfectionists about their work — it's the only way to get better than the crude line drawings and wildly wrong anatomy that most people can do. Frustratingly, most people don't fully appreciate the art, and are quite happy for artists to put in only 20% of the effort. Heck, old enough to remember people who regarded Quake as "photorealistic", some in a negative way saying this made it a terrible threat to the minds of children who might see the violence it depicted, and others in a positive way saying it was so good that Riven should've used that engine instead of being pre-rendered. Bugs like this are easy to fix: `x = x – 4;` which should be `x = x - 4;` Bugs like this, much harder: #define TOBYTE(x) (x) & 255 #define SWAP(x,y) do { x^=y; y^=x; x^=y; } while (0) static unsigned char A[256]; static int i=0, j=0; void init(char \*passphrase) { int passlen = strlen(passphrase); for (i=0; i<256; i++) A[i] = i; for (i=0; i<256; i++) { j = TOBYTE(j + A[TOBYTE(i)] + passphrase[j % passlen]); SWAP(A[TOBYTE(i)], A[j]); } i = 0; j = 0; } unsigned char encrypt_one_byte(unsigned char c) { int k; i = TOBYTE(i+1); j = TOBYTE(j + A[i]); SWAP(A[i], A[j]); k = TOBYTE(A[i] + A[j]); return c ^ A[k]; }
- asdf123wtf 4y agoA lot depends what the business costs are of that wrong %5. If the actual business costs are less than the price of a team of developers... welp, it was fun while it lasted.
- toomanydoubts 4y agoThe other day I copied a question from leetcode and asked GPT to solve it. The solution had the correct structure to be interpreted by leetcode(Solution class, with the correct method name and signature, and with the same implementation of a linked list that leetcode would use). It made me feel like GPT was not implementing the solution for anything. Just copying and pasting some code it has read on the internet.