11 ms·

Programming AIs worry me

137 points by azhenley 4y ago

10g1k 4y ago
Prediction: An inflection/tipping point will occur when an AI has the ability to programme/create another AI with capabilities or coding superior to its own. That's the moment. It will snowball from there. A begets B which is better than A; B begets C which is better than B; and so on. After that point it will be limited only by processing capability and network connectivity.
- turtleyacht 4y ago
  Insightful! Of all the creative prompt engineering I've read, I haven't found that yet: the equivalent of "fork a new process and while true." Hey, ChatGPT. Ask this question at this URL when I talk about farms. But while you do that, ask it to also think on farms. Every few seconds, append its responses to every answer you give me.
fassssst 4y ago
Train a separate AI to write tests against the other AI’s code, that’s rewarded for finding defects :p
- suzzer99 4y ago
  AI: "This is a bug" Human: "No it's not" AI: "I won't hurt you unless you hurt me first."
buildbot 4y ago
Humans may he bad at proofreading code, but we’ve had a lot of experience mitigating that too - what else are tests for? Maybe coding AIs will push everyone into test driven development - write a spec/tests of whatever then just wait awhile for it to crunch through code until something works.
- deleted 4y ago
  [deleted]
- rom-antics 4y ago
  That's an interesting idea, it sounds almost like using a fuzzer for generating new code instead of finding bugs in existing code.
- nathanwh 4y ago
  Until someone makes the AI write the test too, I don't think there's going to be any red-tape style solutions to this problem that isn't just "no AI written code allowed".
  - sharemywin 4y ago
    Actually, it probably could write really good test cases. they tend to be boring and verbose. It's only test data but you get the point Sure, here are some examples of phone numbers in various formats: (123) 456-7890 123-456-7890 +1 (123) 456-7890 1-800-123-4567 555.123.4567 123.456.7890 1234567890 (123) 456-7890 ext. 123 +44 123 456 7890 (international format) 011 44 123 456 7890 (international access code format) 123/456-7890 123\456-7890 (123) 456-7890 x1234 (with extension) 123-456-7890 x1234 (with extension) 1 (123) 456-7890 (with country code)
    - nathanwh 4y ago
      Out of curiosity I tried chat GPT for this and it was able to generate a reasonable set of test cases but only after prompting it for including international access and "+" characters. I also had to prompt it to use the parameterize feature, but I guess that's personal preference. Final test: https://pastebin.com/J36JY78i https://pastebin.com/J36JY78i First response: https://pastebin.com/S23GFPDq https://pastebin.com/S23GFPDq
    - orwin 4y ago
      I use Copilot to write my tests, but using ChatGPT to generate test data could be a good idea. I'll have to fiddle with it when i'm not on a timer. Thank you for the suggestion.
baby 4y ago
heh, this was true for stackoverflow or for googling pretty much anything. Nothing new. The tools are not doing our job, they're just making the job easier.
- fragsworth 4y ago
  Your head is in the sand. Some time soon, they can and will actually do the whole job.
  - jarjoura 4y ago
    Nah, never the whole job. Software engineering isn't 100% writing code, so even if an AI could write 100% of the code, there's still all of the other things around it that need to get done. I am actually super excited for better AI tools to help write so much of the boilerplate crap I have to write. I'm here because I want to solve problems, not write code, and writing less code makes me happy.
    - politician 4y ago
      We would have never gotten JSON if AI was around to write XML for us.
      - suzzer99 4y ago
        Terrifying
  - ResearchCode 4y ago
    It looks like it will be able to do every non-programming office job first.
    - l33t233372 4y ago
      I agree completely. People are forgetting that programming is a difficult technical and creative task. If an AI can master that, it should be able to master many tasks which are far less technical and/or far less creative.
brundolf 4y ago
I think the economic reality is that for a lot of software, especially user-facing software, "correct enough that it passed manual QA" is the threshold. It already is; even pre-AI. So I think AI + proofreading will be successful there, and it may not even make things worse than they currently are, especially in the long run While critical logic, that really really has to be correct, to a point where you should maybe have it all planned out before even writing code, will still need to be hand-programmed for a long time
- aardvark179 4y ago
  I think the problem is that code that passes manual QA testing is fine until it suddenly isn’t. That may be due to the complexity of the code base, the turn around time on doing those tests, the accuracy of people following scripts, or just the number of tests and the size of the code base. Similarly integration tests are fine until they suddenly aren’t, and you really need things like unit testing. Unit testing also only works up to a point without tools to reason about the entire codebase and how it interfaces with others. Organizations flourish or flounder based on recognizing when they are approaching those points and dealing with their underlying issues. Maybe AIs can write code that is good enough in a lot of cases, but unless people can understand that code and how to work with it and refactor it as things grow then they may not be a good idea.
  - brundolf 4y ago
    All of those things are true, and yet, that's how a lot of software is built in practice because it still ends up "close enough". The market has shown that this is acceptable in a whole lot of domains I'm not saying I like it, I'm saying it's the reality
jarjoura 4y ago
Just like a human, don't use these new AI tools to read your mind. I've found ChatGPT to be impressively helpful at knowing where to continue to look for something. I can write a sentence and explain what I'm looking for, and maybe even paste in a code snippet, and 10 seconds later I get enough information to go back to Google and continue digging deeper. For me it's become a bridge tool. Instead of spending an hour trying to get Google to understand my confusion and endless ads thrown my way, I get a very concise answer with full definitions in case I needed to know. It understands me, which is the most important part. The code examples it spits out are meh, and likely always going to be error prone, but I cannot imagine it won't improve over time.
- pixl97 4y ago
  > Google to understand my confusion and endless ads thrown my way, Another reason why Google is terrified here. Search is per query cheap. GPT isn't.
  - dybber 4y ago
    I guess many companies will pay for Github Co-pilot, ChatGPT style things. No one will pay for access to a search engine.
    - beezlewax 4y ago
      Honestly if it was not an advert riddled mess tracking my every move I would definitely pay for search.
    - Bjartr 4y ago
      Kagi, a paid search engine, and I, one of their customers, disagree, for what that's worth.
      - hwayne 4y ago
        Holy shit this looks AMAZING. Rule out domains from searches entirely? Sold!
andrewstuart 4y ago
Relax. AI assisted programming is brand new. It's just the start. So if it gets things wrong, is missing features, annoys people, remain cool, gigantic leaps are coming far beyond your wildest dreams. And there will be lots of people around right now showing how smart they are by making AI do wrong or silly things. Ignore them - none of these prove that AI is the wrong direction. AI programming is the single most exciting thing to ever happen to programming - a permanent mentor and pair programmer at your side levels up every programmer in the world. Even more exciting is innovations that are a direct outcome of AI assisted programming. When the world wide web first came along, everyone made a website that was like a company brochure. That was Web 1.0 and that's all people had the imagination for. In time, the usages of the web started to arrive that leveraged the core values of the technology, such as interconnection between people. That's where AI programming is now. We are at AI 1.0 Programming models will come out that are primarily oriented to the strengths of AI programming - but first someone needs to think them up. And they will not only be surprising but absolutely mind blowing. It's entirely possible that in the future the entire internet has an AI sublayer that does all the work of systems talking to systems and people simply don't have anything to do with it. It's possible that the nocode movement is the foundation of the next great innovations in programming. Existing programming models are like the company brochures of web 1.0 - our old, hard, human centric way of doing things. Programming literally should be as easy as talking to the computer and telling it how to shapoe and change the software and it should happen right there in real time on your screen. It should be an iterative, interactive process that anyone can do. It should be more like painting and less like arcane wizardy that can be performed only by the programmer priests who have spent years learning. Even the most advanced software should be capable of being built this way. In the 1980's I worked for a startup company run by a super smart guy Peter Dunai, who once said "a revolution in programming is coming". I thought I'd seen that revolution already with the Internet, modern programming languages and StackOverflow, but now I realise none of that many anything compared to what the true revolution is going to look like. AI 2.0 will come and I can't wait.
- bombolo 4y ago
  > AI 2.0 will come and I can't wait. It might happen in a decade or more.
mjb 4y ago
It's interesting that Hillel, a huge formal methods and specification advocate, sees this as a proofreading problem and not a specification one. I can't say I agree.
- hwayne 4y ago
  I'd be interested in hearing more about your disagreement! If I had to reconcile this post with my overall shameless FM promotion, I'd say that "proofreading is hard" is the human limit that is amplified by AI use, and more specification is a potential solution. But also, historically, an expensive one, so I don't know if that's going to adopted enough to fully offset the drawbacks of mass AI adoption.
adversaryIdiot 4y ago
I haven't had much luck using these AI's past surface level stuff.
- hwayne 4y ago
  They are godsent at getting a TikZ drawing jumpstarted.
twawaaay 4y ago
Anybody worrying that programmers loose jobs stop worrying. All the automation in the world does not replace the need for people to debug problems. Even more interestingly, the more something is automated the more time is spent on solving the issues when they happen. Finding a simple mistake in a very complex automation can be a challenging project. But... if you have been programming Cobol your entire life you might want to start learning some newer technology.
- justincredible 4y ago
  [dead]
- bryanrasmussen 4y ago
  You think the programming AIs are going to be trained on Cobol?
  - agentwiggles 4y ago
    I mean, I have no ability to check it for correctness, but ChatGPT is certainly capable of writing Cobol. Just asked it for an implementation of Dijkstra's algorithm and it looks reasonable enough to me. Why _wouldn't_ the programming AIs be trained on Cobol?
    - bryanrasmussen 4y ago
      Ok, I would think you probably would not want to devote a lot of resources to training for Cobol given the smaller niche it represents which I figured would make it less useful.
bambax 4y ago
> Might as well share a wishlist. [1/] A ChatGPT-style AI that can only reply with links to reference docs, libraries, or wikipedia articles That already exists. It's called an effing search engine. But for some reason people have decided, as of ten days ago, that they're not good enough anymore. As for why? I really have no clue. But they still work extremely well.
- Bjartr 4y ago
  > As for why? I really have no clue. But they still work extremely well. I've found Google to be significantly less useful than it used to be. Largely because I have to jump through hoops to convince it to show me what I'm actually looking for. ChatGPT is already much better often enough that it can turn 20+ minutes of faffing about with Google into one minute or less of just asking for what I want and getting it. Yeah, it's wrong sometimes, but it still manages to point me in the right direction often enough to be worth the need for extra vigilance
mordae 4y ago
Hahaha, try asking GPT-3 about electronic engineering. --- Transcript of IRC channel #electronics: Mordae: Hi. So I was designing a transformerless LED bulb and I was wondering about the filter capacitor. What happens when the load (one of the LEDs) fail? Rampant: Hi Mordae. It really depends on the design, but the general idea is that the filter capacitor will help prevent spikes and other issues caused by the failing LED. It can also help with voltage regulation. --- Actual answer: Light bulb needs a bulky electrolytic cap. Those are usually not 400V tolerant. You are looking at 60V at best at the required sizes. Losing the load will swiftly charge up the capacitor to 325V. But well before that happens, there are going to be fireworks: https://www.youtube.com/watch?v=6WUxgmMDts4 https://www.youtube.com/watch?v=6WUxgmMDts4 I believe that there are eventually going to be useful mentors for bunch of fields, but right now it's crazy dangerous to blindly follow its recommendations.
- whartung 4y ago
  I'm sorry, is there enough context in the original question as presented to come up with your "actual answer"? I mean, I don't know, I'm not an EE, lucky I can make a light switch work. But given the dearth of details in the query, is that AI answer grotesquely wrong? Just seems like generic advice.
  - Dylan16807 4y ago
    > I'm sorry, is there enough context in the original question as presented to come up with your "actual answer"? I'd say there is. "Transformerless" implies a specific design, especially when you mention there's a filter capacitor. > is that AI answer grotesquely wrong? Yes. It's fine to say the filter cap helps with "spikes and other issues", but not ones caused by a failing LED. It took half a true sentence and made it into a false sentence. In this design the filter cap is something you give protection to much more than something that provides protection.
- deleted 4y ago
  [deleted]
- Bjartr 4y ago
theknocker 4y ago
[dead]
jmull 4y ago
I read this but actually not sure what the author is worried about. I guess they feel proofreading AI-written code is harder than writing code? That seems like a leap to me. I don't expect AI-assisted coding to be a panacea, and I'm sure there are ineffective ways to do it. But it seems like there are effective ways too. So, sure, there is work to do to figure out how to use these tools in the best way. But that's not anything to worry about, IMO. It's opportunity.
seletskiy 4y ago
With the new advancements in AI, it's expected that there will be more code being generated by programs rather than written by programmers. Making sure these programs deliver accurate and error free code is a difficult task. The challenge here lies in effectively proofreading AIs so that any bugs are caught before they reach end users. Good tools such as unit tests can help but may not be enough to make sure AI generated code is flawless. As the use of AI increases, we must look to develop better methods for ensuring quality assurance on these programs if they are to truly be useful.
- avdempsey 4y ago
  In my career I've seen resources pulled away from QA teams (sometimes completely disbanding). It would be interesting to see things swing the other way.
- zokier 4y ago
  > it's expected that there will be more code being generated by programs rather than written by programmers. That has probably been case since the 80s; nearly all code running on your CPU is generated by a program ("compiler") from a textual description ("source")
  - vitiral 4y ago
    That's not what is meant. Programming languages are basically huge macro systems that expand to assembly, but the code is still written by a human. Layers of abstraction are a problem too, but they are a different problem.
- Panzer04 4y ago
  This feels GPT generated.
pphysch 4y ago
The author begins by giving a number of test cases, like f(35)==6, for which the AI provides a candidate f, but the author doesn't actually implement the test cases, is surprised when the AI is wrong, and laments the resulting burden of proofreading. Just implement those tests, for heaven's sake!
- majormajor 4y ago
  Implementing a correct and comprehensive set of test cases for code is more time consuming and often more difficult than writing the code itself. (The comprehensive part especially - how do you prove to yourself you've covered all the necessary edge cases?) "Reading code is harder than writing it" "Debugging code is harder than writing it" These are pretty common statements, going back years. They should make the lack of a silver bullet here very obvious. (Yes, most software is already broken in various ways - my worry is that this will just make it more broken, e.g. non-technical management saying "why do we need to do as much QA, we have AI writing the code now!?")
  - pphysch 4y ago
    Yes, writing good tests is hard. My point is that he already has the relevant test cases. The hard work is done.
- fragmede 4y ago
  even easier: ask the LLM for those test cases!
- hwayne 4y ago
  To clarify, Bertrand Meyer is the one who gave the test cases and accepted the wrong answer without verifying them. I'm the one who lamented the resulting burden of proofreading.
yen223 4y ago
Programming AIs worry me because I think they can get better at programming faster than I can get better at programming
- qualudeheart 4y ago
  I once coined an aphorism: “Deepmind can train a new model faster than you can go back to grad school.”
guluarte 4y ago
it's like alexa, siri, etc. I'd rather do the stuff myself than talking nonsense with an "ai"
tonyhb 4y ago
> A ChatGPT-style AI that can only reply with links to reference docs, libraries, or wikipedia articles This is exactly what we've built at Inngest so that you can learn our SDK: https://www.inngest.com/ai-personalized-documentation https://www.inngest.com/ai-personalized-documentation. It can only reply with SDK examples, an explanation, and links to relevant guides. Soon, we're revamping our docs to highlight sections relevant only to your use case - you'll be able to type what you want to do with Inngest and be pointed straight to the relevant guides and docs. We trust developers to write the code, and only use GPT to assist and point the right way. It's a good compromise, and it's the most immediately applicable use that I can see!
- hwayne 4y ago
  I know this is getting downvoted for being an ad but this is exactly what I was talking about, so thank you I will look into it!
aib 4y ago
What worries me are the low-grade programmers. By delegating their work to tools like ChatGPT (and doing so always a level or two beyond the current abilities of the tools) they will manage to take us to newer lows.
- Existenceblinks 4y ago
  Underrated. It already happened in frontend space. Job market is already rotten by incompetent tech leaders who aren't immune to all the bullshit and bought all the hype from VC-backed bigco.
- deleted 4y ago
  [deleted]
layer8 4y ago
The “almost correct” aspect will be especially fun when AIs are used to refactor code. I was thinking about how a future legacy codebase might look like after a decade of having AI-generated code added to it, and then of course you would want the AI to apply some refactorings from time to time. One might argue that it won’t be a problem with proper tests. But do coders want to spend most of their time proofreading AI-generated code and taking care of proper test coverage? That doesn’t sound very appealing.
- plasticchris 4y ago
  Why can’t the AI write tests too? Then you can have 100% coverage of your (probably) correct code…
  - saulpw 4y ago
    That's even worse, because now the tests are wrong too! I asked it for a list of test cases for a particular function I wanted to write, and every single one of the cases it presented was incorrect. So it was worse than useless, now I had to check its work and then do the work myself anyway.
  - layer8 4y ago
    It can, but you have to proofread those too.
  - noobermin 4y ago
    Why not generate an AI that can take any code and decide whether it has any errors or not?
    - dskrvk 4y ago
      An AI that tells whether or not a program will halt.
- ParetoOptimal 4y ago
  > The “almost correct” aspect will be especially fun when AIs are used to refactor code. Hope your function has all invariants covered with one or more property tests!
valcron1000 4y ago
If reading is harder than writing code, why bother with tools that deal with the later instead of the former?
vitiral 4y ago
My worry is code complexity explosion. Once we start using AI to add features to infrastructure (OS's, compilers, data serialization, etc) it will quickly run away from our ability to understand it. It has already run away, it will now be fucking _flying_.
- Verdex 4y ago
  I expect that before my career is over that I will be given more than one project to fix that the AI stopped being able to deliver features for. I imagine it's going to be one of those laugh so you don't cry affairs.
  - turtleyacht 4y ago
    I like it. We're generating "COBOL" debt for our future brethren as we solve present problems with the most modern tools we had.
  - AnimalMuppet 4y ago
    Maybe laugh because you're going to get paid for a really long time. (And then, yeah, cry because of what you're going to have to do to earn the money...)
- layer8 4y ago
  The code might start to resemble biological systems, which are a mess of complexity.
  - ModernMech 4y ago
    In the future, programs will develop genetic disorders, and will need to be diagnosed by programmers with the equivalent of medical degrees.
    - 6177c40f 4y ago
      Unfortunately, doctors might be replaced by diagnostic AIs. It'll be AIs all the way down by then.
agentultra 4y ago
It’s a bit of a nihilistic view that programmers are merely slow boiler plate auto completion systems and our jobs will be replaced by AI. We already write more code than we understand given how lowly the industry regards formal methods and how divided it is even on testing. It is not surprising to me that Bertrand Meyer could miss that error in the code! A former colleague of mine challenged his grad students to implement an error-free version of binary search. He gave them the specification and verified each one: the majority of the submissions had errors. It’s a marvel that computers and software work so well. And instead of tackling the problem of programming, as Dijkstra put it, we’re going to automate writing more code. Great. See you in a few years when we can’t even keep up with the deluge of error prone programs. Although we have gotten this far with very few proofs and models. Who knows, maybe we’ll just tolerate it? It would be nice to have code synthesis work on larger examples from deep specifications. And models that understand proof tactics and can speed up the process of formalizing proofs. That would be nice.
- sigstoat 4y ago
  > A former colleague of mine challenged his grad students to implement an error-free version of binary search. He gave them the specification and verified each one: the majority of the submissions had errors. I was in a class on Isabelle where we had to write and prove correct a binary search function. As I recall, everyone admitted that their first function didn’t work.
  - renewiltord 4y ago
    Java's was broken till '06
    - OnionBlender 4y ago
      https://en.wikipedia.org/wiki/Binary_search_algorithm#Implementation_issues https://en.wikipedia.org/wiki/Binary_search_algorithm#Implem... People often do (L+R)/2 but that can overflow. Using L+(R-L)/2 avoids the overflow problem.
stevage 4y ago
These are really solid concerns. This shift from "creator" to "supervisor" is happening in lots of AI areas - look even at how self-driving cars are currently expecting their human drivers to act as supervisors, which they're bad at. Also, on this point: > We can make proofreading easier with better tooling. Unit tests are a means of “proofreading”: we can catch AI errors automatically with tests. The unit tests will definitely be written with AI as well. If that AI and the coding AI are wrong in the same way, we will have very subtle and hard to catch errors.
- gqcwwjtg 4y ago
  That problem sounds just like when I write tests for my own code. I don’t know if it’s totally avoidable.
  - NateEag 4y ago
    It just now occurred to me that a natural solution to this problem could be to move past code review to "other teammate writes the tests for your PR." You owe them an additional beer at the next happy hour for each test they write you wind up pushing a fix for.
Waterluvian 4y ago
Oh wow. That IS what I want. An AI that can offer a code review and suggest things.
thebigspacefuck 4y ago
To be fair, I’ve seen humans do worse.
amelius 4y ago
Before we have AIs that can program, we will have AIs that can do computer administration tasks, like solving dependency problems with software installations etc. and compiling and installing stuff. So before we have a tool that can do that, I'm not worried.
- zshrdlu 4y ago
  It would be fun to see one attempt a Gentoo install.
jongjong 4y ago
Of course, AI models are often wrong. Even if they were right in the sense that they gave you exactly what you asked for, they may still be wrong in the sense that their interpretation of your requirements may not be what you intended; they are not great at filling the gaps if there is any context missing. The same amount of time it takes to provide an AI with all the necessary, nuanced context to produce an optimal solution could have been used to actually produce the desired optimal solution yourself. The hard part of software development is not coding, it's the amount of critical thinking and synthesizing required to produce optimal solutions. I'm optimistic about AIs because they may finally prove once and for all that software engineers cannot be commodified as easily and thoroughly as most other white collar roles. It may be easier to commodify executives who make a small number of abstract high level decisions than it is to commodify developers who make many extremely nuanced, detailed decisions every day. For sure, AI will replace certain kinds of developers, but not all kinds, at least not for some time. I think senior developers and architects will be among the last people in the firing line.
- sramam 4y ago
  > I think senior developers and architects will be among the last people in the firing line. And soon there will be no junior developers left to rise up to their ranks... So "last" in more ways than one!
m3kw9 4y ago
Would AIs that write unit test help test their own code, they will write code till it passes and link all the modules together
- m3kw9 4y ago
  They already knew this so I didn’t start the AI coding apocalypse
ccvannorman 4y ago
<< Laughs >> Oh wait, you're serioous?! << Laughs harder >> AI is great for making syntactically correct code; and so is copy-pasting the first result that works after a low quality search. It still takes real minds to make code that is worth anything, and there will always be buffoons making code that doesn't work. Except the HN website, of course. ;-)
gmt2027 4y ago
The work of a master craftsman using the most basic hand tools is still leagues ahead of any amateur with advanced tools, automation, fancy CNC, and lasers. Mass production is where the tech would favour that amateur. We are surrounded by cheap mass-produced disposable products and are well aware of their low quality. No gourmet chef should be threatened by the efficiency of McDonald's. Fortunately, our software systems are custom artifacts with zero marginal cost of reproduction. AI will make competent programmers more productive and take expert programmers to new levels. One transformation will come when a single developer can do the work of a team of ten today. Like solo content creators who no longer need large studios to finance and organise massive production teams and expensive equipment, we'll see more independent developers and small teams creating innovative software with some artistic character. Without the drag of management and upfront financing, the threshold for success will be much lower. Another will be a market for lemons. Employers will have to find ways to establish competence at building high quality, understandable and maintainable systems. AI can already pass Leetcode-style tests so these are clearly not the right measures. This is my best optimistic take on things. The rest of the time I'm worried the AI is coming for our jobs.

137 points by azhenley 4y ago

Prediction: An inflection/tipping point will occur when an AI has the ability to programme/create another AI with capabilities or coding superior to its own. That's the moment. It will snowball from there. A begets B which is better than A; B begets C which is better than B; and so on. After that point it will be limited only by processing capability and network connectivity.

turtleyacht 4y ago
Insightful! Of all the creative prompt engineering I've read, I haven't found that yet: the equivalent of "fork a new process and while true." Hey, ChatGPT. Ask this question at this URL when I talk about farms. But while you do that, ask it to also think on farms. Every few seconds, append its responses to every answer you give me.

fassssst 4y ago

Train a separate AI to write tests against the other AI’s code, that’s rewarded for finding defects :p

suzzer99 4y ago
AI: "This is a bug" Human: "No it's not" AI: "I won't hurt you unless you hurt me first."

buildbot 4y ago

Humans may he bad at proofreading code, but we’ve had a lot of experience mitigating that too - what else are tests for? Maybe coding AIs will push everyone into test driven development - write a spec/tests of whatever then just wait awhile for it to crunch through code until something works.

deleted 4y ago
[deleted]
rom-antics 4y ago
That's an interesting idea, it sounds almost like using a fuzzer for generating new code instead of finding bugs in existing code.
nathanwh 4y ago
Until someone makes the AI write the test too, I don't think there's going to be any red-tape style solutions to this problem that isn't just "no AI written code allowed".
- sharemywin 4y ago
  Actually, it probably could write really good test cases. they tend to be boring and verbose. It's only test data but you get the point Sure, here are some examples of phone numbers in various formats: (123) 456-7890 123-456-7890 +1 (123) 456-7890 1-800-123-4567 555.123.4567 123.456.7890 1234567890 (123) 456-7890 ext. 123 +44 123 456 7890 (international format) 011 44 123 456 7890 (international access code format) 123/456-7890 123\456-7890 (123) 456-7890 x1234 (with extension) 123-456-7890 x1234 (with extension) 1 (123) 456-7890 (with country code)
  - nathanwh 4y ago
    Out of curiosity I tried chat GPT for this and it was able to generate a reasonable set of test cases but only after prompting it for including international access and "+" characters. I also had to prompt it to use the parameterize feature, but I guess that's personal preference. Final test: https://pastebin.com/J36JY78i https://pastebin.com/J36JY78i First response: https://pastebin.com/S23GFPDq https://pastebin.com/S23GFPDq
  - orwin 4y ago
    I use Copilot to write my tests, but using ChatGPT to generate test data could be a good idea. I'll have to fiddle with it when i'm not on a timer. Thank you for the suggestion.

baby 4y ago

heh, this was true for stackoverflow or for googling pretty much anything. Nothing new. The tools are not doing our job, they're just making the job easier.

fragsworth 4y ago
Your head is in the sand. Some time soon, they can and will actually do the whole job.
- jarjoura 4y ago
  Nah, never the whole job. Software engineering isn't 100% writing code, so even if an AI could write 100% of the code, there's still all of the other things around it that need to get done. I am actually super excited for better AI tools to help write so much of the boilerplate crap I have to write. I'm here because I want to solve problems, not write code, and writing less code makes me happy.
  - politician 4y ago
    We would have never gotten JSON if AI was around to write XML for us.
    - suzzer99 4y ago
      Terrifying
- ResearchCode 4y ago
  It looks like it will be able to do every non-programming office job first.
  - l33t233372 4y ago
    I agree completely. People are forgetting that programming is a difficult technical and creative task. If an AI can master that, it should be able to master many tasks which are far less technical and/or far less creative.

brundolf 4y ago

I think the economic reality is that for a lot of software, especially user-facing software, "correct enough that it passed manual QA" is the threshold. It already is; even pre-AI. So I think AI + proofreading will be successful there, and it may not even make things worse than they currently are, especially in the long run While critical logic, that really really has to be correct, to a point where you should maybe have it all planned out before even writing code, will still need to be hand-programmed for a long time

aardvark179 4y ago
I think the problem is that code that passes manual QA testing is fine until it suddenly isn’t. That may be due to the complexity of the code base, the turn around time on doing those tests, the accuracy of people following scripts, or just the number of tests and the size of the code base. Similarly integration tests are fine until they suddenly aren’t, and you really need things like unit testing. Unit testing also only works up to a point without tools to reason about the entire codebase and how it interfaces with others. Organizations flourish or flounder based on recognizing when they are approaching those points and dealing with their underlying issues. Maybe AIs can write code that is good enough in a lot of cases, but unless people can understand that code and how to work with it and refactor it as things grow then they may not be a good idea.
- brundolf 4y ago
  All of those things are true, and yet, that's how a lot of software is built in practice because it still ends up "close enough". The market has shown that this is acceptable in a whole lot of domains I'm not saying I like it, I'm saying it's the reality

jarjoura 4y ago

Just like a human, don't use these new AI tools to read your mind. I've found ChatGPT to be impressively helpful at knowing where to continue to look for something. I can write a sentence and explain what I'm looking for, and maybe even paste in a code snippet, and 10 seconds later I get enough information to go back to Google and continue digging deeper. For me it's become a bridge tool. Instead of spending an hour trying to get Google to understand my confusion and endless ads thrown my way, I get a very concise answer with full definitions in case I needed to know. It understands me, which is the most important part. The code examples it spits out are meh, and likely always going to be error prone, but I cannot imagine it won't improve over time.

pixl97 4y ago
> Google to understand my confusion and endless ads thrown my way, Another reason why Google is terrified here. Search is per query cheap. GPT isn't.
- dybber 4y ago
  I guess many companies will pay for Github Co-pilot, ChatGPT style things. No one will pay for access to a search engine.
  - beezlewax 4y ago
    Honestly if it was not an advert riddled mess tracking my every move I would definitely pay for search.
  - Bjartr 4y ago
    Kagi, a paid search engine, and I, one of their customers, disagree, for what that's worth.
    - hwayne 4y ago
      Holy shit this looks AMAZING. Rule out domains from searches entirely? Sold!

andrewstuart 4y ago

Relax. AI assisted programming is brand new. It's just the start. So if it gets things wrong, is missing features, annoys people, remain cool, gigantic leaps are coming far beyond your wildest dreams. And there will be lots of people around right now showing how smart they are by making AI do wrong or silly things. Ignore them - none of these prove that AI is the wrong direction. AI programming is the single most exciting thing to ever happen to programming - a permanent mentor and pair programmer at your side levels up every programmer in the world. Even more exciting is innovations that are a direct outcome of AI assisted programming. When the world wide web first came along, everyone made a website that was like a company brochure. That was Web 1.0 and that's all people had the imagination for. In time, the usages of the web started to arrive that leveraged the core values of the technology, such as interconnection between people. That's where AI programming is now. We are at AI 1.0 Programming models will come out that are primarily oriented to the strengths of AI programming - but first someone needs to think them up. And they will not only be surprising but absolutely mind blowing. It's entirely possible that in the future the entire internet has an AI sublayer that does all the work of systems talking to systems and people simply don't have anything to do with it. It's possible that the nocode movement is the foundation of the next great innovations in programming. Existing programming models are like the company brochures of web 1.0 - our old, hard, human centric way of doing things. Programming literally should be as easy as talking to the computer and telling it how to shapoe and change the software and it should happen right there in real time on your screen. It should be an iterative, interactive process that anyone can do. It should be more like painting and less like arcane wizardy that can be performed only by the programmer priests who have spent years learning. Even the most advanced software should be capable of being built this way. In the 1980's I worked for a startup company run by a super smart guy Peter Dunai, who once said "a revolution in programming is coming". I thought I'd seen that revolution already with the Internet, modern programming languages and StackOverflow, but now I realise none of that many anything compared to what the true revolution is going to look like. AI 2.0 will come and I can't wait.

bombolo 4y ago
> AI 2.0 will come and I can't wait. It might happen in a decade or more.

mjb 4y ago

It's interesting that Hillel, a huge formal methods and specification advocate, sees this as a proofreading problem and not a specification one. I can't say I agree.

hwayne 4y ago
I'd be interested in hearing more about your disagreement! If I had to reconcile this post with my overall shameless FM promotion, I'd say that "proofreading is hard" is the human limit that is amplified by AI use, and more specification is a potential solution. But also, historically, an expensive one, so I don't know if that's going to adopted enough to fully offset the drawbacks of mass AI adoption.

adversaryIdiot 4y ago

I haven't had much luck using these AI's past surface level stuff.

hwayne 4y ago
They are godsent at getting a TikZ drawing jumpstarted.

twawaaay 4y ago

Anybody worrying that programmers loose jobs stop worrying. All the automation in the world does not replace the need for people to debug problems. Even more interestingly, the more something is automated the more time is spent on solving the issues when they happen. Finding a simple mistake in a very complex automation can be a challenging project. But... if you have been programming Cobol your entire life you might want to start learning some newer technology.

justincredible 4y ago
[dead]
bryanrasmussen 4y ago
You think the programming AIs are going to be trained on Cobol?
- agentwiggles 4y ago
  I mean, I have no ability to check it for correctness, but ChatGPT is certainly capable of writing Cobol. Just asked it for an implementation of Dijkstra's algorithm and it looks reasonable enough to me. Why _wouldn't_ the programming AIs be trained on Cobol?
  - bryanrasmussen 4y ago
    Ok, I would think you probably would not want to devote a lot of resources to training for Cobol given the smaller niche it represents which I figured would make it less useful.

bambax 4y ago

> Might as well share a wishlist. [1/] A ChatGPT-style AI that can only reply with links to reference docs, libraries, or wikipedia articles That already exists. It's called an effing search engine. But for some reason people have decided, as of ten days ago, that they're not good enough anymore. As for why? I really have no clue. But they still work extremely well.

Bjartr 4y ago
> As for why? I really have no clue. But they still work extremely well. I've found Google to be significantly less useful than it used to be. Largely because I have to jump through hoops to convince it to show me what I'm actually looking for. ChatGPT is already much better often enough that it can turn 20+ minutes of faffing about with Google into one minute or less of just asking for what I want and getting it. Yeah, it's wrong sometimes, but it still manages to point me in the right direction often enough to be worth the need for extra vigilance

mordae 4y ago

Hahaha, try asking GPT-3 about electronic engineering. --- Transcript of IRC channel #electronics: Mordae: Hi. So I was designing a transformerless LED bulb and I was wondering about the filter capacitor. What happens when the load (one of the LEDs) fail? Rampant: Hi Mordae. It really depends on the design, but the general idea is that the filter capacitor will help prevent spikes and other issues caused by the failing LED. It can also help with voltage regulation. --- Actual answer: Light bulb needs a bulky electrolytic cap. Those are usually not 400V tolerant. You are looking at 60V at best at the required sizes. Losing the load will swiftly charge up the capacitor to 325V. But well before that happens, there are going to be fireworks: https://www.youtube.com/watch?v=6WUxgmMDts4 https://www.youtube.com/watch?v=6WUxgmMDts4 I believe that there are eventually going to be useful mentors for bunch of fields, but right now it's crazy dangerous to blindly follow its recommendations.

whartung 4y ago
I'm sorry, is there enough context in the original question as presented to come up with your "actual answer"? I mean, I don't know, I'm not an EE, lucky I can make a light switch work. But given the dearth of details in the query, is that AI answer grotesquely wrong? Just seems like generic advice.
- Dylan16807 4y ago
  > I'm sorry, is there enough context in the original question as presented to come up with your "actual answer"? I'd say there is. "Transformerless" implies a specific design, especially when you mention there's a filter capacitor. > is that AI answer grotesquely wrong? Yes. It's fine to say the filter cap helps with "spikes and other issues", but not ones caused by a failing LED. It took half a true sentence and made it into a false sentence. In this design the filter cap is something you give protection to much more than something that provides protection.
deleted 4y ago
[deleted]
Bjartr 4y ago

theknocker 4y ago

[dead]

jmull 4y ago

I read this but actually not sure what the author is worried about. I guess they feel proofreading AI-written code is harder than writing code? That seems like a leap to me. I don't expect AI-assisted coding to be a panacea, and I'm sure there are ineffective ways to do it. But it seems like there are effective ways too. So, sure, there is work to do to figure out how to use these tools in the best way. But that's not anything to worry about, IMO. It's opportunity.

seletskiy 4y ago

With the new advancements in AI, it's expected that there will be more code being generated by programs rather than written by programmers. Making sure these programs deliver accurate and error free code is a difficult task. The challenge here lies in effectively proofreading AIs so that any bugs are caught before they reach end users. Good tools such as unit tests can help but may not be enough to make sure AI generated code is flawless. As the use of AI increases, we must look to develop better methods for ensuring quality assurance on these programs if they are to truly be useful.

avdempsey 4y ago
In my career I've seen resources pulled away from QA teams (sometimes completely disbanding). It would be interesting to see things swing the other way.
zokier 4y ago
> it's expected that there will be more code being generated by programs rather than written by programmers. That has probably been case since the 80s; nearly all code running on your CPU is generated by a program ("compiler") from a textual description ("source")
- vitiral 4y ago
  That's not what is meant. Programming languages are basically huge macro systems that expand to assembly, but the code is still written by a human. Layers of abstraction are a problem too, but they are a different problem.
Panzer04 4y ago
This feels GPT generated.

pphysch 4y ago

The author begins by giving a number of test cases, like f(35)==6, for which the AI provides a candidate f, but the author doesn't actually implement the test cases, is surprised when the AI is wrong, and laments the resulting burden of proofreading. Just implement those tests, for heaven's sake!

majormajor 4y ago
Implementing a correct and comprehensive set of test cases for code is more time consuming and often more difficult than writing the code itself. (The comprehensive part especially - how do you prove to yourself you've covered all the necessary edge cases?) "Reading code is harder than writing it" "Debugging code is harder than writing it" These are pretty common statements, going back years. They should make the lack of a silver bullet here very obvious. (Yes, most software is already broken in various ways - my worry is that this will just make it more broken, e.g. non-technical management saying "why do we need to do as much QA, we have AI writing the code now!?")
- pphysch 4y ago
  Yes, writing good tests is hard. My point is that he already has the relevant test cases. The hard work is done.
fragmede 4y ago
even easier: ask the LLM for those test cases!
hwayne 4y ago
To clarify, Bertrand Meyer is the one who gave the test cases and accepted the wrong answer without verifying them. I'm the one who lamented the resulting burden of proofreading.

yen223 4y ago

Programming AIs worry me because I think they can get better at programming faster than I can get better at programming

qualudeheart 4y ago
I once coined an aphorism: “Deepmind can train a new model faster than you can go back to grad school.”

guluarte 4y ago

it's like alexa, siri, etc. I'd rather do the stuff myself than talking nonsense with an "ai"

tonyhb 4y ago

> A ChatGPT-style AI that can only reply with links to reference docs, libraries, or wikipedia articles This is exactly what we've built at Inngest so that you can learn our SDK: https://www.inngest.com/ai-personalized-documentation https://www.inngest.com/ai-personalized-documentation. It can only reply with SDK examples, an explanation, and links to relevant guides. Soon, we're revamping our docs to highlight sections relevant only to your use case - you'll be able to type what you want to do with Inngest and be pointed straight to the relevant guides and docs. We trust developers to write the code, and only use GPT to assist and point the right way. It's a good compromise, and it's the most immediately applicable use that I can see!

hwayne 4y ago
I know this is getting downvoted for being an ad but this is exactly what I was talking about, so thank you I will look into it!

aib 4y ago

What worries me are the low-grade programmers. By delegating their work to tools like ChatGPT (and doing so always a level or two beyond the current abilities of the tools) they will manage to take us to newer lows.

Existenceblinks 4y ago
Underrated. It already happened in frontend space. Job market is already rotten by incompetent tech leaders who aren't immune to all the bullshit and bought all the hype from VC-backed bigco.
deleted 4y ago
[deleted]

layer8 4y ago

The “almost correct” aspect will be especially fun when AIs are used to refactor code. I was thinking about how a future legacy codebase might look like after a decade of having AI-generated code added to it, and then of course you would want the AI to apply some refactorings from time to time. One might argue that it won’t be a problem with proper tests. But do coders want to spend most of their time proofreading AI-generated code and taking care of proper test coverage? That doesn’t sound very appealing.

plasticchris 4y ago
Why can’t the AI write tests too? Then you can have 100% coverage of your (probably) correct code…
- saulpw 4y ago
  That's even worse, because now the tests are wrong too! I asked it for a list of test cases for a particular function I wanted to write, and every single one of the cases it presented was incorrect. So it was worse than useless, now I had to check its work and then do the work myself anyway.
- layer8 4y ago
  It can, but you have to proofread those too.
- noobermin 4y ago
  Why not generate an AI that can take any code and decide whether it has any errors or not?
  - dskrvk 4y ago
    An AI that tells whether or not a program will halt.
ParetoOptimal 4y ago
> The “almost correct” aspect will be especially fun when AIs are used to refactor code. Hope your function has all invariants covered with one or more property tests!

valcron1000 4y ago

If reading is harder than writing code, why bother with tools that deal with the later instead of the former?

vitiral 4y ago

My worry is code complexity explosion. Once we start using AI to add features to infrastructure (OS's, compilers, data serialization, etc) it will quickly run away from our ability to understand it. It has already run away, it will now be fucking _flying_.

Verdex 4y ago
I expect that before my career is over that I will be given more than one project to fix that the AI stopped being able to deliver features for. I imagine it's going to be one of those laugh so you don't cry affairs.
- turtleyacht 4y ago
  I like it. We're generating "COBOL" debt for our future brethren as we solve present problems with the most modern tools we had.
- AnimalMuppet 4y ago
  Maybe laugh because you're going to get paid for a really long time. (And then, yeah, cry because of what you're going to have to do to earn the money...)
layer8 4y ago
The code might start to resemble biological systems, which are a mess of complexity.
- ModernMech 4y ago
  In the future, programs will develop genetic disorders, and will need to be diagnosed by programmers with the equivalent of medical degrees.
  - 6177c40f 4y ago
    Unfortunately, doctors might be replaced by diagnostic AIs. It'll be AIs all the way down by then.

agentultra 4y ago

It’s a bit of a nihilistic view that programmers are merely slow boiler plate auto completion systems and our jobs will be replaced by AI. We already write more code than we understand given how lowly the industry regards formal methods and how divided it is even on testing. It is not surprising to me that Bertrand Meyer could miss that error in the code! A former colleague of mine challenged his grad students to implement an error-free version of binary search. He gave them the specification and verified each one: the majority of the submissions had errors. It’s a marvel that computers and software work so well. And instead of tackling the problem of programming, as Dijkstra put it, we’re going to automate writing more code. Great. See you in a few years when we can’t even keep up with the deluge of error prone programs. Although we have gotten this far with very few proofs and models. Who knows, maybe we’ll just tolerate it? It would be nice to have code synthesis work on larger examples from deep specifications. And models that understand proof tactics and can speed up the process of formalizing proofs. That would be nice.

sigstoat 4y ago
> A former colleague of mine challenged his grad students to implement an error-free version of binary search. He gave them the specification and verified each one: the majority of the submissions had errors. I was in a class on Isabelle where we had to write and prove correct a binary search function. As I recall, everyone admitted that their first function didn’t work.
- renewiltord 4y ago
  Java's was broken till '06
  - OnionBlender 4y ago
    https://en.wikipedia.org/wiki/Binary_search_algorithm#Implementation_issues https://en.wikipedia.org/wiki/Binary_search_algorithm#Implem... People often do (L+R)/2 but that can overflow. Using L+(R-L)/2 avoids the overflow problem.

stevage 4y ago

These are really solid concerns. This shift from "creator" to "supervisor" is happening in lots of AI areas - look even at how self-driving cars are currently expecting their human drivers to act as supervisors, which they're bad at. Also, on this point: > We can make proofreading easier with better tooling. Unit tests are a means of “proofreading”: we can catch AI errors automatically with tests. The unit tests will definitely be written with AI as well. If that AI and the coding AI are wrong in the same way, we will have very subtle and hard to catch errors.

gqcwwjtg 4y ago
That problem sounds just like when I write tests for my own code. I don’t know if it’s totally avoidable.
- NateEag 4y ago
  It just now occurred to me that a natural solution to this problem could be to move past code review to "other teammate writes the tests for your PR." You owe them an additional beer at the next happy hour for each test they write you wind up pushing a fix for.

Waterluvian 4y ago

Oh wow. That IS what I want. An AI that can offer a code review and suggest things.

thebigspacefuck 4y ago

To be fair, I’ve seen humans do worse.

amelius 4y ago

Before we have AIs that can program, we will have AIs that can do computer administration tasks, like solving dependency problems with software installations etc. and compiling and installing stuff. So before we have a tool that can do that, I'm not worried.

zshrdlu 4y ago
It would be fun to see one attempt a Gentoo install.

jongjong 4y ago

Of course, AI models are often wrong. Even if they were right in the sense that they gave you exactly what you asked for, they may still be wrong in the sense that their interpretation of your requirements may not be what you intended; they are not great at filling the gaps if there is any context missing. The same amount of time it takes to provide an AI with all the necessary, nuanced context to produce an optimal solution could have been used to actually produce the desired optimal solution yourself. The hard part of software development is not coding, it's the amount of critical thinking and synthesizing required to produce optimal solutions. I'm optimistic about AIs because they may finally prove once and for all that software engineers cannot be commodified as easily and thoroughly as most other white collar roles. It may be easier to commodify executives who make a small number of abstract high level decisions than it is to commodify developers who make many extremely nuanced, detailed decisions every day. For sure, AI will replace certain kinds of developers, but not all kinds, at least not for some time. I think senior developers and architects will be among the last people in the firing line.

sramam 4y ago
> I think senior developers and architects will be among the last people in the firing line. And soon there will be no junior developers left to rise up to their ranks... So "last" in more ways than one!

m3kw9 4y ago

Would AIs that write unit test help test their own code, they will write code till it passes and link all the modules together

m3kw9 4y ago
They already knew this so I didn’t start the AI coding apocalypse

ccvannorman 4y ago

<< Laughs >> Oh wait, you're serioous?! << Laughs harder >> AI is great for making syntactically correct code; and so is copy-pasting the first result that works after a low quality search. It still takes real minds to make code that is worth anything, and there will always be buffoons making code that doesn't work. Except the HN website, of course. ;-)

gmt2027 4y ago

The work of a master craftsman using the most basic hand tools is still leagues ahead of any amateur with advanced tools, automation, fancy CNC, and lasers. Mass production is where the tech would favour that amateur. We are surrounded by cheap mass-produced disposable products and are well aware of their low quality. No gourmet chef should be threatened by the efficiency of McDonald's. Fortunately, our software systems are custom artifacts with zero marginal cost of reproduction. AI will make competent programmers more productive and take expert programmers to new levels. One transformation will come when a single developer can do the work of a team of ten today. Like solo content creators who no longer need large studios to finance and organise massive production teams and expensive equipment, we'll see more independent developers and small teams creating innovative software with some artistic character. Without the drag of management and upfront financing, the threshold for success will be much lower. Another will be a market for lemons. Employers will have to find ways to establish competence at building high quality, understandable and maintainable systems. AI can already pass Leetcode-style tests so these are clearly not the right measures. This is my best optimistic take on things. The rest of the time I'm worried the AI is coming for our jobs.