7 ms·
Generative AI and the widening software developer knowledge gap
- Zetobal 3y agoThat's not a new problem I've met more than one react dev that can't write vanilla js.
- ActionHank 3y agoHeard someone refer to Angular as a new language the other day.
- Izkata 3y agoOverheard years ago when I was out at lunch: "Luckily I don't have to use javascript, just jquery."
- tiborsaas 3y agoWait till they ask why would they ever need that :)
- sho 3y ago> AI tools are already automating large parts of the software engineering role Are they? I hear a lot of breathless claims about this but no-one I know is automating anything with AI. It's a useful suggestion engine/menial task helper, but anyone who's used it for more than a day knows its severe limitations. Hell, i wish it was a lot more powerful. I have a bunch of bulma CSS to convert to tailwind. Both ChatGPT and Gemini Ultra fail miserably at what you'd think is a simple, by-the-numbers task, but nope, their results are so problematic it ends up taking more time to fix than just doing it yourself in the first place. They're useful "sparring partners", but that's about it...
- ToucanLoucan 3y ago> Are they? I hear a lot of breathless claims about this but no-one I know is automating anything with AI. It's also impossible to take those aforementioned breathless claims at face value with the absolutely absurd hype around AI. I'm not even arguing good or bad here, I'm just saying no one will shut up about this. And it's still not AI, it's machine learning. Nothing wrong with that but when a product's name itself is already bullshit...
- eurekin 3y agoAnd it's not even online machine learning, depending how does one see the very limited in context learning. The offline learning is also a multi million dollar endeavor atm for most capable models. That does not take into account the cost of acquiring gpus. Somehow all of this is missing from most articles. I'm very happy people start pointing that out
- danielbln 3y agoTechnically ML is the process that is used to create a model/product, which in turn was coined "AI". Not great naming, but are people really still dying on that hill? "Cloud" is also a stupid name for a managed datacenter/IAAS/PAAS, it's a collection of water droplets after all. Yet here we are. What value does it add to lament these naming fails, especially after it's long permeated the public Zeitgeist.
- ToucanLoucan 3y agoBecause AI already was in the public zeitgeist as a term for something that is decidedly not what is being offered by companies like OpenAI, and beyond it's meaning in the zeitgeist, the wording itself, Artificial Intelligence, belies much more than these models actually are, which is generative models. They are not intelligent in any way a layman would call intelligent. > "Cloud" is also a stupid name for a managed datacenter/IAAS/PAAS "Cloud computing" though as referred to datacenters makes a degree of sense, certainly more than calling GPT an actual mind. And it seems pretty clear to me that based on the way these companies and their CEOs are leaning into fantasies of Skynet and such, that they're not only encouraging the conflation, but using it to sell the product. And by all means, sell your product, but if you open that conversation by lying to my face, don't expect to close a sale.
- haliskerbas 3y agoCompletely agree here. I use it daily both ChatGPT Plus and CoPilot, it’s great as a slightly better google to fire off queries to while I investigate something else. But it does sometimes fail miserably and I wish it was better.
- ActionHank 3y agoI've actually started asking people when they comment that it has been an amazing life-changing thing, how did it improve things. 90% of the time was people who are code adjacent that don't often write "code" who then use Copilot to write a one off script. They have no idea the size of the chasm between a one off script and writing production code.
- CuriouslyC 3y agoAs someone who finds ChatGPT life changing and amazing, here's what you're missing: 1. I'm a strong software engineer, but there are a lot of places where I have a conceptual understanding, and can describe the properties of a correct solution, but I don't know the "jargon" of libraries. ChatGPT lets me describe the properties of the solution declaratively and it can write the library code that implements that behavior for commonly used libraries. 2. My specialty is machine learning/AI, so there are things like Devops where I have an understanding of docker/terraform/etc and can use them but I don't know the best way to utilize those to take full advantage of them. ChatGPT can tell me about devops best practices and how to use the tools/structure the configs to avoid a lot of footguns. This also applies to areas like front-end, where you can ask about front end best practices given a particular project's properties, and it can even update existing code to implement the things it tells you about. 3. ChatGPT is great at writing boring code like tests, documentation, etc. Not writing that code makes coding much more engaging in general.
- sefrost 3y agoI have also noticed that the biggest proponents in my organisation are those that don’t code or code very little! Personally, as I have used the product more I have found it more useful especially with boilerplate type of code - especially unit tests.
- sandspar 3y agoDo you have any friends who are in school right now? A friend of mine told me about a web development bootcamp of his where nearly every student relied so heavily on ChatGPT that few of them learned how to do the course material. The article makes the point that generative AI is a pipeline threat, one which heavily affects juniors and students.
- lumost 3y agoI saw an ad recently "You'll never code as well as ChatGPT." I suspect we'll see some changes to both how and why people code, and it's entirely likely that application types which are difficult for AI to code will become scarcer. Similar dynamics have happened with cloud/mobile/commodity hardware/PC. Every time there is a change, there are folks who claim that the old way was easier and had been working forever.
- sandspar 3y agoI suspect as much as well. If AI makes it easier to produce certain products, then advertisers will do their damndest to create demand for those products. And if AI makes production cheaper, safer, and faster, then planners will orient their constructions and designs around making use of those efficiencies. Similar to how the widespread adoption of the car changed our cities, so too will AI change what we build, AI's limitations and all. I wouldn't be surprised if, at some point, we have a mental model of "things AI can build" and companies just assume that the majority of our solutions to problems will come from within that set. It doesn't even matter if those AI solutions don't fit the problems very well, as long as they're cheap. The smaller set of "things we still have to get humans to build" will cost more and therefore be de-emphasized and seen as a last resort. An equivalent might be producing features that introduce legal vulnerabilities. You can imagine lots of planners saying, "Can we do without that feature?"
- 082349872349872 3y agoI like this graphic of draft (working) horses vs. other (companion/sport) horses over time (1800-2010 in France): https://www.researchgate.net/publication/338480301/figure/fig1/AS:845430833283085@1578577826802/Evolution-of-the-horse-population-in-France-from-1800-to-2010-translated-from-French.ppm https://www.researchgate.net/publication/338480301/figure/fi... It shows clearly that farm (and transport) automation wiped out the draft horse population, but that the other horse population, having traversed the same bottleneck (ca. 1970), has rebounded. (and in fact there were absolutely more "others" in 2010 than there were in 1840) Moral: try to be the sort of programmer a little girl (little AI?) would love? > It had come to [Squealer's] knowledge, he said, that a foolish and wicked rumour had been circulated at the time of Boxer's removal. Some of the animals had noticed that the van which took Boxer away was marked "Horse Slaughterer," and had actually jumped to the conclusion that Boxer was being sent to the knacker's. It was almost unbelievable, said Squealer, that any animal could be so stupid. Surely, he cried indignantly, whisking his tail and skipping from side to side, surely they knew their beloved Leader, Comrade Napoleon, better than that? But the explanation was really very simple. The van had previously been the property of the knacker, and had been bought by the veterinary surgeon, who had not yet painted the old name out. That was how the mistake had arisen. —EAB
- empath-nirvana 3y agoare you using chatgpt 4 for that? The free version of chatgpt isn't close to as good as the latest paid version. Even copilot is better than the free version of chatgpt.
- orwin 3y agoI use the enterprise version of copilot, its basically a great google in all cases, a good stackOverflow in some cases, and a fantastic docstring writer. Probably a bit more usefull if used with a language that is both known and strict like java, but if you want to do python or script with it, one advice: don't. You'll go nowhere, slowly if you have to use sightly complex libraries like the very well known pydantic.
- englishspot 3y agofor simple tasks/code that I already know the answer to but just need a refresher, ChatGPT is great. when I'm dealing with something really mysterious, it starts to break down no matter the prompt, and I'm left with trying to figure it out for myself.
- emmender2 3y agoJoel Spolsky had a great article on leaky abstractions. LLMs for code are leaky abstractions. They work many-a-time. But when they break, good luck fixing it. yann-lecunn also put it well. If something works 95% of the time, and you compose it 10 times, it only works 59% of the time. In the real world of software engineering, we cannot build on something that works 95% of the time reliably. And LLM apologists will immediately say that code written by humans has bugs too. Of course it does.
- mrbungie 3y agoWouldn't this apply to any kind of numerical or logical model (i.e. a model learned over preexisting data) used in a software solution?
- emmender2 3y agoindeed, and that goes into the heart of it. ie, things we construct by the computer are deterministic. the turing machine (and other equivalent models like the lambda calculus etc) being the canonical machine that models our computations. Arguably, all human knowledge is symbolic and determistic - even though it may model probabilistic phenomena.
- BoxOfRain 3y agoMy use-case for LLMs in writing software is 'documentation I can talk to'. I don't have ChatGPT write my code but it's a much faster and more pleasant alternative to wading knee-deep in spam on forums or grepping through badly-written documentation you need a degree in exegesis to interpret.
- didntcheck 3y agoYeah, I've almost never used ChatGPT to generate something for me - neither text nor code - but I find it very useful for answering questions and helping me understand concepts when it would be hard to google my query And that's extremely valuable. One of the reasons people still go for in-person formal education, despite the wealth of free resources for many topics online, is the ability to ask questions back and get fast, personalized responses. We're now finally getting that by computer. It's also good for those "I could work this out myself, but it would take 20m of research, or bugging someone else to get a quick answer" situations - no need to feel guilty about bugging a computer
- kjkjadksj 3y agoIt seems a little dangerous to put so much trust in a model known to output incorrect information
- kjkjadksj 3y agoHow do you verify chatgpt is giving you a correct interpretation of these docs without spending the time to read them yourself anyhow?
- BoxOfRain 3y agoA rather militant approach to testing for the most part, even if I don't know how to do something off the top of my head I know what I want my code to be doing so in my opinion having decent testing in place limits the risk of a ChatGPT misinterpretation causing harm. I've not done away with documentation wholesale either, if I'm in doubt I'll revert to the usual way of doing things. I use them in conjunction with each-other as well, I can use ChatGPT as a 'really fuzzy find' for things like 'what's the name of x function in a library that sounds a bit like y whose name I've forgotten' then go and look at the docs in a way search engines are generally quite bad at.
- CuriouslyC 3y agoThey are, and if you're not observing this you're observing programmers so mediocre they don't know to use good tools, or programmers who are so incredible at their micro-niche that there's a good chance they're one of the sources for what ChatGPT would try to spit back at them. ChatGPT can write great documentation, great test data, great utility functions and passable tests. It's great at refactors that are conceptually simple but just messy enough to be outside the ability of IDE refactor tools. It's great at taking things like models or data and generating code that does something for each model or row in the data given a template. It's great at taking stub classes and fleshing them out based on usage examples/comments. ChatGPT's coding ability is directly related to your choice of language, how you structure your code base and how you prompt. If your code base is a bunch of small strongly typed functions written in a popular language like typescript/java that get composed together and you feed it the defs it needs in context, it can be surprisingly good. If you're writing 300+ line imperative spaghetti (particularly in a less popular language) it's gonna lose the script unless you ask it to make small changes.
- caeril 3y ago> Are they? Yes, they are. We are automating every bit of codegen we possibly can, and will likely never hire another frontend developer ever again. Our backend/systems developers will be slowly migrating to overseeing the codegen services themselves. > Both ChatGPT and Gemini Ultra fail miserably You need to build tooling around the APIs. Copy-paste into ChatGPT is obviously not what anybody is talking about. You determine what you need for the system prompt for each semi-generic use-case (which takes some trial-and-error), you determine the optimal chunk size for each cycle, then you scaffold your RAG (vector database, LSP queries, etc, whatever you need for your scenario), then you specify your loss function and your prompt modifier, maybe add in some static analyzers or formal verifiers for additional loss inputs, and then you execute over your requirements. In some cases, you might also fine-tune an open-source base model if you need to, and integrate it into your pipeline. Anyone talking about "automating" software development by ctrl+v-ing into a ChatGPT textarea doesn't really understand what is going on in the real world.
- sandspar 3y agoAligns with my latest thinking as well. Even if an AI can write better stories than a human can, there are futures where human-written stories are even more valuable for all that. The supply and demand for human writers may remain good, if many would-be writers give up or never learn the craft, and there's still a niche demand for "primitive" human written stories. To me it seems like a good idea to bet on craft, albeit craft serving a niche market.
- dragonelite 3y agoI can imaging a future where human writers will claim they don't use AI to write their stories while they most definitely are using AI. Kind of like the scourge of natural fitness influencers using gear.
- sandspar 3y agoThat's funny and likely. I wonder if some enterprising author will sell hand-written copies of their novel.
- kjkjadksj 3y agoWhat you describe is how the world already works with ghostwriters. Music too. Drake would have his writers living in tents in a basement until the project was done.
- BoxOfRain 3y agoI think you have a good point here; there's a reason that deep into the digital age there's still plenty of analogue cameras, valve amplifiers, vinyl records, and all manner of 'obsolete' things like that being made.
- kjkjadksj 3y agoThere are so many artists that thrift stores are filled with artwork. It was never about having a quantity of art. Provenance is everything. Its why silicon valley execs don’t drink boxed wine, and why ai art isn’t going to change much of the art world.
- typon 3y agoCopy pasting from stackoverflow has been a meme for almost a decade now. I don't think that destroyed software engineering. Generative AI goes further than that, but fundamentally don't alter software engineering, at least at their current levels of power. You can't give them high level tasks - you have to handhold them. Yes verification is easier than creativity, but I haven't written a sorting algorithm, a Web server, an LRU cache or something similar from scratch in a long time. I reach for a library I know works well. If the LLM helps me write code at a higher level of abstraction than libraries, so what? It's going to be a leaky abstraction just like libraries currently are - the best engineers who can traverse the abstraction hierarchy will still be in demand.
- emmender2 3y ago100% agree. the key is to be able to traverse the abstraction hierarchy all the way from the physics of the hardware to the end-user, and that arguably is what any engineer must learn.
- mfrommil 3y agoMy prediction is that to be competitive, companies will eventually need to rely on AI-produced code to some extent or risk being slower and less efficient than competitors. It would be like not using email or messaging and only using snail mail for all written communication. But AI is nowhere close to perfect now, and will have flaws for a long time. Having AI write code is like having a so-so junior engineer, who can complete the task, but makes mistakes, so needs their code reviewed closely. And is unable to architect anything complex, that still needs to be done by the leads/managers/senior folks. So more and more of the simple, low complexity coding tasks will be done by AI, while the value of importance of senior engineers will be as high as ever, since they need to oversee the AI's outputs. What I wonder is how junior engineers, who will be starting their careers out as more expensive or weaker coders to AI, will get the experience necessary to become the senior engineers that need to guide/review the AI's work?
- graphe 3y agoLike they used cheap Indian coders to save money and now everything is better? It'll be a test of try to pay less and probably a bunch of spaghetti code fixes after.
- ActionHank 3y agoThere are an unsettling number of NA startups that seem to aggressively hire remote workers or contractors in Asia\Africa and then a couple mid to senior people in NA timezones. They build their product and then just seem to fizzle out. My guess that the technical debt and lack of talent retention kills them.
- mywittyname 3y agoTechnical people are more valuable than the code they produce. The good ones have domain expertise and can guide product development in a way that takes advantages of emerging tech, long before it becomes mainstream, and positions the company to capitalize on market movements sooner. Business that don't have technical leaders in their senior leadership aren't tech companies. They will fall behind quickly because they are busy chasing what's hot yesterday vs what's going to be hot soon. 10 years ago, I worked for a F500 company that fired their research team who was working on generative AI (and made solid progress) because senior leadership was all about investing in blockchain. Remember blockchain? I'm willing to bet those same leaders are all about the "AI future" now that it's in the magazines. But the problem is they are competing with companies who saw the value of generative AI years ago, before it was mainstream. Lucky for them, the company has enough money to buy the startup competition for a few billion.
- jordanpg 3y ago> but are we sleepwalking into a future where no one knows the basics? Maybe the basics are just being redefined. Perhaps there are some irreducible concepts to which programming can be reduced. Maybe the programmers of the future need only know those concepts and not anything at all about code. In science fiction, in the far future, people just tell computers what to do and they do it. Presumably, these computers were built by humans at some point but were able to improve themselves. And that was the end of the need for programmers. Maybe we just need to move the ball down the field just far enough for the programmers of today to unburden the people of tomorrow from needing to learn how to code at all.
- apantel 3y agoI think the premise is wrong. ChatGPT has supercharged my understanding of Java by exposing me to so many different parts of the language and development patterns that I didn’t know before because it literally knows ALL of Java. If you look at the code it generates in order to check it for errors and understand what it has given you, you’ll be constantly learning about the language.
- ReptileMan 3y agoBut you know enough java to be able to accumulate new knowledge. That is the critical barrier. To know enough in a subject so you are able to learn on your own
- drooby 3y agoI see some of my coworkers going in the wrong direction and pulling others down around them. I have a coworker that is both not a native English speaker and not a great engineer. He is leaning too heavily on Copilot and CharGPT and our code reviews are getting longer and more difficult because he is producing more code that traverses in the wrong direction. Meanwhile - our CTO is saying "if you're not leveraging gen AI in your coding, you're going to get left behind" Sigh.... Gen AI has a multiplying effect. And that goes in either direction or seems.
- deleted 3y ago[deleted]
- ReptileMan 3y agoTLDR - if you are over 40 now, you will be like the guys nowadays that know FORTRAN and COBOL. >A COBOL programmer, tired of all the extra work and chaos caused by the impending Y2K bug, decides to have himself cryogenically frozen for a year so he can skip all of it. >He gets himself frozen, and eventually is woken up when several scientists open his cryo-pod. >"Did I sleep through Y2K? Is it the year 2000?", he asks. >The scientists nervously look at each other. Finally, one of them says "Actually, it's the year 9999. We hear you know COBOL."
- magwa101 3y ago[dead]
- perrygeo 3y agoWriting a prompt with sufficient detail to generate the "correct" code is not a priori easier than just writing the code! There's no logical reason to think it should. Most genAI users intuitively known this and attempt to strike a balance between prompt engineering and traditional debugging techniques. Which suggest to me that, if anything, knowledge is even more critical since you're now describing the problem precisely in a natural language (hard!) and debugging/editing/reading code written by someone else (even harder!).