7 ms·
The demo on the homepage for the completion of the findMaxElement function is a good example of what is to come. Or maybe where we are at now? The six lines of
by st3fan 2y ago
The demo on the homepage for the completion of the findMaxElement function is a good example of what is to come. Or maybe where we are at now?
The six lines of Python suggested for that function can also be replaced with a simple “return max(arr)”. The suggested code works but is absolute junior level.
I am terrified of what is to come. Not just horrible code but also how people who blindly “autocomplete” this code are going to stall in their skill level progress.
You may score some story points but did you actually get any better at your craft?
- MaKey 2y agoThe silver lining is that the value of your skills is going up.
- shcheklein 2y agoOn the other hand it might become a next level of abstraction. Machine -> Asm -> C -> Python -> LLM (Human language) It compiles human prompt into some intermediate code (in this case Python). Probably initial version of CPython was not perfect at all, and engineers were also terrified. If we are lucky this new "compiler" will be becoming better and better, more efficient. Never perfect, but people will be paying the same price they are already paying for not dealing directly with ASM.
- MVissers 2y agoYup! No goal to become a programmer– But I like to build programs. Build a rather complex AI-ecosystem simulator with me as the director and GPT-4 now Claude 3.5 as the programmer. Would never have been able to do this beforehand.
- saurik 2y agoI think there is a big difference between an abstraction layer that can improve -- one where you maybe write "code" in prompts and then have a compiler build through real code, allowing that compiler to get better over time -- and an interactive tool that locks bad decisions autocompleted today into both your codebase and your brain, involving you still working at the lower layer but getting low quality "help" in your editor. I am totally pro- compilers and high-level languages, but I think the idea of writing assembly with the help of a partial compiler where you kind of write stuff and then copy/paste the result into your assembly file with some munging to fix issues is dumb. By all means, though: if someone gets us to the point where the "code" I am checking in is a bunch of English -- for which I will likely need a law degree in addition to an engineering background to not get evil genie with a cursed paw results from it trying to figure out what I must have meant from what I said :/ -- I will think that's pretty cool and will actually be a new layer of abstraction in the same class as compiler... and like, if at that point I don't use it, it will only be because I think it is somehow dangerous to humanity itself (and even then I will admit that it is probably more effective)... but we aren't there yet and "we're on the way there" doesn't count anywhere near as much as people often want it to ;P.
- sdesol 2y ago> Machine -> Asm -> C -> Python -> LLM (Human language) Something that you neglected to mention is, with every abstraction layer up to Python, everything is predictable and repeatable. With LLMs, we can give the exact same instructions, and not be guaranteed the same code.
- 12345hn6789 2y agoassuming you have full control over which compiler youre using for each step ;) What's to say LLMs will not have a "compiler" interface in the future that will reign in their variance
- sdesol 2y ago> assuming you have full control over which compiler youre using for each step ;) With existing tools, we know if we need to do something, we can. The issue with LLMs, is they are very much black boxes. > What's to say LLMs will not have a "compiler" interface in the future that will reign in their variance Honestly, having a compiler interface for LLMs isn't a bad idea...for some use cases. What I don't see us being able to do is use natural language to build complex apps in a deterministic manner. Solving this problem would require turning LLMs into deterministic machines, which I don't believe will be an easy task, given how LLMs work today. I'm a strong believer in that LLMs will change how we develop and create software development tools. In the past, you would need Google and Microsoft level of funding to integrate natural language into a tool, but with LLMs, we can easily have LLMs parse input and have it map to deterministic functions in days.
- theptip 2y agoI’m not sure why that matters here. Users want code that solves their business need. In general most don’t care about repeatability if someone else tries to solve their problem. The question that matters is: can businesses solve their problems cheaper for the same quality, or at lower quality while beating the previous Pareto-optimal cost/quality frontier.
- thesz 2y ago
- vages 2y agoIt may be a “level of abstraction”, but not a good one, because it is imprecise. When you want to make changes to the code (which is what we spend most of our time on), you’ll have to either (1) modify the prompt and accept the risk of using the new code or (2) modify the original code, which you can’t do unless you know the lower level of abstraction. Recommended reading: https://ian-cooper.writeas.com/is-ai-a-silver-bullet https://ian-cooper.writeas.com/is-ai-a-silver-bullet
- svachalek 2y agoKeep in mind that this is the stupidest the LLM will ever be and we can expect major improvements every few months. On the other hand junior devs will always be junior devs. At some point python and C++ will be like assembly now, something that’s always out there but not something the vast majority of developers will ever need to read or write.
- sdesol 2y ago> we can expect major improvements every few months. I'm not sure this is grounded in reality. We've already seen articles related to how OpenAI is behind schedule with GPT-5. I do believe things will improve over time, mainly due to advancements in hardware. With better hardware, we can better brute force correct answers. > junior devs will always be junior devs Junior developers turn into senior developers over time.
- smcnally 2y ago> I'm not sure this is grounded in reality. We've already seen articles related to how OpenAI is behind schedule with GPT-5. Progress by Google, meta, Microsoft, Qwen and Deepseek is unhampered by OpenAI’s schedule. Their latest — including Gemini 2.0, Llama 3.3, Phi 4 — and the coding fine tunes that follow are all pretty good.
- sdesol 2y ago> unhampered by OpenAI’s schedule Sure, but if the advancements are to catch up to OpenAI, then major improvements by other vendors are nice and all, but I don't believe that was what the commenter was implying. Right now the leaders in my opinion are OpenAI and Anthropic and unless they are making major improvements every few months, the industry as a whole is not making major improvements.
- smcnally 2y agoOpenAI and Anthropic are definitely among the leaders. Playing catch-up to these leaders' mind-share and technology is some of the motivation for others. Calling the progress being made in the space by Google (Gemini), MSFT (Phi), Meta (llama), Alibaba (Qwen) "nice and all" is a position you might be pleasantly surprised to reconsider if this technology interests you. And don't sleep on Apple and AMZ - In the space covered by Tabby, Copilot, aider, Continue and others, capabilities continue to improve considerably month-over-month. In the segments of the industry I care most about, I agree 100% with what the commenter said w/r/t expecting major improvements every few months. Pay even passing attention to huggingface and github and see work being done by indies as well as corporate behemoths happening at breakneck pace. Some work is pushing the SOTA. Some is making the SOTA more widely available. Lots of it is different approaches to solving similar challenges. Most of it benefits consumers and creators looking use and learn from all of this.
- runeblaze 2y agoI mean you can treat it as just a general pseudocode-ish implementation of an O(n) find_max algorithm. Tons of people use Python to illustrate algorithms. (Not to hide your point though -- people please review your LLM-generated code!)
- tippytippytango 2y agoThis is self correcting. Code of this quality won't let you ship things. You are forced to understand the last 20%-30% of details the LLM can't help you with to pass all your tests. But, it also turns out, to understand the 20% of details the LLM couldn't handle, you need to understand the 80% the LLM could handle. I'm just not worried about this, LLMs don't ship.
- tyingq 2y agoIn the case where it write functionally "good enough" code that performs terribly, it rewards the LLM vendor...since the LLM vendor is also often your IaC vendor. And now you need to buy more infra.
- grahamj 2y agoThis needs to be shouted from the rooftops. If you could do it yourself then LLMs can be a great help, speeding things up, offering suggestions and alternatives etc. But if you’re asking for something you don’t know how to do you might end up with junk and not even know it.
- cootsnuck 2y agoBut if that junk doesn't work (which it likely won't for any worthwhile problem) then you have to get it working. And to get it working you almost always have to figure out how the junk code works. And in that process I've found is where the real magic happens. You learn by fixing, pruning, optimizing. I think there's a whole meta level of the actual dynamic between human<>LLM interactions that is not being sufficiently talked about. I think there's, potentially, many secondary benefits that can come from using them simply due to the ways you have to react to their outputs (if a person decides to rise to that occasion).
- 999900000999 2y agoLLMs also love to double down on solutions that don't work. Case in point, I'm working on a game that's essentially a website right now. Since I'm very very bad with web design I'm using an LLM. It's perfect 75% of the time. The other 25% it just doesn't work. Multiple LLMs will misunderstand basic tasks. Let's add properties and invent functions. It's like you had hired a college junior who insists their never wrong and keeps pushing non functional code. The entire mindset is whatever it's close enough, good luck. God forbid you need to do anything using an uncommon node module or anything like that.
- smcnally 2y ago> LLMs also love to double down on solutions that don't work. “Often wrong but never in doubt” is not proprietary to LLMs. It’s off-putting and we want them to be correct and to have humility when they’re wrong. But we should remember LLMs are trained on work created by people, and many of those people have built successful careers being exceedingly confident in solutions that don’t work.
- deltaburnt 2y agoSo now you have an overconfident human using an overconfident tool, both of which will end up coding themselves into a corner? Compilers at least, for the most part, offer very definitive feedback that act as guard rails to those overconfident humans. Also, let's not forget LLMs are a product of the internet and anonymity. Human interaction on the internet is significantly different from in person interaction, where typically people are more humble and less overconfident. If someone at my office acted like some overconfident SO/reddit/HN users I would probably avoid them like the plague.
- smcnally 2y agoA compiler in the mix is very helpful. That and other sanity checks wielded by a skilled engineer doing code reviews can provide valuable feedback to other developers and to LLMs. The knowledgeable human in the loop makes the coding process and final products so much better. Two LLMs with tool usage capabilities reviewing the code isn't as good today but is available today. The LLMs overconfidence is based on it spitting out the most-probable tokens based on its training data and your prompt. When LLMs learn real hubris from actual anonymous internet jackholes, we will have made significant progress toward AGI.
- ripped_britches 2y agoThe most underrated thing I do on nearly every cursor suggestion is to follow up with “are there any better ways to do this?”.
- smcnally 2y agoA deeper version of the same idea is to ask a second model to check the first model’s answers. aider’s “architect” is an automated version of this approach. https://aider.chat/docs/usage/modes.html#architect-mode-and-the-editor-model https://aider.chat/docs/usage/modes.html#architect-mode-and-...
- avandekleut 2y agoI always ask it to "analyze approached to achieve X and then make a suggestion, no code" in the chat. Then a refinement step where I give feedback on the generated code. I also always try to give it an "out" between making changes and keeping it to same to stave off the bias of action.
- cootsnuck 2y agoYea, the "analyze and explain but no code yet" approach works well. Let's me audit its approach beforehand.
- csomar 2y ago> I am terrified of what is to come. Don't worry. Like everything else in life, you get what you pay for.
- 55555 2y agoI used to know things. Then they made Google, and I just looked things up. But at least I could still do things. Now we have AI, and I just ask it to do things for me. Now I don't know anything and I can't do anything.
- deltaburnt 2y agoI feel like I've seen this comment so many times but actually genuine. The cult like dedication is kind of baffling.
- nyarlathotep_ 2y agoProgrammers (and adjacent positions) of late strike me as remarkably shortsighted and myopic. Cheering for remote work leading to loads of new positions being offered overseas opposed to domestically, and now loudly celebrating LLMs writing "boilerplate" for them. How folks don't see the consequences of their actions is remarkable to me.
- yellowapple 2y agoIn both cases, you get what you pay for.
- worble 2y ago> The suggested code works but is absolute junior level This isn't far the current status quo. Good software companies pay for people who write top quality code, and the rest pay juniors to work far above their pay grade or offshore it to the cheapest bidder. Now it will be offloaded to LLM's instead. Same code, different writer, same work for a contractor who knows what they're doing to come and fix later. And so the cycle continues.
- dizhn 2y agoAnybody care to comment whether the quality of the existing code influences how good the AI's assistance is? In other words, would they suggest sloppy code where the existing code is sloppy and better (?) code when the existing code is good?
- cootsnuck 2y agoWhat do you think? (I don't mean that in a snarky way.) Based on how LLMs work, I can't see how that would not be the case. But in my experience there are nuances to this. It's less about "good" vs "bad"/"sloppy" code and more about discernable. If it's discernably sloppy (i.e. the type of sloppy a beginning programmer might do which is familiar to all of us) I would say that's better than opaque "good" code (good really only meaning functional). These things predict tokens. So when you use them, help them increase their chances of predicting the thing you want. Good comments on code, good function names, explain what you don't know, etc. etc. The same things you would ideally do if working with another person on a codebase.
- sirsinsalot 2y agoReminds me of the 2000s outsourcing hype. I made a lot of money cleaning up that mess. Entire projects late, buggy, unreadable and unmaintainable. Business pay big when they need to recover from that kind of thing and save face to investors.
- generalizations 2y ago> people who blindly “autocomplete” this code are going to stall in their skill level progress AI is just going to widen the skill level bell curve. Enables some people to get away with far more mediocre work than before, but also enables some people to become far more capable. You can't make someone put in more effort, but the ones who do will really shine.
- shihab 2y agoI think that example says more about the company that chose to put that code as a demo in their homepage.
- aja12 2y agoAs a cybersecurity professional (as in, the more cybersecurity problems there are, the less likely I am to ever find myself out of a job), I'm rooting for AI!