9 ms·
To AI or not to AI
- qweiopqweiop 1y agoNot asking for feedback is the killer for me. Even most junior developers will ask for more information if they don't have enough context/confidence to complete a task.
- hatefulmoron 1y agoI often ask Claude to scan through the code first and then come back with questions related to the task. It sometimes comes back with useful questions, but most of the time it acts like a university student looking for participation marks from a tutorial; choosing questions to signal understanding rather than be helpful.
- qudat 1y agoGitHub just released spec-kit which I think attempts to get the human more involved in the spec/planning/task building process. You basically instruct the LLM to generate these docs and you tweak them to flesh it out all fix mistakes. Then you tell the LLM to work on a single task at a time, reviewing in small chunks.
- mattmanser 1y agoThat's how everyone is already using Claude Code, it's not GitHub's idea. You go into plan mode, get it to iterate on the idea, then ask it to make (and save) a to do list md. Then you get it to run through the to-do list, checking tasks off as it goes.
- jrexilius 1y agoI have taken to appending "DO NOT START WRITING CODE." to almost every prompt.. I try to get it to analyze and ask questions and summarize what its going to do first, and even then it will sometimes ignore that and jump into writing (the wrong) code. A big part of the wrangling seems to be getting it to analyze or reason before charging down a wrong path.
- mattmanser 1y agoIf you use Claude Code you can go into plan mode, where it doesn't write code, you can back and forth.
- jmkni 1y agoGemini is terrible for this
- chrischen 1y agoI don't understand why people take bad coding practices and just let AI run with it and then expect nothing but poor quality code. Nothing about the AI revolution here changes how good software has always been written. Write tests, use a typed language, review code. If you have good patterns, good procedures, AI fits right in and fills in the blanks perfectly. Poor AI results tend to be the pot calling the kettle black.
- righthand 1y agoI think bad practices will always be around as most code on Guthub was probably written with bad practices. The well is poisoned.
- rhetocj23 1y agoOut of curiosity, how do you think the model producers will/would attempt to discern what information on the web is of high quality vs not so high quality (i.e. poisonous)? Akin to clean/drinkable water vs dirty water/harmful water in the well.
- righthand 1y agoI don’t think they will. The well will always have some level of poison if all information has bias and intent. Bad software design is bad grammar, it’s ubiquitous.
- rhetocj23 1y agoThe real problem is the quality of knowledge and education of engineers. No amount of AI fixes that (until you complete displace labour as an input that is).
- dweinus 1y agoI mean, it sounds like reviews and tests are already their standard practice, and explicitly part of their AI practice. So it should have worked, right?
- jmkni 1y agoI can relate to a lot of this. Where I find AI most useful is getting it to do tasks I already know how to do, but would take time. If you understand the problem you are trying to solve well enough to explain it to the LLM, you can get good results, you can also eyeball the outputted code and know right away if it's what you are after. Getting it to do things you don't know how to do is where it goes off the rails IMO
- serbuvlad 1y agoExactly. AI is your intern, not your contractor.
- Yoric 1y agoMore precisely, AI is your intern who won't improve during the internship, ever.
- jmkni 1y agoAnd is also a pathological liar lol
- Yoric 1y agoYes, that, too.
- rhetocj23 1y agoJust to play devils advocate - why do you think this is so? And if you have a compelling thesis, why hasn't this spread to the investing community?
- ewoodrich 1y agoThey mean it doesn't learn from experience/mistakes after spending time on your codebase. There are workarounds like documenting in CLAUDE.md/CODEX.md/.roorules which dump it back into the active context but are hit and miss in my experience. It's definitely better than nothing but Claude still routinely ignores important directives whenever it's in the right mood.
- xpil 1y agoMy preferred approach in similar situations is to ask an LLM for an initial solution or code snippet, then take over manually - no endless prompt tweaking, just stop prompting and start coding. Finally (optionally), I let the LLM do a final pass to review my completed solution for bugs, optimizations, etc. The key win is skipping the prompt refinement loop, which is (A) tedious and time-consuming, and (B) debilitating in the long run.
- esafak 1y agoFiller content. > Our marketing director (that’d be me) said that if we don’t write something about it, we will be left behind... Write when you have something to say. What was I supposed to learn here?
- wintermutestwin 1y agoI recently spent over an hour trying to get ChatGPT to give me some pretty simple rsync commands. It kept giving me command line parameters that didn't work on the version of rsync on my mac. With ~50% of the failures, it would go down troubleshooting rabbit holes and the rest of the time it would "realize" that it was giving incorrect version responses. I tell it to validate each parameter against my version moving forward and it clearly doesn't do that. I am sure I could have figured it out on my own in 5 mins, but I couldn't stop watching the trainwreck of this zeitgeist tech wasting my time doing a simple task. I am not a coder (much), but I have to wonder if my experience is common in the coding world? I guess if you are writing code against the version that was the bulk of its training then you wouldn't face this specific issue. Maybe there are ways to avoid this (and others) pitfall with prompting? As it is, I do not see at all how LLMs could really save time on programming tasks without also costing more time dealing with its quirks.
- BryanLegend 1y agoI've recently been misled by ChatGPT a lot as well. I think it's the router. I'm on the free plan so I assume they're just being tight with the GPU cycles.
- wintermutestwin 1y agoI am on a $20 plan and using the "thinking" version of 5.
- iamnotagenius 1y ago[dead]
- Yoric 1y agoPretty much my experience, yes.
- chrisweekly 1y agoWhy involve an LLM at all, if you're looking up docs for a particular tool like rsync?
- amelius 1y agoYes, this is how I use AI. Indeed, self-invented abstractions are a bridge too far for AI. You have to keep it close to the path already walked before by thousands of developers. This makes AI more of a search engine on steroids than anything else.
- rhetocj23 1y agoChatGPT is literally just a search engine that Google shouldve moved to, but waited because they didnt want to touch their assets in place.
- falcor84 1y agoI agree that Google were too slow to move, but entirely disagree with the first part. ChatGPT is very much not a "search engine". Arguably it is an "Answer engine", but more so, it is a conversational partner - I almost never use ChatGPT to just get one response; the real benefit is being able to follow up with it until I'm satisfied. It's an entirely different medium of interaction as compared to search engines.
- amelius 1y agoGPT is orders of magnitude more expensive to run, though.
- richardguerre_ 1y agoI don't like vibe coding as much as actual coding, but the biggest improvement in my workflow was shifting left even more. Now I dedicate at least one session to just writing a spec file, and have it ask me clarifying questions on my requirements and based on what it finds in the codebase and online. I ask it to also break down the implementation plan in phases with a checklist for each phase. I then start at least one new session per phase and make sure to nail down that phase before continuing. The nice thing is if it gets annoying to vibe code it, I or someone on my team can just use the spec to implement things.
- rel_ic 1y agoUsing AI to improve facebook ads... y'all are the breakers from the Dark Tower series.
- zenmac 1y ago>There is never enough context. We learned quickly that the more context we provided and the smaller the issues, the better the results. However, no matter how much context we provided, the AI would still mess things up because it didn’t ask us for feedback. AI would just not understand if it didn’t have enough information to finish a task, it would assume, a lot, and fail. Is it me or does it feels like the genie in the bottle thing. Remember a TV show where the guy and his friend sat down with the Genie like a lawyer to make sure every angle is covered (going to spare you the details here). That is what it feels like interacting to a LLM sometimes.
- mnky9800n 1y agoI think the AI acts like that shitty coworker that is super smart but never tells you what they are thinking so they are likely capable of doing whatever you want them to do but working on a team is asking too much and they are apparently not capable of doing that. Because AI promises that you can interact with it like it is human because of it's chat capabilities but it never ever does something like, "hey, i don't understand this part, can you tell me more of what you mean here?"
- rsynnott 1y agoWell, except that (I think I know the scene you're referring to), it ultimately worked. The LLM, on the other hand, will feel no need to stick to its 'promises'. (Really the genie is closer to the traditional sci-fi AI in that it's legalistic and rules-bound; the LLM very much isn't.)
- jwpapi 1y agoThis aligns very well with my experience and what I’ve commented on other posts!
- BinaryIgor 1y ago"We just don’t think we will incorporate AI to do more than that, given the current state of things. We will, however, keep an eye in case the technology changes fundamentally." I wonder whether LLMs are capable of doing more; probably, we need another paradigm for that; still, they are very, very useful when used right
- falcor84 1y ago> I wonder whether LLMs are capable of doing more I don't see how that is a question. I come up with new ideas to improve the LLM-based tools I'm using at least once a day, and the vast majority of these are plain engineering changes that I could do on my own if I wanted to put the effort into it. I think that even if God comes down from heaven to prevent us from further training the LLMs themselves (if God is listening to Yudkowsky's prayers), then we would still have a good few decades of extensively improving the capabilities of LLM-based tools to extract a massive amount of further productivity by just building better agentic wrappers and pipelines, applying proper software development and QA methodology.
- BinaryIgor 1y agoTrue!
- avighnay 1y agoI decided to adopt AI assisted coding for a recent project. Not sure what defines 'vibe coding' but the process I ended up was a iterative interaction at a measured pace. I used Gemini AI studio for this and I was very pleased at the result and decided to open source it. I have completely captured and documented the development transcript. Personally it has give me considerable productivity boost. My only irritation was the unnecessarily over politeness that AI adopts in My take is AI yields good ROI when you know exactly what you want at the end of the process and when you want to compare and contrast decision choices during the process. I have used it for all artifacts of the project: - Core code base - Test cases - Build scripts - Documentation - Sample apps - Utilities Transcript - https://gingerhome.github.io/gingee-docs/docs/ai-transcript/index.html https://gingerhome.github.io/gingee-docs/docs/ai-transcript/... Project - https://github.com/gingerhome/gingee https://github.com/gingerhome/gingee
- ASalazarMX 1y ago"AI, but verify" -- Winston Churchill (alternate universe)
- christoff12 1y ago> However, no matter how much context we provided, the AI would still mess things up because it didn’t ask us for feedback. The proceeding without clarifying or asking questions thing really grinds my gears.