11 ms·
AI Blindspots – Blindspots in LLMs I've noticed while AI coding
- ezyang 2y agoHi Hacker News! One of the things about this blog that has gotten a bit unwieldy as I've added more entries is that it's a sort of undifferentiated pile of posts. I want some sort of organization system but I haven't found one that's good. Very open to suggestions!
- datadrivenangel 2y agoMaybe organize them more clearly split between observed pitfalls/blindspots and prescriptions. Some of the articles (Use automatic formatting) are Practice forward, while others are pitfall forward. I like how many of the articles have examples!
- smusamashah 2y agoHow about listing all if these on 1 single page? Will be easy to navigate/find.
- ezyang 2y agoThey are listed on one page right now! Haha
- elicash 2y agoThey're indexed on one page, but you can't scan/scroll through these short posts without clicking because the content itself isn't all on a single page, at least not that I can find. (I also like the other idea of separating out pitfalls vs. prescriptions.)
- lelandfe 2y agoWordpress’s approach to this is giving each post a short description in addition to the main content. The excerpt gets displayed on the main list, which helps both to grok the post and keep the list from becoming unwieldy.
- smusamashah 2y agoAs in, all content on one page where the link just takes you to appropriate heading on the same page. These days you can do a lot on a single html.
- cookie_monsta 2y agoSome sort of navigation would be nice a prev/next or some way to avoid having to go back to the links page all the time. All of the pages that I visited were small enough that you could probably wrap them them <details> tags[1] and avoid navigation altogether [1] https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details https://developer.mozilla.org/en-US/docs/Web/HTML/Element/de...
- rav 2y agoMy suggestion: Change the color of visited links! Adding a "visited" color for links will make it easier for visitors to see which posts they have already read.
- sfink 2y agoWhen I saw the title, I knew what this was going to be. It made me want to immediately write a corresponding "Human Blindspots" blog post to counteract it, because I knew it was going to be the usual drivel about how the LLMs understand <X> but sometimes they don't quite manage to get the reasoning right, but not to worry because you can nudge them and their logical brains will then figure it out and do the right thing. They'll stop hallucinating and start functioning properly, and if they don't, just wait for the next generation and everything will be fine. I was wrong. This is great! I really appreciate how you not only describe the problems, but also describe why they happen using terminology that shows you understand how these things work (rather than the usual crap that is based on how people imagine them to work or want them to work). Also, the examples are excellent. It would be a bunch of work, but the organization I would like to see (alongside the current, not replacing it, because the one-page list works for me already) would require sketching out some kind of taxonomy of topics. Categories of ways that Sonnet gets things wrong, and perhaps categories of things that humans would like them to do (eg types of tasks, or skill/sophistication levels of users, or starting vs fixing vs summarizing/reviewing vs teaching, or whatever). But I haven't read through all of the posts yet, so I don't have a good sense for how applicable these categorizations might be. I personally don't have nearly enough experience using LLMs to be able to write it up myself. So far, I haven't found LLMs very useful for the type of code I write (except when I'm playing with learning Rust; they're pretty good for that). I know I need to try them out more to really get a feel for their capabilities, but your writeups are the first I've found that I feel I can learn from without having to experience it all for myself first. (Sorry if this sounds like spam. Too gushing with the praise? Are you bracing yourself for some sketchy URL to a gambling site?)
- joshka 2y agoWhat about adding a bit more structure and investing in a pattern language approach like what you might find in a book by Fowler or a site like https://refactoring.guru/ https://refactoring.guru/. You're much of the way there with the naming and content, but could refactor the content a bit better into headings (Problem, Symptoms, Examples, Mitigation, Related, etc.) You could even pretty easily use an LLM to do most of the work for you in fixing it up. Add a short 1-2 sentence summary[1] to each item and render that on the index page. [1]: https://gohugo.io/content-management/summaries/ https://gohugo.io/content-management/summaries/
- mncharity 2y agoIn "Keep Files Small", there seems a lacuna: "for example, on Cursor 0.45.17, applying 55 edits on a 64KB file takes)."
- incognito124 2y agoThere was a blog posted here which had a slider for scoring different features (popularity, personal choice, etc). The rankings updated live with slider moves. Also, take a look at https://news.ycombinator.com/item?id=40774277 https://news.ycombinator.com/item?id=40774277
- Sxubas 2y agoTo be honest, current format worked perfectly for me: I ended up reading all entries without feeling something was off in how they were organized. I really really liked that each section had a concrete example, please don't remove that for future entries. Thank you for sharing your insights! Very generous.
- jonas21 2y agoMaybe you should ask Claude.
- datadrivenangel 2y agoAlmost all of these are good things to consider with human coders as well. Product managers take note! https://ezyang.github.io/ai-blindspots/requirements-not-solutions/ https://ezyang.github.io/ai-blindspots/requirements-not-solu...
- fizx 2y agoThe community seems rather divided as to whether these are intrinsic, or we solve these with today's tech, and more training, heuristics and workarounds.
- mystified5016 2y agoRecently I've been writing a resume/hire-me website. I'm not a stellar writer, but I'm alright, so I've been asking various LLMs to review it by just dropping the HTML file in. Every single one has completely ignored the "Welcome to nginx!" Header at the top of the page. I'd left it in half as a joke to amuse myself but I expected it would get some kind of reaction from the LLMs, even if just a "it seems you may have forgotten this line" Kinda weird. I even tried guiding them into seeing it without explicitly mentioning it and I could not get a response.
- SparkyMcUnicorn 2y agoHave you tried "Let's get this production ready" as a prompt for this or any other coding tasks? Sometimes when I ask for "production ready" it can go a bit too far, but I've found it'll usually catch things like this that I might miss.
- eschaton 2y agoWhy would you expect it to “get some kind of reaction?” That strongly implies that you perceive what the LLM doing as “understanding” the tokens you’re feeding it, which *is not something LLMs are capable of*.
- deleted 2y ago[deleted]
- namaria 2y agoIt didn't ignore it. There just wasn't any pattern in the training data about responding to such a line. Having the mental model that the text you feed to an LLM influences the output but is not 'parsed' as 'instructions' helps understand its behaviors. The website GP linked is searching for a zoo of problems and missing the biology behind. LLMs don't have blindspots, they don't reason nor hallucinate. They don't follow instructions. They pattern match on high dimensional vector spaces.
- antasvara 2y agoThis highlights a thing I've seen with LLM's generally: they make different mistakes than humans. This makes catching the errors much more difficult. What I mean by this is that we have thousands of years of experience catching human mistakes. As such, we're really good at designing systems that catch (or work around) human mistakes and biases. LLM's, while impressive and sometimes less mistake-prone than humans, make errors in a fundamentally different manner. We just don't have the intuition and understanding of the way that LLM's "think" (in a broad sense of the word). As such, we have a hard time designing systems that account for this and catch the errors.
- tharkun__ 2y agoI don't think that's universally true. We have different humans with different levels of ability to catch errors. I see that with my teams. Some people can debug. Some can't. Some people can write tests. Some can't. Some people can catch stuff in reviews. Some can't. I asked Sonnet 3.7 in Cursor to fix a failing test. While it made the necessary fix, it also updated a hard-coded expected constant to instead be computed using the same algorithm as the original file, instead of preserving the constant as the test was originally written. Guess what? Guess the number of times I had to correct this from humans doing it in their tests over my career! And guess where the models learned the bad behavior from.
- __MatrixMan__ 2y agoI agree. I've been been struck by how remarkably understandable the errors are. It's quite often something that I'd have done myself if I wasn't paying attention to the right thing.
- fn-mote 2y ago> Some people can debug. Some can't. Some people can write tests. Some can't. Wait… really? No way do I want to work with someone who can’t debug or write tests. I thought those were entry stakes to the profession. People whose skills you use in other ways because they are more productive? Maybe. But still. Clean up after yourself. It’s something that should be learned in the apprentice phase.
- teraflop 2y ago> I had some test cases with hard coded numbers that had wobbled and needed updating. I simply asked the LLM to keep rerunning the test and updating the numbers as necessary. Why not take this a step farther and incorporate this methodology directly into your test suite? Every time you push a code change, run the new version of the code and use it to automatically update the "expected" output. That way you never have to worry about failures at all!
- ezyang 2y agoIn fact, the test framework I was using at the time (jest) did in fact support this. But the person who had originally written the tests hadn't had the foresight to use snapshot tests for this failing test!
- diggan 2y agoI don't know if your message is a continuation of the sarcasm (I feel like maybe no?), but I'm pretty sure parent's joke is that if you just change the expected values whenever the code changes, you aren't really effectively "testing" anything as much as "recording" outputs.
- Mc91 2y agoOne thing I do is go to Leetcode, see the optimal big O time and space solutions, then give the LLM the Leetcode medium/hard problem, and limit it to the optimal big O time/space solution and suggest the method (bidirectional BFS). I ask for the solution in some fairly mainstream modern language (although not Javascript, Java or Python). I also say to do it as compact as possible. Sometimes I reiterate that. It's just a function usually, but it does not always compile. I'd set this as a low bar for programming. We haven't even gotten into classes, architecture, badly-defined specifications and so on. LLMs are useful for programming, but I'd want them to clear this low hurdle first.
- bongodongobob 2y agoYou're using a shitty model then or are lying. 4o one or two shotted the first 12 days of advent of code for me without anything other than the problem description.
- xigency 2y agoAhh yes. Because that other AI model is 100% perfect. Gee whiz. Man the people working on these machines, selling them, and using them lack the very foundational knowledge of information theory. Let alone understanding the humanities and politics. Subjectively speaking, humans will never be satisfied with any status quo. Ergo there is no closed-form solution to meeting human wants. Now disrupting humans' needs, for profit, that is well understood. Sam Altman continuing to stack billions after allegedly raping his sister.
- bongodongobob 2y agoWell then live like someone would before the industrial revolution. The hypocrisy of saying shit like this on "The Internet" is always funny to me.
- xigency 2y agoIt's not ironic that technologists use very barebones and minimal websites with minimal automation. It's telling. I already ditched my smartphone last month because it was 100% spam, scammers, and bots giving me notifications. Apparently it's too much to ask to receive a well-informed and engaged society without violence and theft. So I don't take anyone at their word and even less so would trust automated data mining, tracking and profiling that seeks to guide my decision making. Buy me a drink first SV before you crawl that far up my ass.
- logicchains 2y agoI found Gemini Flash Thinking Experimental is almost unusable in an agent workflow because it'll eventually accidentally remove a closing bracket, breaking compilation, and be unable to identify and fix the issue even with many attempts. Maybe it has trouble counting/matching braces due to fewer layers?
- ezyang 2y agoYeah, Sonnet 3.5/3.7 are doing heavy lifting. Maybe the SOTA Gemini models would do better, I haven't tried them. Generating correct patches is a funny minigame that isn't really solved, despite how easy it is to RL on.
- logicchains 2y agoo3-mini works well enough for me, it makes mistakes but generally it can always fix them eventually. Interestingly I found even if I include the line numbers as comments in the code it sees, it still often gets the line numbers wrong for edits (most often, off by one errors, likely due to it mixing up whether the line numbers are inclusive or exclusive). What does work a bit better is asking it to provide regex matching the first and last line of what it wants to replace, along with nearby line numbers (so if there are multiple matches in that file for the regex, it gets the right one).
- diggan 2y ago> Maybe the SOTA Gemini models would do better, I haven't tried them As I had to upgrade my Google Drive storage like a month ago, I gave them all a try. Short version: If you have paid plan with OpenAI/Claude already, none of them come even close, for coding at least. I thought I was trying the wrong models at first, but after confirming it seems like Google is just really far behind.
- woah 2y agoStrange to read this and the parent comment, since Cursor has never made a single error applying patches for me. The closest it's come is when the coding model adds unnecessary changes which of course is a completely different thing.
- taberiand 2y agoBased on the list, LLMs are at a "very smart junior programmer" level of coding - though with a much broader knowledge base than you'd expect from even a senior. They lack bigger-picture thinking, and default to doing what is asked of them instead of what needs to be done. I expect the models will continue improving though, I feel like most of it comes down to the ephemeral nature of their context window / the ability to recall and attach relevant information to the working context when prompted.
- threeseed 2y agoI wonder if people who say LLMs are a smart junior programmer have ever used LLMs for coding or actually worked with a junior programmer before. Because for me the two are not even remotely comparable. If I ask Claude to do a basic operation on all files in my codebase it won't do it. Half way through it will get distracted and do something else or simply change the operation. No junior programmer will ever do this. And similar for the other examples in the blog.
- zarathustreal 2y agoSince when is “do something on every file in my codebase” considered coding?
- 2y ago
- dataviz1000 2y agoAre you using Cursor? I'm using Github Copilot in VSCode and I'm wondering if I will get more efficiency from a different coding assistant.
- dsabanin 2y agoYou will. Cursor is much further along on the journey of building an actually powerful AI coding system. Since they are smaller, they can afford to iterate more quickly and experiment with a much tighter feedback loop.
- diggan 2y agoI've tried Copilot, Aider and Cursor and the best I've found is to just use the various chat interfaces. I sometimes throw hundreds of lines straight in there, and the models seem to understand the full context much better than any "LLM Editor" I've tried so far. Then different models are good for different things (obvious maybe). For example, O1 Pro is miles ahead any models when it comes to overall architecture, R1 is great for finding nasty bugs and Sonnet great for small and fast feature additions/modifications with strict requirements.
- ezyang 2y agoI have used Cursor and my own MCP codemcp. Cursor has a lot of nice QoL that you can't get from an MCP package; the TAB is really good for traditional coding. Haven't used copilot so I don't have a comparison there. Definitely use agent mode.
- hooloovoo_zoo 2y agoIt doesn't matter. They're all thin layers on functionally equivalent models. Stick with whatever text editor you prefer.
- boredtofears 2y agoGreat read, I can definitely confirm a lot of these myself. Would be nice to see this aggregated into some kind of "best practices" document (although hard to say how quickly it'd be out of date).
- submeta 2y ago> Preparatory refactoring > Current LLMs, without a plan that says they should refactor first, don’t decompose changes in this way. They will try to do everything at once. Just today I leaned the hard way. I had created an app for my spouse and myself for sharing and reading news-articles, some of them behind paywalls. Using Cursor I have a FastAPI backend and a React frontend. When I added extracting the article text in markdown and then summarizing it, both using openai, and when I tasked Cursor with it, the chaos began. Cursor (with the help of Claude 3.7) tackled everything at once and some more. It started writing a module for using openai, then it also changed the frontend to not only show the title and url, but also the extracted markdown and the summary, by doing that it screwed up my UI, deleted some rows in my database, came up with as module for interacting with Openai that did not work, the ectraction was screwed, the summary as well. All of this despite me having detailed cursorrules. That‘s when I realized: Divide and conquer. Ask it to write one function that workd, then one class where the function becomes a method, test it, then move on to next function. Until every piece is working and I can glue them together.
- AustinDev 2y agoUse claude 3.5 if you have detailed instructions that you want it to follow. I've found over many hours of using these models that 3.7 loves to go off-script no matter how many rules you provide.
- pomatic 2y agoThis is the way, small bite-sized pieces of the elephant. Unfortunately it means you do need to understand programming concepts, composition and to a lesser degree, architecture. On the positive side - these are new tools, and we need to learn how to work with them. They do have the power to nX times the person who has a bit of knowledge and can also adapt to their ways.
- noname120 2y agoAider has an architect mode exactly for that purpose
- irskep 2y agoOne thing I do to avoid this problem is to ask the LLM to make a plan and write it to a doc. Then in a new session, have it read the doc and tell it to implement a specific part of the plan. It saves you almost as many brain cycles as just having the LLM do it all in one go, but gives you direct control over how things happen and how much gets done at once. You can also tweak the plan by hand or iterate on it with the LLM.
- colonCapitalDee 2y ago> Preparatory Refactoring says that you should first refactor to make a change easy, and then make the change. The refactor change can be quite involved, but because it is semantics preserving, it is easier to evaluate than the change itself. > In human software engineering, a common antipattern when trying to figure out what to do is to jump straight to proposing solutions, without forcing everyone to clearly articulate what all the requirements are. Often, your problem space is constrained enough that once you write down all of the requirements, the solution is uniquely determined; without the requirements, it’s easy to devolve into a haze of arguing over particular solutions. > When you’re learning to use a new framework or library, simple uses of the software can be done just by copy pasting code from tutorials and tweaking them as necessary. But at some point, it’s a good idea to just slog through reading the docs from top-to-bottom, to get a full understanding of what is and is not possible in the software. > The Walking Skeleton is the minimum, crappy implementation of an end-to-end system that has all of the pieces you need. The point is to get the end-to-end system working first, and only then start improving the various pieces. > When there is a bug, there are broadly two ways you can try to fix it. One way is to randomly try things based on vibes and hope you get lucky. The other is to systematically examine your assumptions about how the system works and figure out where reality mismatches your expectations. > The Rule of Three in software says that you should be willing to duplicate a piece of code once, but on the third copy you should refactor. This is a refinement on DRY (Don’t Repeat Yourself) accounting for the fact that it might not necessarily be obvious how to eliminate a duplication, and waiting until the third occurrence might clarify. These are lessons that I've learned the hard way (for some definition of "learned", these things are simple but not easy), but I've never seen them phrased to succinctly and accurately before. Well done OP!
- duxup 2y ago"Preparatory Refactoring says that you should first refactor to make a change easy, and then make the change. " Amen. I'll be refactoring something and a coworker will say "Wow you did that fast." and I'll tell them I'm not done... those PRs were just to prepare for the final work. Sometimes after all my testing I'll even leave the "prepared" changes in production for a bit just to be 100% sure something strange wasn't missed. THEN the real changes can begin.
- admiralrohan 2y agoEven in the age of Vibe coding, I always try to learn as much as possible. For example, yesterday I was working with the Animation library Motion which I never worked earlier. I used the code suggested by AI but at least picke 2-3 basic animation concepts while reviewing the code. Kind of unfocused passive learning I always tried even before AI.
- worldsayshi 2y ago> Even in the age of Vibe coding, I always try to learn as much as possible. Even? It kind of has become easier than ever to learn new ways to code? Just as it opens up building things that you previously wouldn't because of time constraints, you can now learn how to X in language Y in a few minutes instead of hours. Although I suppose it may be easier than ever for the brain to think that "I can look this up whenever so I might just forget about it".
- akomtu 2y agoLLMs aren't AI. They are more like librarians with eidetic memory: they can discuss in depth any book in the library, but sooner or later you notice that they don't really understand what they are talking about. One easy test for AI-ness is the optimization problem. Give it a relatively small, but complex program, e.g. a GPU shader on shadertoy.com, and tell it to optimize it. The output is clearly defined: it's an image or an animation. It's also easy to test how much it's improved the framerate. What's good is this task won't allow the typical LLM bullshitting: if it doesn't compile or doesn't draw a correct image, you'll see it. The thing is, the current generation of LLMs will blunder at this task.
- ezyang 2y agoThe thing is that, as many junior engineers can attest, randomly blundering around can still give you something useful! So you definitely can get value out of AI coding with the current generation of models.
- xigency 2y agoI can't wait to see the future of randomly blundered tech as we continue to sideline educated, curious, and discerning human engineers from any salaried opportunity to apply their skills. I've been working as a computer programmer professionally since I was 14 years old and in the two decades since I've been able to get paid work about ~50% of the time. Pretty gnarly field to be in I must say. I rather wish I had studied to be a dentist. Then I might have some savings and clout to my name and would know I am helping to spread more smiles. And for the cult of matrix math if >50% of people are dissatisfied with the state of the something, don't be surprised if a highly intelligent and powerful entity becoming aware of this fact engages in rapid upheaval.
- deleted 2y ago[deleted]
- kleton 2y agoMost of the things are applicable to the current top models, but he frequently references Claude sonnet, which is not even above the fold on the leaderboard
- prmph 2y agoWhich models in your opinion are on the leaderboard?
- kleton 2y agohttps://lmarena.ai/?leaderboard https://lmarena.ai/?leaderboard
- cootsnuck 2y agoThat leaderboard is not for a coding use case -- just general chat. Click on the web dev leaderboard they have and Claude has the top spots. It is well known that Claude 3.7 sonnet is the go-to choice for many people for coding right now.
- lukev 2y agoThis is exceptionally useful advice, and precisely the way we should be talking about how to engage with LLMs when coding. That said, I take issue with "Use Static Types". I've actually had more success with Claude Code using Clojure than I have Typescript (the other thing I tried.) Clojure emphasizes small, pure functions, to a high degree. Whereas (sometimes) fully understanding a strong type might involve reading several files. If I'm really good with my prompting to make sure that I have good example data for the entity types at each boundary point, it feels like it does a better job. My intuition is that LLMs are fundamentally context-based, so they are naturally suited to an emphasis on functions over pure data, vs requiring understanding of a larger type/class hierarchy to perform well. But it took me a while to figure out how to build these prompts and agent rules. A LLM programming in a dynamic language without a human supervising the high-level code structure and data model is a recipe for disaster.
- oglop 2y agoI just talk to an LLM like it’s a person who is smart, meaning I expect it to be confidently wrong now and then but I don’t have to worry about hurting its feelings. They are remarkably similar to people, though others seems to not think that so maybe it is a case of some people finding them easier to work with compared to others. I wonder what drives that. Maybe it’s the difference between a person who thinks life unfolds before then vs the person who views life as a bundle, with each day a fold making up your experience and through this stack you discern the structure which is your life, which sure seems how these things work.
- torginus 2y agoI have one more - LLMs are terrible at counting and arithmetic - if your code gen relies on cutting off the first two words of a constant string - you better check if you need to cut off 12 characters like the LLM says. If it adds 2 numbers, it might be suspect. If you need it to decode a byte sequence, where getting the numbers from the exact right position is necessary.. you get the idea. Took me a day to debug my LLM-generated code - and of course, like all fruitless and long debugging sessions, this one started with me assuming that it can't possibly get this wrong - yet it did.
- icelancer 2y agoMy favorite eval are based on cv2 dlib work, primarily face_recognition. Up until 3.7 Sonnet, it consistently got things wrong in terms of face embeddings and general coding practices around them. 3.7 Sonnet is much better. o3-mini-high is not bad. They do improve!
- atleastoptimal 2y agoPretty much everyone's career security over the next few years (until AGI) is to aggressively pay attention to and arbitrage on AI blindspots
- fritzo 2y agoRule of three is obsolete in age of AI assist. New rule is rule of 10ish.
- NiloCK 2y agoI don't agree with this, or maybe I don't get it. The point of DRY isn't to save the time on typing - it's to retain a single source of truth. If you've used an LLM to recreate some mechanism for you system in 8 different places, and that mechanism needs to change ... good luck finding them all.
- fritzo 2y agoSpecifically within a single file, I find LLMs can easily extend and maintain large sets of slightly varying concrete code, yet they struggle when that code is compressed into abstractions. The concrete examples form a dataset. Attention generalizes from that dataset. I'll agree that rule of three continues to apply for patterns across files, where there is less guarantee that all patterns will be read or written together in any given AI action.
- duxup 2y agoI find LLMs WANT TO ANSWER TOO MUCH. If I give them too little data, they're not curious and they'll try to craft an answer when it's nearly impossible for them to be right. I'll type and hit enter too early and I get an answer and think "This could never be right because I gave you broken sentences and too little." but there it goes answering away, dead wrong. I would rather the LLM say "yo I don't know what you're talking I need more" but of course they're not really thinking so they don't do that / likely can't. The LLM nature to run that word math and string SOMETHING together seems like an very serious footgun. Reminds me of the movie 2010 when they discuss how the HAL 9000 couldn't function correctly because it was told to lie despite its core programming to tell the truth. HAVING to answer seems like a serious impediment for AI. I see similar-ish things on google's gemini AI when I ask a question and it says the answer is "no" but then gives all the reasons the answer is clearly "yes".
- jredwards 2y agoThe most annoying thing I've found is that they always assume I'm right. If I ask a question, they assume the answer is yes, and will bend over backwards in an obsequious manner to ensure that I'm correct. "Why of course, sir, we should absolutely be trying to compile python to assembly in order to run our tests. Why didn't I think of that? I'll redesign our testing strategy immediately."
- j_bum 2y agoUgh , I agree. I would imagine this all comes from fine tuning, or RLHF, whatever is used. I’d bet LLMs trained on the internet without the final “tweaking” steps would roast most of my questions … which is exactly what I want when I’m wrong without realizing it.
- mulmboy 2y agoYeah I get this. Often I'll prompt it like "my intern looked at this and said maybe you should x. What do you think?" Seems to help.
- bredren 2y ago
- cytocync 2y ago[dead]
- shihab 2y agoToday I came across an interesting case where 3 well-known LLMs (O1, sonnet 3.7 and Deepseek R1) found a "bug" that actually didn't exist. Very briefly, in a fused cuda kernel, I was using thread i to do some stuff on locations i, i+N, i+2*N of an array. Later in the same kernel, same thread operated on i,i+1,i+2. All LLMs flagged the second part as bug. Not the most optimized code maybe, but definitely not a bug. It wasn't a complicated kernel (~120 SLOC) either, and the distance between the two code blocks was about only 15 LOC.
- deleted 2y ago[deleted]
- meltyness 2y agoThe Rust<->Typing axis mentioned as a blindspot definitely resonates. As a novice in the language, the amount of type-inference that good Rust incorporates can make things opaque, absent rust-analyzer.
- rzk 2y ago> The LLM knows nothing about your requirements. When you ask it to do something without specifying all of the constraints, it will fill in all the blanks with the most probable answers from the universe of its training set. Maybe this is fine. But if you need something more custom, it’s up to you to actually tell the LLM about it. Reminds of the saying: “To replace programmers with AI, clients will have to accurately describe what they want. We're safe.”
- jonahx 2y ago> “To replace programmers with AI, clients will have to accurately describe what they want. We're safe.” I've had similar sentiments often and it gets to the heart of things. And it's true... for now. The caveat is that LLMs already can, in some cases, notice that you are doing something in a non-standard way, or even sub-optimal way, and make "Perhaps what you meant was..." type of suggestions. Similarly, they'll offer responses like "Option 1", "Option 2", etc. Ofc, most clients want someone else to sort through the options... Also, LLMs don't seem to be good at assessment across multiple abstraction levels. Meaning, they'll notice a better option given the approach directly suggested by your question, but not that the whole approach is misguided and should be re-thought. The classic XY problem (https://en.wikipedia.org/wiki/XY_problem https://en.wikipedia.org/wiki/XY_problem). In theory, though, I don't see why they couldn't keep improving across these dimensions. With that said, even if they do, I suspect many people will still pay a human to interact with the LLM for them for complex tasks, until the difference between human UI and LLM UI all but vanishes.
- daxfohl 2y agoYeah, the difference having a human in the loop makes is the ability to have that feedback. Did you think about X? Requirement Y is vague. Z and W seem to conflict. Up to now, all our attempts to "compile" requirements to code have failed, because it turns out that specifying every nuance into a requirements doc in one shot is unreasonable; you may as well skip the requirements in English and just write them in Java at that point. But with AI assistants, they can (eventually, presumptively) enable that feedback loop, do the code, and iterate on the requirements, all much faster and more precisely than a human could. Whether that's possible remains to be seen, but I'd not say human coders are out of the woods just yet.
- taherchhabra 2y agoMonorepo vs seperate repo for frontend and backend. Which one is better for AI coding?
- fooker 2y agoI have noticed a very interesting deficiency. I work on compilers and a bunch of the code I write is for generating other code, and I have to do some second order reasoning about the behavior of the generated code. I haven't been able to make LLMs do this well.
- DeathArrow 2y agoI am fiddling with tools like Cursor, Aider, Augment Code, Roo Code and LLMs like GPT, Sonnet, Grok, Deepseek to try to decide whether I can use AI for what I need, and if yes, identify some good workflows. I've read experiences of other people and tried my own ideas. I've burnt countless tokens, fast searches and US dollars. Working with AI for writing code is painful. It can break the code in ways you've never imagined and introduce bugs you never thought are possible. Unit testing and integration testing doesn't help much, because AI can break those, too. You can ask AI to run in loop, fixing compile errors, fixing tests, do builds, run the app and do API calls, to have the project building and tests passing. AI will be happy to do that, burning lots of dollars while at it. And after AI "fixes" the problem it introduced, you will still have to read every goddam line of the code to make sure it does what is supposed to. For greenfield projects, some people recommended crafting a very detailed plan with very detailed description and very detailed specs and feed that into the AI tool. AI can help with that, it asks questions I would never ask for an MVP and suggests stuff I would never implement for an MVP. Hurray, we have a very, very detailed plan, ready to feed into Cursor & Friends. Based on the very detailed plan, implementation takes few hours. Than, fixing compile errors and fixing failing tests takes a few more days. Then I manually test the app, see it has issues, look in the code to see where the issues can be. Make a list. Ask Cursor & Friends to fix issues one by one. They happily do it and they happily introduce compilation errors again and break tests again. So the fixing phase that last days begins again. Rinse and repeat until hopefully we spend a few weeks together (AI and I) instead on me building the MVP myself in half time. One tactic which seems a bit faster, is to just make a hierarchical tree of features, ask Cursor & Friends to implement a simple skeleton, then ask them to implement each feature, verifying myself the implementation after each step. For example, if I need to log in users, just ask to add logging in code, the ask to add an email sender service, then ask to add email verification code. Structuring the project using Vertical Slice Architecture and opening each feature folder in Cursor & Friends seems to improve the situation as the AI will have just enough context to modify or add something but can't break other parts of the code. I dislike that AI can introduce inconsistencies in code. I had some endpoint which used timestamps and AI used three different types for that DateTime, DateTimeOffset and long (UNIX time). It also introduced code to convert between the types and lots of bugs. The AI uses some folder structure for a part of the solution and other structure for other parts. It uses some naming conventions in some parts and other naming conventions in other parts. It uses multiple libraries for the same thing, like multiple JSON serializing libraries. It does things in a particular way in some parts of the application and in another way in other parts. It seems like tens of people are working in the same solution without anyone reading the code of the others. While asking AI to modify something, it will be very happy to modify things that you didn't ask to. I still need to figure out a good workflow, to reduce time and money spent, to reduce or eliminate inconsistency, to reduce bugs and compile errors. As an upside using AI to help with planning seems to be good, if I want to write the code myself, because the plan can be very thorough and I usually lack time and patience to make a very detailed plan.
- seanwilson 2y ago> The eternal debate between dynamic and static type systems concerns the tradeoff between ease of prototyping and long term maintainability ... Unfortunately, the training corpus highly emphasizes Python and JavaScript. Anyone have experience here with how well strong static types help LLMs? You'd think it would be a great match, where the type errors give feedback to the LLM on what to fix. And the closer the types got to specifying the shape of the solution, the less guidance the LLM would need. Would be interesting to see how well LLMs do at translating unit test examples and requirements in English into a set of types that describe the program specification, and then have the LLM generate the code from that. I haven't kept up here, but guessing this is really interesting for formal verification, where types can accurately capture complex specifications but can be challenging to write. I find it quite sad that it's taken so long to get traction on using strong static types to eliminate whole classes of errors at the language level, and instead we're using super AI as a bandaid to churn out bug fixes and write tests in dynamically types languages for properties that static types would catch. Feels backwards.
- sourtrident 2y agoI've noticed coding with LLMs feels like pair programming with an overly confident intern - brilliant ideas, but needs reminders about humility and structure before burning down your repo. Keeps things lively though.
- AtlasBarfed 2y agoIt's kind of weird so a lot of my queers just really aren't fulfilled that well. One recent example is give me the names of 200 dragons from literature or media and it really gave up after about 80. And there's literally a web page that says 200 famous dragons as well as a Wikipedia page. Maybe it's some free tier limits of chatgpt. It's just strange to see these stories about AI services solving extremely advanced math and I ask it about a simple and basic a question as there is something it should be in his wheelhouse with a breath-based large amount of media ingestion... it should be able to answer fairly easily...