9 ms·
Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically ent
by tehnub 1y ago
Interesting exchange on the use of AI coding tools:
curious how much did you write the code by hand of it?
Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution.
https://x.com/karpathy/status/1977758204139331904 https://x.com/karpathy/status/1977758204139331904
- oblio 1y agoWe're still not ready for ouroboros.
- gyomu 1y ago> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!
- SchemaLoad 1y agoEverything is unless your app is a React todolist or leatcode questions.
- SeanAnderson 1y agoor a typical CRUD app architecture, or a common design pattern, or unit/integration test scaffolding, or standard CI/CD pipeline definitions, or one-off utility scripts, etc... Like 80% of writing coding is just being a glorified autocomplete and AI is exceptional at automating those aspects. Yes, there is a lot more to being a developer than writing code, but, in those instances, AI really does make a difference in the amount of time one is able to spend focusing on domain-specific deliverables.
- positron26 1y agoIt has gotten to the point that I don't modify or write SQL. Instead I throw some schema and related queries in and use natural language to rubber duck the change, by which point the LLM can already get it right.
- MasterScrat 1y agoAnd even for "out of distribution" code you can still ask question about how to do the same thing but more optimized, could a library help for this, why is that piece of code giving this unexpected output etc
- notatoad 1y agopeople say this like it's a criticism, but damn is it ever nice to start writing a simple crud form and just have copilot autocomplete the whole thing for me.
- goalieca 1y agoBack in the 90s you could drag and drop a vb6 applet in Microsoft word. Somehow we’ve regressed.. Edit: for the young, wysiwyg (what you see is what you get) was common for all sorts of languages from c++ to Delphi to html. You could draw up anything you wanted. Many had native bindings to data sources of all kinds. My favourite was actually HyperCard because I learned it in grade school.
- squeaky-clean 1y agoWysiwyg kind of fell apart once we had to stop assuming everyone had an 800x600 or 1024x768 screen, because what you saw was no longer what others got.
- hackit2 1y agoMost of the internet still assumes you're using a 96 DPI monitor. Tho the rise of mobile phone has changed that it seems like the vast majority of the content consumed on mobile lends itself to being scaled to any DPI - eg.. movies, pictures, youtube ect.
- eternauta3k 1y agoNot a big issue with QT layouts (still have to test the result though)
- benterix 1y agoNot entirely, in these RAD tools you also had flexible layout choices and obviously you could test it for various window sizes (although the maximum was the one supported by your graphics card). Too bad many chose the lazy way and just enforced fixed window size at 800x600.
- 1y ago
- meowface 1y agoHN's cynicism towards AI coding (and everything else ever) is exhausting. Karpathy would probably cringe reading this.
- trial3 1y agookay but he literally does have a bridge that non-deterministically might take you to the wrong place to sell you
- meowface 1y agoThe original context of this sub-thread was Karpathy saying how AI coding tools were pretty useless for him when working on this particular project.
- troupo 1y agoIndeed. And only Karpathy is entitled to say that AI tools produce wrong code for him. And he's only entitled to say it for this project only. If anyone else says this, "the skepticism is exhausting", and their experience is completely irrelevant.
- kasey_junk 1y agoGo look at the comments on HN whenever someone posts about their AI coding workflow. It will be littered with negative comments that either imply or outright say that the poster is either shilling, ignorant or working only on toy examples. The grievance attitude seems to exist in both directions and is actually what is exhausting.
- troupo 1y ago> It will be littered with negative comments that either imply or outright say that the poster is either shilling, ignorant or working only on toy examples. And they would be often be right. Coupled with the fact that most of the glowing "omg I only code with AI" posts don't even try to show what code or products they are working on. And yes, the absolute vast majority of people who are skeptical are skeptical precisely because they use these tools every day themselves.
- KeplerBoy 1y agoI don't know. I successfully use it for small changes on VHDL FPGA designs these days.
- dahcryn 1y agosimple CRUD, is as common in many many business applications or backend portals, are a good fit for AI assistance imho. And fix some designs here and there, where you can't be bothered to keep track of the latest JS/CSS framework
- lukev 1y agoGenerative AI for coding isn't your new junior programmer, it's the next generation of app framework.
- Zenst 1y agoI wished such sentiments prevailed in upper management, as it is true. Much like owning a car that can drive itself - you still need to pass a driving test to be allowed to use it.
- VerminOctopus1 1y agoReally such an annoying genre of comment. Yes I’m sure your groundbreaking bespoke code cannot be written by LLMs, however for the rest of us that build and maintain 99% of the software people actually use, they are quite useful.
- allochthon 1y agoI've had some success with a multi-threaded software defined radio (SDR) app in Rust that does signal processing. It's been useful for trying something out that's beyond my experience. Which isn't to say it's been easy. It's been a learning experience to figure out how to work around Claude's limitations.
- teleforce 1y agoI wonder if the new GenAI architecture namely DDN or distributed discrete networks being discussed recently can outperform the conventional architecture of GAN and VAE. As the name suggests, it can provide multitude of distributions for training and inference purposes [1]. [1] Show HN: I invented a new generative model and got accepted to ICLR (90 comments): https://news.ycombinator.com/item?id=45536694 https://news.ycombinator.com/item?id=45536694
- CapsAdmin 1y agoI work on this typed lua language in lua, and sometimes use llms to help fix internal analyzer stuff, which works 30% of the time for complex, and sometimes not at all, but helps me find a solution in the end. However when I ask an llm to generate my typed lua code, with examples and all, on how the syntax is supposed to be, it mostly gets it wrong. my syntax for tables/objects is: local x: {foo = boolean} but an llm will most likely gloss over this and always use : instead of = local x: {foo: boolean}
- kasey_junk 1y agoAre you using a coding agent or just an llm chat interface? Do you have a linter or compiler that will catch the misuse that you’ve hooked up to the agent?
- CapsAdmin 1y agoI've dabbled with claude code in this particular project, but not much. My short experience with it is that it's slow, costly and goes off the rails easily. I prefer to work with more isolated parts of the code. But again, I don't really know all that much about agents. One thing I wanted to do on my project is reorganize all the tests, which sounds like an agent job. But I'd imagine I need to define some hard programmatic constraints to make sure tests are not lost or changed in the process.
- kasey_junk 1y agoAgents aren’t magic. They are loops with tool calls in them that help keep agents on track. And most of the agent systems have some manner of hook that you can put your own tools in to enforce things like types and styles. I’ve had good experiences writing small scripts and linters to enforce things that agents get wrong frequently. What’s nice about those is that the agents are very good at writing them and they are easy to verify. Plus they are valuable for new humans devs as well.
- pmarreck 1y agoI've had success in the past with getting it to write YueScript/Moonscript (which is not a very large part of its training data) by pointing it to the root URL for the language docs and thus making that part of the context. If your typed version of Lua has a syntax checker, you could also have it try to use that first on any code it's generated
- random_cynic 1y ago[dead]
- dude250711 1y agoHow convenient! You know, my code is somewhat far off the data distribution too.
- SeanAnderson 1y agoThis makes sense, right? It's a relatively novel thing to be writing. I don't find it to be a damning remark like other comments here seem to be concluding. If anything, the fact that Karpathy reached towards Claude/Codex in an attempt to gain value is indicative that, in previous coding efforts, those tools were helpful to him.
- simonw 1y agoYeah, if your goal is "build the tightest 8,000 line implementation of training an LLM from scratch, with a focus on both conciseness and educational value" I don't think it's particularly surprising that Claude/Codex weren't much help.
- bringmeiron 1y ago> If anything, the fact that Karpathy reached towards Claude/Codex in an attempt to gain value is indicative that, in previous coding efforts, those tools were helpful to him. This is good for bitcoin.
- krackers 1y agoIt's not _that_ far off distribution though. The math and concepts are well understood.
- rootusrootus 1y agoThat is a good thing to hear from someone as reputable as Karpathy. The folks who think we're on the cusp of AGI may want to temper their expectations a bit. I do love Claude Code, because one thing I periodically need to do is write some web code, which is not my favorite type of coding but happens to have incredibly good coverage in the training data. Claude is a much better web developer than I am. But for digging into the algorithmic core of our automation tooling, it doesn't have nearly as much to work with and makes far more mistakes. Still a net win I'm happy to pay for, even if it's never anything more than my web developer slave.
- bdangubic 1y ago> But for digging into the algorithmic core of our automation tooling What I find fascinating is reading this same thing in other context like “UI guru” will say “I would not let CC touch the UI but I let it rip on algorithmic core of our automation tooling cause it is better at it than me…”
- Filligree 1y agoBoth can be true. LLMs tend to be mediocre at (almost) everything, so they're always going to be worse than the user at whatever the user is an expert in. But 'mediocre' isn't 'useless'.
- rootusrootus 1y agoI completely agree. I'm definitely not an expert web developer. I know enough to build functional tools, but it's not exactly art that I'm making. But the core of our tooling is my primary focus, I wrote it, I've spent a lot of time perfecting it. Claude can easily impress me with things like the CSS magic it weaves, because I am unsophisticated.
- vunderba 1y ago100%. I find the "LLMs are completely useless" and the "LLMs will usher in a new era of messianic programming" camps to be rather reductive. I've already built some pretty large projects [1] with the assistance of agentic tooling like Claude Code. When it comes to the more squirrely algorithms and logic, they can fall down pretty hard. But as somebody who is just dreadful at UI/UX, having it hammer out all the web dev scaffolding saves me a huge amount of time and stress. It's just a matter of tempering one's expectations. [1] https://animated-puzzles.specr.net https://animated-puzzles.specr.net
- deleted 1y ago[deleted]
- sva_ 1y agohttps://nitter.net/karpathy/status/1977755427569111362 https://nitter.net/karpathy/status/1977755427569111362
- bringmeiron 1y agoClearly he has little idea what he's talking about. AI can write better code than 99% of developers. This embarrassingly anti-AI shill included. If he used the AI tool my company is developing the code would have been better and shipped sooner.
- throwaway0123_5 1y agoAnti-AI shill? A cofounder of OpenAI?
- bringmeiron 1y agoYou have found the joke.
- freedomben 1y agoI think you are running into Poe's law here.
- satvikpendem 1y agoThat's funny that the coiner of the term vibe coding has eventually found it not useful anymore.
- JimDabell 1y agoThat’s not what he said. This is the new project: > My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. This is how he described vibe coding: > There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It's possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always, I don't read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it. The code grows beyond my usual comprehension, I'd have to really read through it for a while. Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until it goes away. It's not too bad for throwaway weekend projects, but still quite amusing. I'm building a project or webapp, but it's not really coding - I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works. Vibe coding is clearly aimed at having fun hacking around on something that doesn’t matter, and he’s doing the opposite of that with this project. The fact that he’s not using vibe coding for something that is completely inappropriate for vibe coding is neither surprising nor a failure of vibe coding.
- hansmayer 1y ago... or maybe he just forgot to include the claude.md ? :)
- RA_Fisher 1y ago> too far off the data distribution. I guess his prompts couldn’t provide sufficient information either (there’s no limit). Sounds more like a user issue to me. :) I don’t think there’s anyone that can type faster than ChatGPT.
- kubb 1y agoHe probably just doesn’t know how to prompt correctly (heheh).
- nurettin 1y agoBackprop and transformers isn't exactly off the grid coding, but I can see how it would require a lot of patience to force claude into writing this.
- martingalex2 1y agoIsn't the point that now Andrej's published this, it will be in-distribution soon?
- samus 1y agoThe llama.cpp maintainers working on supporting Qwen3-next are also not enthused by LLM output. They had to go over everything and fix it up. https://github.com/ggml-org/llama.cpp/pull/16095#issuecomment-3314882956 https://github.com/ggml-org/llama.cpp/pull/16095#issuecommen...