40 ms·
Something that's been amusing me about Cursor is that I feel a lot of the excitement about it isn't so much about Cursor, it's people realizing that LLMs have g
by simonw 2y ago
Something that's been amusing me about Cursor is that I feel a lot of the excitement about it isn't so much about Cursor, it's people realizing that LLMs have got REALLY good at writing code now. GPT-4 was good for the past year, but the latest models (especially Claude 3.5 Sonnet) are spectacular.
Those of us who've been copying and pasting LLM-generated code back and forth from ChatGPT and Claude.ai for the past year had figured this out already, so Cursor wasn't a huge surprise for us.
For a lot of people Cursor is the first time they've really appreciated how good this stuff has got - so it's getting a massive amount of buzz, much of which should really be credited to Claude 3.5 Sonnet.
(Cursor have done an excellent job designing editor features on top of that model and deserve credit for that, I'm just a bit tickled at some of the excitement which basically boils down to "Huh, LLMs can write good code now".)
- GiorgioG 2y agoI must be living in a different universe, every single LLM I've tried are shit at the details/nuances of writing code.
- swah 2y ago80% of folks are doing web stuff, as soon as you deviate from that the quality must go from 9/10 to 3/10 or whatever Also have you tried shorter blocks ? "Write a function like this" instead of "write the app"
- sarreph 2y agoClaude has no trouble with SQL queries, writing tests, utility functions, component sketches (React). But as soon as I start giving it reasonably complex stuff like interdependent generics (TypeScript) or intricate custom logic implementation on top of well-known frameworks, it quite often goes around in circles or outputs code with incorrect syntax. You know, the stuff that I would actually feel as though I was receiving intellectual help with. Current LLMs are _great_ at doing chores. They are not so great at producing cohesive, professional-grade code.
- steve_adams_86 2y agoI tried getting Claude to help me model an event-based store with moderate type safety and it was a total disaster. It was randomly trying to convince me to turn it into a state machine using a library in Python (I was working in TypeScript). This is so far from awesome as far as productivity goes. I do find cursor suggestions useful here and there, but it’s typically filling in the simpler details of an implementation I had to design without its help. Ultimately it doesn’t save nearly as much time as I hoped it might a couple years ago. Still, cool development in terms of dev tools and I’ll keep experimenting.
- GiorgioG 2y agoLLMs...60 percent of the time it works every time.
- deleted 2y ago[deleted]
- kamaal 2y ago>>I tried getting Claude to help me model an event-based store with moderate type safety and it was a total disaster. That's just not how you work with a LLM. You don't give LLMs problems to solve. You do the thinking part and solve problems, and ask the LLM to implement small blocks of code, like the smallest possible, and you incrementally go from there. This really is some what like asking the whole question in the Google search bar, not seeing any result and saying Google search doesn't work. >>it’s typically filling in the simpler details of an implementation I had to design without its help. Most complex solutions are just reusing and connecting simpler solutions.
- GiorgioG 2y ago> ...and ask the LLM to implement small blocks of code, like the smallest possible, and you incrementally go from there. So it's autocomplete++? Talk about an insane level of hype for an incremental improvement to what we already have...if that's actually what we can hope to expect from these LLMs.
- joegibbs 2y agoWhich language are you using, and what kind of code are you writing? So far I've had good results for Javascript/Typescript and SQL for web apps and Unreal C++ for game development. I can imagine it would do way worse though if you were writing something with a lot of domain-specific knowledge, really cutting-edge stuff or in a more obscure language like Fortran - but then again, I think that most developers would also struggle more in those conditions. Part of what I find useful about LLMs is that it's almost like rubber duck debugging, except that the duck talks back. You have to explain to the LLM exactly what it is that you want it to do to get the right result, then it gives you a solution. Mostly the solution is right, but when it's wrong you'll better understand the problem through the act of writing it out.
- GiorgioG 2y ago> You have to explain to the LLM exactly what it is that you want it to do to get the right result, then it gives you a solution. Mostly the solution is right, but when it's wrong you'll better understand the problem through the act of writing it out. The solution is either correct or incorrect. In general its level of incorrectness is not trivial. My language of choice is C#. Anytime I ask it to generate a series of classes (say a data model of a particular domain, using specific well-known patterns), it gets it wrong, quite wrong and it would take more effort to explain it to unfuck itself, by which time it would have forgotten 3 deliberate instructions from 5 interactions ago. It’s a brilliant moron.
- kamaal 2y ago>>The solution is either correct or incorrect. Not even humans write 100% correct solutions. Everyday programming is not a theorem proving exercise. Programs are correct enough, to cover a agreeable amount of test cases. >>Anytime I ask it to generate a series of classes Generating a class is too big and broad of question to ask an LLM to generate. My chunks of inputs and outputs to/from an LLM are often on the lines of a single for loop, if block or a small chunk of code that can't be written in more than one way. There are several steps from here to a complete class. LLM is your classic socratic thinking tool. You really to have to learn to ask and work in chunks of questions that are smallest, and involve only one change from the previous question, and are atomic enough to do a easy roll back. And you need to build from here. As experience with the series of books The Little Schemer shows, this unfortunately is not everyone's cup of tea. A lot of people struggle to keep a train of thought in their minds and work in that workflow. To that extent I see LLM are not for everyone. Just like how Google search was great and helped a lot of people, but the majority just couldn't bring themselves to sit in front of a screen and work for hours stitching solutions from various places on internet into a working solution.
- dartos 2y agoI would love to see someone on stream use AI really effectively. I really try, but for the life of me, I can’t get AI to generate anything useful beyond simple tests. They’re still autocomplete++ for me.
- jay377 2y agoA good stream example might be this on a podcast, its got a few cuts but generally can get the point and its by a complete beginner, 1 month in https://www.youtube.com/watch?v=ehK-QqPstJ4 https://www.youtube.com/watch?v=ehK-QqPstJ4
- thejosh 2y agoAutocomplete++ is the killer feature for me, especially for more tedious thingsike SQL column naming.
- patcon 2y agoAgreed! But also... Rather than your seeing someone stream "really effective 99th percentile AI coding", maybe it would be more illustrative for us all to see someone stream the "50th percentile developer unassisted" experience. I suspect that's the thing we're all missing to help us understand the potential value of AI coding: how slow and ineffectively the median software developer works, bc they just quietly slog away and don't talk much about how they work :) I humbly include myself on that thought btw
- JTyQZSnP3cQGa8B 2y agoI also don’t understand the appeal. I never wrote boilerplate code in my life. I recently spent the whole day debugging open source third parties on multiple OSes while using Wireshark to understand how the whole thing worked. Then I typed 5 lines for a fix in less than 30 seconds. How would an LLM help me here?
- dartos 2y ago> I never wrote boilerplate code in my life. You either have not written very much code, don’t know what boilerplate means, or are intentionally operating under an overly broad definition
- airstrike 2y agoHow would you compare GPT-4o with Claude 3.5 Sonnet?
- notarealllama 2y agoNot parent but Claude any day of the week, it seems to keep context and do what I ask it better.
- GiorgioG 2y agoI've gave up on GPT-4o a while back, I've spent the last 1/2 hour (more like wasted it) trying to get Claude 3.5 Sonnet to do my bidding (nothing super complicated) and it's a complete mess. It does forget plenty. It gets things wrong. It's like the weather man, you just can't trust a word he says.
- simonw 2y agoFor my prompting style and the languages I work with - Python, JavaScript, SQL - I think Claude 3.5 Sonnet is slightly better.
- airstrike 2y agoThanks! It's interesting how this perception seems to vary based on language.
- klysm 2y agoI don’t share the sentiment that it’s “very good”. It’s strongly mediocre at basic tasks and regularly fails at anything complex.
- glzone1 2y agoI'm getting what I consider very good results. Yes, I have to iterate it and talk to it, spec very clearly, but if you know pretty precisely what you want I've found it very reliable. I usually try to actually provide AWS resource IDs / VPC id's, bucket names and everything and at least with AWS / Python stuff it is good (yes, some corners still but the trend is very good). My prompts aren't short but it spell out very clearly what I want. To process 2 million XML files let's say stored in S3, using an SQS queue ZZZ that will pass in S3 object ID's in this format, store results back to RDS Postgresql here. Use Loguru or whatever for logging. I provide sample XML files, sample results, schema for database (DDL) etc etc. Then you can iterate on this - deploy using docker to ECS, use fargate, then do some scale out when queue depth is deep, go to asyncio or multi-process or multi-threading etc etc and just build. It goes quick.
- klysm 2y agoThat doesn’t sound like an inherently complex programming task - just simple mindless munging
- simonw 2y agoHave we been using the same models? Claude 3.5 Sonnet writes me excellent code in a variety of different languages. I have a bunch of examples here: https://simonwillison.net/tags/claude-3-5-sonnet/ https://simonwillison.net/tags/claude-3-5-sonnet/
- cynicalpeace 2y agoCursor wow'ed me a year ago when I first used it. Been using it ever since. People are only starting to notice it now because of the virality machine. AI has been reasonably "good" at code for awhile. I think Karpathy finally talking positively about it a few weeks ago is what started off the current hype cycle.
- simonwanker 2y ago[flagged]