4 ms·
I build a pretty popular LLM tool. I think learning when/how to use them is as big a mental hurdle as it was learning to google well or whether something is goo
by parhamn 3y ago
I build a pretty popular LLM tool. I think learning when/how to use them is as big a mental hurdle as it was learning to google well or whether something is googlable or not.
In the realm of coding here are a few things its really good at:
- Translating code, generating cross language clients. I'll feed it a golang single file API backend and tell it to generate the typescript client for that. You can add hints like e.g "use fetch", "allow each request method to have a header override", "keep it typesafe, use zod", etc
- Basic validation testing. It's pretty good at generating scaffold tests that do basic validation (Opus is good at writing trickier tests) as your building.
- Small module completion. I write an interface of a class/struct with it's methods and some comments and tell it to fill in. A recent one I did looked something like (abbreviated):
type CacheDir struct { dir string, maxObjectLifetime: Duration, fileLocks sync.Map }
type (cd *CacheDir) Get(...)
type (cd *CacheDir Set(...)
type (cd *CacheDir) startCleanLoop()
Opus does a really good job generating the code and basic validation tests for this.
One general tip: you have to be comfortable spending 5 minutes crafting a detailed query assuming the task takes longer than that. Which can be weird at first if you take yourself seriously as a human.
Note that I hadn't been able to do much of this with GPT-4 Turbo with with Claude Opus it really feels capable.
- ThalesX 3y agoJust to answer to the turbo aspect, I've seen a big downgrade in quality when comparing 4 to 4-turbo, and even the new preview which is explicitly supposed to follow my instructions better. So I'm running a first pass through 4 and then combinging it with 4-turbo to take advantage of the larger context window and then running 4 on it again to get a better quality output.
- parhamn 3y agoYou really need to try Opus. Try a provider that works across models (one in my bio).
- Eager 3y agoIt's incredible how far behind HN of all places is w.r.t. what the current best tech is. So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along. Gemini Advanced is also a great model, but for other reasons. That thing really knows a boat load of low level optimization tricks.
- beepbooptheory 3y agoI'm sure you know what your talking about, but pushing the point that what is "best" or worth talking about is something that changes like every month does not really help defend against the case that most of this is just hype-churn or marketing.
- Eager 3y agoI'm not pushing what to talk about so much as pushing the point not to talk about stuff that is obsolete and starting to smell. It's that hype-churn marketing that is a motivating factor for the groups to innovate, much like Formula 1. It might be distasteful, but that doesn't mean it isn't working.
- shruggedatlas 3y ago> So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along. So what is the SOTA, in your opinion?
- ThalesX 3y ago> So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along. I'm talking about 4-turbo, 4-turbo preview and self hosted LLama2. What in God's name is not SOTA about this?
- dukeyukey 3y ago> It's incredible how far behind HN of all places is w.r.t. what the current best tech is. Good fucking Christ, Opus has been out for like 8 days, I've had holidays longer than that!
- jonny_eh 3y ago> Translating code, generating cross language clients Can any convert a native iPhone app to an Android one?
- CamperBob2 3y agoPiece by piece, sure. The context window is too small to just feed it a massive source dump all at once.
- OtherShrezzing 3y ago>- Small module completion. I write an interface of a class/struct with it's methods and some comments and tell it to fill in. A recent one I did looked something like (abbreviated): Are they considerably better than existing non-AI tools + manual coding for this? In VSCode and Visual Studio, when working with an interface in C# for example, I can click two context menus to have it generate an implementation with constructors, getters, & setters included, leaving only the business logic code to write manually. You've mention you have to describe to the AI in comments, and then I assume spend time on a step to verify the AI has correctly interpreted your request & implemented. I can definitely see the advantage for LLMs when writing unit tests on existing code, but short of very limited situations, I'm really finding it difficult to find the 55% efficiency improvements claimed by the likes of GitHub's AI Copilot.
- joenot443 3y agoThat sounds crazy useful and I think speaks most to the maturity of C# and Microsoft's commitment to making it so ergonomic. I'm pretty curious about that feature, I'd love something similar for C++ in VS Code, but thus far I've been doing a pretty similar Copilot flow to the parent comment. It's nothing groundbreaking, but a nice little productivity boost. If I had to take that or a linter, I'd take the linter. Totally agree on the 55% figure being hogwash.
- creato 3y agoVisual Studio (not VSCode) has this for C++, though it can be a bit finicky. It’s infinitely better than AI autocomplete, which just makes shit up half the time.