4 ms·
That’s factually untrue. I’m using models to work on frameworks with nearly zero preexisting examples to train on, doing things no one’s ever done with them, an
by kstrauser 7mo ago
That’s factually untrue. I’m using models to work on frameworks with nearly zero preexisting examples to train on, doing things no one’s ever done with them, and I know this because I ecosystem around these young frameworks.
Models can RTFM (and code) and do novel things, demonstrably so.
- allthetime 7mo agoYeah. I work with bleeding edge zig. If you just ask Claude to write you a working tcp server with the new Io api, it doesn’t have any idea what it’s doing and the code doesn’t compile. But if you give it some minimal code examples, point it to the recent blog posts about it, and paste in relevant points from std it does incredibly well and produce code that it has not been trained on.
- SV_BubbleTime 7mo agoIt’s always been about context, then being able to communicate it. Manager people or managing a hyper-knowledgeable intern (LLM). If you know what you need, actually want what you want (super difficult), and have the ability to provide context to someone else… management has always been easier for you than others. I find one of the more interesting things about the current “AI debate” is that many programmers are autistic or at least close one side of an empathetic spectrum that they’ve always had trouble communicating what is needed for a task and why. So it’s hard for me to take the opinions going around.
- qweiopqweiop 7mo agoIt also needs a validation loop. Give it the compiler output and I bet it would fix that code even without examples/a blog post.
- coldtea 7mo ago>I’m using models to work on frameworks with nearly zero preexisting examples to train on Zero preexisting examples of your particular frameworks. Huge number of examples of similar existing frameworks and code patterns in their training set though. Still not a novel thing in any meaningful way, not any more than someone who has coded in dozens of established web frameworks, can write against an unfamiliar to them framework homegrown at his new employer.
- thunky 7mo ago> Still not a novel thing in any meaningful way Right. What you're saying is that barely anyone is doing truly novel work. 100% agree.
- coldtea 7mo agoAlmost no e.g. web or app or enterprise or even game developer is doing any novel work, does that come as a surprise? And is that fact supposed to be an argument in favor of how LLMs can do novel work and move the state of the art (which is what we're arguing about). I mean, "LLMs can do novel work because: barely anyone is doing truly novel work" doesn't really compute as an argument.
- thunky 7mo agoWhat I'm saying is that LLMs don't have to do truly novel work in order to be useful. They are useful because the lion's share of all work is a variation on an existing theme (even if the creator may not realize it).
- kstrauser 7mo agoI'm not talking about web frameworks. I'm talking about other frontiers with darn near zero preexisting examples, and no code samples to borrow from in any language or in any similar framework (because there is no such thing). "LLMs can only emit things they've been trained on" is wholly obsolete.
- coldtea 7mo ago>I'm not talking about web frameworks. I'm talking about other frontiers Such as what? >with darn near zero preexisting examples Whatever it is, you'd be surprised.
- deleted 7mo ago[deleted]