4 ms·

>That's a problem with all LLMs. They will average what they are trained on, without any reflection what is good and what is not. They shouldn't be called intel

>That's a problem with all LLMs. They will average what they are trained on, without any reflection what is good and what is not. They shouldn't be called intelligent (AI), they're information meatgrinders. This is an issue with your prompting not the models. If you tell someone "Do this thing" they'll just do it, LLM's too. And how it is done will probably be terrible. If you ask someone "come up with 5 ways to accomplish this, compare and contrast between them, list pros and cons, think through best practices, maintainability, readability, security, performance and cost. Come to a final decision and then do it" their response will be different. So too will the LLM's. You can even have the LLM present its opinions and recommendation but pause and wait for your choice on the go-ahead.

js8 3y ago
Fair enough, let's put aside that in Copilot, prompting is automatic. Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory. Compare that with googling a human-written stack overflow answer on the topic, there usually is some good soul who pointed out the inconsistency, if there is one. Look, I like grinded meat. But the fact is, without a detailed analysis, it's hard to tell what's actually in it.
- criley2 3y ago
  >Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory. And? As a Sr. Engineer, this is exactly how all the slop my Jr's and Mids send me looks. Copy pasted, Stack Overflow, and when I read through Jr code my mind is boggled. I want to shake them "did you even read your own code. do you even know what your code is doing? Why did you do this?" Forget style, best practices, etc, In my opinion, LLM's produce code that is as good or better than most Jr engineers and as a Sr it is my responsibility to audit, review and test all code. As a Sr level engineer I spend 90% of my time judging/fixing/improving others code, and less than 10% of my time writing my own. LLM is just another source, and unlike the Jr, I can quickly ask it why it did what it did or to refine it. You ask the Jr to iterate on the project and you won't hear from them until they mention a blocker at tomorrow's stand-up (or you just pair it out and spend 2 hours teaching them, while the LLM turns it around in 15 seconds. There is value in teaching of course, but we build quickly, too).

js8 3y ago

Fair enough, let's put aside that in Copilot, prompting is automatic. Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory. Compare that with googling a human-written stack overflow answer on the topic, there usually is some good soul who pointed out the inconsistency, if there is one. Look, I like grinded meat. But the fact is, without a detailed analysis, it's hard to tell what's actually in it.

criley2 3y ago
>Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory. And? As a Sr. Engineer, this is exactly how all the slop my Jr's and Mids send me looks. Copy pasted, Stack Overflow, and when I read through Jr code my mind is boggled. I want to shake them "did you even read your own code. do you even know what your code is doing? Why did you do this?" Forget style, best practices, etc, In my opinion, LLM's produce code that is as good or better than most Jr engineers and as a Sr it is my responsibility to audit, review and test all code. As a Sr level engineer I spend 90% of my time judging/fixing/improving others code, and less than 10% of my time writing my own. LLM is just another source, and unlike the Jr, I can quickly ask it why it did what it did or to refine it. You ask the Jr to iterate on the project and you won't hear from them until they mention a blocker at tomorrow's stand-up (or you just pair it out and spend 2 hours teaching them, while the LLM turns it around in 15 seconds. There is value in teaching of course, but we build quickly, too).