7 ms·
I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. Al
by dopylitty 3y ago
I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in.
People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically valueless because the time taken to fix it would be equivalent to the time taken to write it in the first place.
Similarly people who are poor programmers or have only a surface level understanding of a topic (especially management types who are trying to appear technical) look at LLM output and think it’s ready to ship but good programmers recognize that the output is broken in so many ways large and small that it’s not worth the time it would take to fix compared to just writing from scratch.
- catchnear4321 3y agoeveryone you described share something in common. they aren’t good at using language models.
- mikro2nd 3y agoNor are 99.9% of humanity. I think that's the point.
- skybrian 3y agoProgrammers don't think that, though, or least not all the time. You could say similar things about Stack Overflow, and yet we use it.
- jprete 3y agoStack Overflow responses are well known to be misranked. I’ve heard a rule of thumb that the actual correct answer is typically about #3.
- moffkalast 3y agoAnd #1 is usually broken or wrong, due to its (typically) old age. The longer it has to accumulate upvotes the less relevant it becomes.
- dboreham 3y agoFor any managers reading: Chat GPT and Stack Overflow are not the same kind of thing.
- airstrike 3y agoIndeed they're not. And GPT-4 tends to outperform SO in my experience.
- nobody_r_knows 3y ago[flagged]
- Centigonal 3y agohttps://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
- Alifatisk 3y agoWhy the anger?
- Centigonal 3y agoGell-Mann Amnesia!
- iterminate 3y agoGell-Bott Amensia.
- foooorsyth 3y agoYep. ChatGPT is like having a junior engineer confidently asking to merge broken garbage into your codebase all the time. Adds negative value for anyone that knows what they’re doing.
- dharmab 3y agohahahah. A friend of mine has a problem with a contractor at his workplace that tries to PR in shell scripts written with Copilot. My friend spends an hour to explain why a script generated in 5 minutes is horrifically awful and will likely take down the company. He's legitimately angry about it.
- Jeff_Brown 3y agoIt seems like the only ways to subordinate programming tasks are to write tests for your subordinate's code, or to review it tediously yourself, or to just trust the hell out of them.
- simonw 3y agoBut with one crucial difference: it's a junior programmer that can make changes based on your feedback in a few seconds, not a few hours. And it never gets tired or frustrated. I find treating it like an intern is amazing productive: https://simonwillison.net/2023/Sep/29/llms-podcast/#code-interpreter-as-a-weird-kind-of-intern https://simonwillison.net/2023/Sep/29/llms-podcast/#code-int...
- hutzlibu 3y agoLLMs are not worthless for programming. You just cannot expect it to ship a full programm for you, but for generating functions with limited scope, I found it very useful. How to make use of a new and common libary for example. But of course you have to check and test. And for text I know people who use it succesfully (professionally) to generate texts for them as a summary from some data. They still have to proof read, but it saves them time, so it is valuable.
- Roark66 3y ago>LLMs are not worthless for programming. They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards. For an example. I've used Gpt4 as a sort of Google on steroids with prompts like "do subnets in gcloud span azs" and ", "in gcloud secret manager can you access secrets across regions". I very quickly learned to ask "is it true" after every answer and to never rely on a given answer too much(verify it quickly, don't let misinformation get you too far down the wrong route). So is it useful? Yes, but can it lead you down the wrong path? It very well can. The least experience you have in the field the easier it will happen. >You just cannot expect it to ship a full programm for you, but for generating functions with limited scope, I found it very useful Entire functions? Wow. I found it useful for generating skeletons I then have to fill by hand or tweak. I don't think I ever got anything out of Gpt4 that is useful as is (maybe except short snippets 3 lines long). However, I found it extremely useful in parsing emails received from people or writing nice sounding replies. For that it is really good (in English).
- gumballindie 3y ago> They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards. I basically gave up on llms because i was spending more time figuring out what it did wrong than actually getting value. People without programming skill are still impressed by them. But they yet have to learn or deliver anything of value even with the help of chat bots.
- 3y ago
- School-Cotton 3y agoI honestly don’t understand how people can say LLMs are useless for coding. Have you tried ChatGPT 4, or are you basing this take on the obsolete 3.5? I’m a professional programmer and I think LLMs are extremely useful.
- foooorsyth 3y agoI’ve used GPT 4. It’s not helpful in any domain in which I’m already proficient. If I’m having to use a new language or platform for whatever reason it’s mildly quicker than alt-tabbing to stack overflow, but probably not worth the subscription. For graphics tasks GenAI is absurdly helpful for me. I can code but I can’t draw. Getting icons and logos without having to pay a designer is great.
- ftxbro 3y ago> I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. This is true for media articles but for LLMs I feel like it's the opposite. Like people who aren't specialists don't fully appreciate how great it is at those tasks.