5 ms·
LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs
by Buttons840 1y ago
LLMs aren't my rubber duck, they're my wrong answer.
You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me.
I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.
- Buttons840 1y agoI'm probably suffering undiagnosed ADHD, and will get stuck and spend minutes picking a function name and then writing a docstring. LLMs do help with this even if they get the code wrong, because I usually won't bother to fix their variables names or docstring unless needed. LLMs can reliably solve the problem of a blank-page.
- linotype 1y agoThis. I have ADHD and starting is the hardest part for me. With an LLM it gets me from 0 to 20% (or more) and I can nail it for the rest. It’s way less stressful for me to start now.
- raihansaputra 1y agovery much agree. although lately with how good it is i get hyperfocused and spent more time then i allocated because i ended up wanting to implement more than i planned.
- linotype 1y agoIt’s a struggle right? First world LLM problems.
- bayesianbot 1y agoBeen suffering the same, I'm used to having so many days (weeks/months) when I just don't get that much done. With LLMs I can take these days and hack around / watch videos / play games while the LLM is working on background and just check the work. Best part is it often leads to some problematic situation that gets me involved and often I'll end up getting a real day of work out of it after I get started.
- carlmr 1y agoSo much this, the blank page problem is almost gone. Even if it's riddled with errors.
- msgodel 1y agoYeah, keeping me in the flow when I hit one of those silly tasks my brain just randomly says "no let's do something else" to has been the main productivity improving feature of LLMs.
- mystified5016 1y agoYes! So many times my brain just skips right over some tasks because it takes too much effort to start. The LLM can give you something to latch onto and work with. It can lay down the starting shape of a function or program and even when it's the wrong shape, you still have something to mold into the correct shape. The thing about ADHD is that taking a task from nothing to something is often harder than turning that something into the finished product. It's really weird and extremely not fun.
- pipes 1y agoThis is the complete opposite for me! I really like a blank page, the thought of writing a prompt destroys my motivation as does reviewing the code that an LLM produces. As an aside, I'm seeing more an more crap in PRs. Nonsensical use of language features. Really poorly structured code but that is a different story. I'm not anti LLMs for coding. I use them too. Especially for unit tests.
- albrewer 1y ago> LLMs can reliably solve the problem of a blank-page. This has been the biggest boost for me. The number of choices available when facing a blank page is staggering. Even a bad/wrong implementation helps collapse those possibilities into a countable few that take far less time to think about.
- bsder 1y agoLLMs are a decent search engine a la Google circa 2005. It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge. The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.
- andrekandre 1y ago> the conversational interface, for some reason, seems to turn off the natural skepticism that people have n=1 but after having chatgpt "lie" to me more than once i am very skeptical of it and always double check it, whereas something like tv or yt videos i still find myself being click-baited or grifted (iow less skeptical) much more easily still... any large studies about this would be very interesting...
- myvoiceismypass 1y agoI get irrationally frustrated when ChatGPT hallucinates npm packages / libraries that simply do not exist. This happens… weekly for me.
- bdangubic 1y agojust ask it to write and publish them and you good :)
- gessha 1y agoJia Tan will have to work 24/7 :)
- wvenable 1y agoWeird. I used to have that happen when it first came out but I haven't experienced anything like that in a long time. Worst case it's out of date rather than making stuff up.
- protocolture 1y ago
- therealpygon 1y agoLLMs follow instructions. Garbage in = garbage out generally. When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well. On the other hand, I find a lot of the loosely-goosey vibe coding approach to be useless and gives a lot of false impressions about how useful LLMs can be, both too positive and too negative.
- GiorgioG 1y agoSo what you’re saying is you need to be very specific and detailed when writing your specifications for the LLM to spit out the code you want. Sounds like I can just skip the middle man and code it myself.
- AndrewKemendo 1y agoNot in 10 seconds
- dingnuts 1y agoI've seen very long prompts that are as long as a school essay and those didn't take ten seconds either
- darkwater 1y agoTo some extent those fail in the same category of cheaters that put way more effort into cheating an exam than doing it properly. Or people paying 10/15 bucks a month to access a private Usenet server to download pirate content.
- Zamaamiro 1y agoYou probably didn’t write up a detailed prompt with perfect specifications in 10 seconds, either. In my experience, it doesn’t matter how good or detailed the prompt is—after enough lines of code, the LLM starts making design decisions for you. This is why I don’t accept LLM completions for anything that isn’t short enough to quickly verify that it is implemented exactly as I would have myself. Usually, that’s boilerplate code.
- seattle_spring 1y agoThis has been my experience as well. The biggest problem is that the answers look plausible, and only after implementation and experimentation do you find them to be wrong. If this happened every once in a while then it wouldn't be a big deal, but I'd guess that more than half of the answers and tutorials I've received through ChatGPT have ended up being plain wrong. God help us if companies start relying on LLMs for life-or-death stuff like insurance claim decisions.
- dabraham1248 1y agoI'm not sure if you're being sarcastic, but in case you're not... From https://arstechnica.com/health/2023/11/ai-with-90-error-rate-forces-elderly-out-of-rehab-nursing-homes-suit-claims/ https://arstechnica.com/health/2023/11/ai-with-90-error-rate... "UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges" Also "The use of faulty AI is not new for the health care industry."
- pwdisswordfishz 1y ago> If this happened every once in a while then it wouldn't be a big deal, but I'd guess that more than half of the answers and tutorials I've received through ChatGPT have ended up being plain wrong. It would actually have been more pernicious that way, since it would lull people into a false sense of security.
- Affric 1y agoYep. I like maths, I hate graphing. Tedious work even with state of the art libraries and wrappers. LLMs do it for me. Praise be.
- lanstin 1y agoYeah, I write a lot of little data analysis scripts and stuff, and I am happy just to read the numbers, but now I get nice PNGs of the distributions and so on from LLM, and people like that.
- AndrewKemendo 1y agoOut of curiosity can you give me an example prompt(s) you’ve used and been disappointed I see these comments all the time and they don’t reflect my experience so I’m curious what your experience has been
- Buttons840 1y agoI asked Chat GPT 4o to write an Emacs function to highlight a line. This involves setting the "mark" at the beginning, and the "point" at the end. It would only set the point, so I corrected it "no, you have to set both", but even after correction it would move the point to the beginning, and then moved the point again to the end, without ever touching the mark.
- DevDesmond 1y agoFrom my experience, (and to borrow terminology from a HN thread not long ago), I've found that once a chat goes bad, your context is "poisoned"; It's auto completing from previous text that is nonsense, so, further text generation from there exist in the world of nonexistent nonsense as well. It's much better to edit your message and try again. I also think that language matters - An Emacs function is much more esoteric than say, JavaScript, Python, or Java. If I ever find myself looking for help with something that's not in the standard library, I like provide extra context, such as examples from the documentation.
- anonzzzies 1y agoThere are so many examples where all current top models just will loop forever even if you instruct them literally the code. We know many of them, but for instance in a tailwind react project with some degree of complexity (nested components), if you ask for something to scroll in it's space, it will never figure out min-h-0 even if you tell it. It will just loop forever rewriting the code adding and removing things, to the point of it just putting comments like 'This will add overflow' and writing js to force scroll, and it will never work even if you literally tell it what to do. Don't know why, all big and small models have this, and I found Gemini is currently the only model that sometimes randomly has the right idea but then still cannot resolve it. For this we went back to not using tailwind and back to global vanilla css, which I never thought I would say, is rather nice.
- materiallie 1y agoThis is my experience, too. As a concrete example, I'll need to write a mapper function to convert between a protobuf type and Go type. The types are mirror reflections of each other, and I feed the complete APIs of both in my prompt. I've yet to find an LLM that can reliability generate mapping code between proto.Foo{ID string} to gomodel.Foo{ID string}. It still saves me time, because even 50% accuracy is still half that I don't have to write myself. But it makes me feel like I'm taking crazy pills whenever I read about AI hype. I'm open to the idea that I'm prompting wrong, need a better workflow, etc. But I'm not a luddite, I've "reached up and put in the work" and am always trying to learn new tools.
- lazyasciiart 1y agoAn LLM ability to do a task is roughly correlated to the number of times that task has been done on the internet before. If you want to see the hype version, you need to write a todo web app in typescript or similar. So it's probably not something you can fix with prompts, but having a model with more focus on relevant training data might help.
- KTibow 1y agoThese days, they'll sometimes also RL on a task if it's easy to validate outputs and if it seems worth the effort.
- akoboldfrying 1y agoThis honestly seems like something that could be better handled with pre-LLM technology, like a 15-line Perl script that reads one on stdin, applies some crufty regexes, and writes the other to stdout. Are there complexities I'm not seeing?
- xarope 1y agoI have to upvote this, because this is how I felt after trying three times (that I consciously decided to give an LLM a try, versus having it shoved down my throat by google/ms/meta/etc) and giving up (for now).