6 ms·
> Much Much more productive world by just knuckling down and learning how to do the work. The fact everyone that say they've become more productive with LLMs w
by skydhash 2y ago
> Much Much more productive world by just knuckling down and learning how to do the work.
The fact everyone that say they've become more productive with LLMs won't say how exactly. I can talk about how VIM have make it more enjoyable to edit code (keybinding and motions), how Emacs is a good environment around text tooling (lisp machine), how I use technical books to further my learning (so many great books out here). But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them. It's all claims that it's great with no further elaboration on the workflows.
> I haven't written a line of code in weeks. Just paragraphs of text that specify what I want and then guidance through and around pitfalls in a simple iterative loop of useful working code.
Code is intent described in terms of machinery actions. Those actions can be masked by abstracting them in more understandable units, so we don't have to write opcodes, but we can use python instead. Programming is basically make the intent clear enough so that we know what units we can use. Software engineering is mostly selecting the units in a way to do minimal work once the intent changes or the foundational actions do.
Chatting with a LLM look to me like your intent is either vague or you don't know the units to use. If it's the former, then I guess you're assuming it is the expert and will guide you to the solution you seek, which means you believe it understands the problem more than you do. The second is more strange as it looks like playing around with car parts, while ignoring the manuals it comes with.
What about boilerplate and common scenarios? I agree that LLMs helps a great deal with that, but the fact is that there are perfectly good tools that helped with that like snippets, templates, and code generators.
- Nadya 2y agoEver seen someone try and search something on Google and they are just AWFUL at it? They can never find what they're looking for and then you try and can pull it up in a single search? That's what it is like watching some people try to use LLM's. Learning how to prompt an LLM is as much a learned skill as much as learning how to phrase internet searches is a learned skill. And as much as people decried that "searching Google isn't a real skill" tech-savvy people knew better. Same thing except now it's also many tech-savvy people joining in with the tech-unsavvy in saying that prompting isn't a real skill...but people who know better know that it is. On average, people are awfully bad at describing exactly what it is they want. Ever speak with a client? And you have to go back and forward for a few hours to finally figure out what it is they wanted? In that scenario you're the LLM. Except the LLM won't keep asking probing questions and clarifications - it will simply give them what they originally asked for (which isn't what they want). Then they think the LLM is stupid and stop trying to ask it for things. Utilizing an LLM to its full potential is a lot of iterative work and, at least for the time being, requires having some understanding of how it works underneath the hood (eg. would you get better results by starting a new session or asking it to forget previous, poorly worded instructions?).
- ben_w 2y ago> On average, people are awfully bad at describing exactly what it is they want. Ever speak with a client? And you have to go back and forward for a few hours to finally figure out what it is they wanted? One of them it was the entire duration of me working for them. They didn't understand why it was taking so long despite constantly changing what they asked for.
- codr7 2y agoBuilding the software is usually like 10% of the actual job, we could do a better job of teaching that. The other 90% is mostly mushy human stuff, fleshing out the problem, setting expectations etc. Helping a group of people reach a solution everyone is happy with has little to do with technology.
- ben_w 2y agoMostly agree. Until ChatGPT, I'd have agreed with all of that. > Helping a group of people reach a solution everyone is happy with has little to do with technology. This one specific thing, is actually something that ChatGPT can help with. It's not as good as the best human, or even a middling human with 5 year's business experience, but rather it's useful because it's good enough at so many different domains that it can be used to clarify thoughts and explain the boundaries of the possible — Google Translate for business jargon, though like Google Translate it is also still often wrong — the ultimate "jack of all trades, master of none".
- codr7 2y agoWe're currently in the shiny toy stage, once the flaws are thoroughly explored and accepted by all as fundamental I suspect interest will fade rapidly. There's no substance to be found, no added information; it's just repeating what came before, badly, which is exactly the kind of software that would be better off not written if you ask me. The plan to rebuild society on top of this crap is right up there with basing our economy on manipulating people into buying shit they don't need and won't last so they have to keep buying more. Because money.
- pbrowne011 2y ago> But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them. It's all claims that it's great with no further elaboration on the workflows. To give an example, one person (a researcher at DeepMind) recently wrote about specific instances of his uses of LLMs, with anecdotes about alternatives to each example. [1] People on HN had different responses with similar claims with elaborations on how it has changed some of their workflows. [2] While it would be interesting to see randomized controlled trials on LLM usage, hearing people's anecdotes brings to mind the (often misquoted) phrase: "The plural of anecdote is data". [3] [4] [1] https://nicholas.carlini.com/writing/2024/how-i-use-ai.html https://nicholas.carlini.com/writing/2024/how-i-use-ai.html [2] https://news.ycombinator.com/item?id=41150317 https://news.ycombinator.com/item?id=41150317 [3] http://blog.danwin.com/don-t-forget-the-plural-of-anecdote-is-data/ http://blog.danwin.com/don-t-forget-the-plural-of-anecdote-i... [4] originally misquoted as "Anecdote is the plural of data."
- kristianp 2y ago> (often misquoted) phrase You misquoted it there! It should be: The plural of anecdote is data.
- stavros 2y agoIt's actually "the plural of 'anecdote' is not 'data'".
- pbrowne011 2y ago
- ben_w 2y ago> But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them. I'm an iOS dev, my knowledge of JS and CSS is circa 2004. I've used ChatGPT to convert some of my circa 2009 Java games into browser games. > Chatting with a LLM look to me like your intent is either vague or you don't know the units to use Or that you're moving up the management track. Managers don't write code either. Some prefer it that way.
- joseluis 2y agoThe key difference is that this is a multidisciplinary conversational interface, and a tool in itself for interrelating structured meaning and reshaping it coherently enough so that it can be of great value both in the specific domain of the dialog, and in the potential to take it on any tangent in any way that can be expressed. Of course it has limitations and you can't be sleep at the wheel, but that's true of any tool or task.
- DiogenesKynikos 2y agoFor one, I spend less time on Stackoverflow. LLMs can usually give you the answer to little questions about programming or command-line utilities right away.
- fragmede 2y agoHere's one from simonw https://gist.github.com/simonw/97e29b86540fcc627da4984daf5b7f9f https://gist.github.com/simonw/97e29b86540fcc627da4984daf5b7... There are more to be found on his blog on the ai-assisted-programming tag. https://simonwillison.net/tags/ai-assisted-programming/ https://simonwillison.net/tags/ai-assisted-programming/
- sweeter 2y agoOr people say "I've been pumping out thousands of lines of perfectly good code by writing paragraphs and paragraphs of text explaining what I want!" its like what are you programming dog? and they will never tell you, and then you look at their github and its like a dead simple starter project. I recently built a Brainfuck compiler and TUI debugger and I tested out a few LLM's just to see if I could get some useful output regarding a few niche and complicated issues, and it just gave me garbage that looked mildly correct. Then I'm told its because I'm not prompting hard enough... I'd rather just learn how to do it at that point. Once I solve that problem, I can solve it again in the future in .25x the time.
- KHRZ 2y agoOne good use case is unit tests, since they can be trivial while at the same time being cumbersome to make. I could give the LLM code for React components, and it would make the tests and setup all the mocks which is the most annoying part. Although making "all the tests" will typically involve asking the LLM again to think of more edge cases and be sure to cover everything.
- rtsil 2y agoHere's the thing. 99% of people aren't writing compilers or debuggers, they're writing glorified CRUDs. LLM can save a lot of time for these people, just like 99% of people only use basic arithmetic operations, and MS Excel saves a lot of time for these people. It's not about solving new problems, it's about solving old and known problems very fast.
- csmpltn 2y ago> "99% of people aren't writing compilers or debuggers" Look, I get the hype - but I think you need to step outside a bit before saying that 99% of the software out there is glorified CRUDs... Think about the aerospace/defense industries, autonomous vehicles, cloud computing, robotics, sophisticated mobile applications, productivity suites, UX, gaming and entertainment, banking and payment solutions, etc. Those are not small industries - and the software being built there is often highly domain-specific, has various scaling challenges, and takes years to build and qualify for "production". Even a simple "glorified CRUD", at a certain point, will require optimizations, monitoring, logging, debugging, refactoring, security upgrades, maintenance, etc... There's much more to tech than your weekend project "Facebook but for dogs" success story, which you built with ChatGPT in 5 minutes...
- px1999 2y agoSpecifically within the last week, I have used Claude and Claude via cursor to: - write some moderately complex powershell to perform a one-off process - add typescript annotations to a random file in my org's codebase - land a minor feature quickly in another codebase - suggest libraries and write sample(ish) code to see what their rough use would look like to help choose between them for a future feature design - provide text to fill out an extensive sales RFT spreadsheet based on notes and some RAG - generat some very domain-specific realistic sounding test data (just naming) - scaffold out some PowerPoint slides for a training session There are likely others (LLMs have helped with research and in my personal life too) All of these are things that I could do (and probably do better) but I have a young baby at the moment and the situation means that my focus windows are small and I'm time poor. With this workflow I'm achieving more than I was when I had fully uninterrupted time.
- Workaccount2 2y agoJust be mindful that it is one's interest to push the "LLMs suck, don't waste your time with them" narrative once they figure out how to harness LLMs. "Jason is a strong coder, and he despises AI tools!"
- Kiro 2y agoYou clearly have made up your mind that it can't be right but to me it's like arguing against breathing. There are no uncertainties or misunderstandings here. The productivity gains are real and the code produced is more robust. Not in theory, but in practice. This is a fact for me and you trying to convince me otherwise is just silly when I have the result right in front of me. It's also not just boilerplate. It's all code.
- mhuffman 2y ago>There are no uncertainties or misunderstandings here. The productivity gains are real and the code produced is more robust. Not in theory, but in practice. So, that may be a fact for you but there are mixed results when you go out wide. For example [1] has this little nugget: >The study identifies a disconnect between the high expectations of managers and the actual experiences of employees using AI. >Despite 96% of C-suite executives expecting AI to boost productivity, the study reveals that, 77% of employees using AI say it has added to their workload and created challenges in achieving the expected productivity gains. Not only is AI increasing the workloads of full-time employees, it’s hampering productivity and contributing to employee burnout. So not everyone is feeling the jump in productivity the same way. On this very site, there are people claiming they are blasting out highly-complex applications faster than they ever could, some of them also claiming they don't even have any experience programming. Then others claiming that LLMs and AI copilots just slow them down and cause much more trouble than they are worth. It seems like just with programming itself, that different people are getting different results. [1]https://www.forbes.com/sites/bryanrobinson/2024/07/23/employees-report-ai-increased-workload/ https://www.forbes.com/sites/bryanrobinson/2024/07/23/employ...
- Workaccount2 2y agoI have used chatGPT to write test systems for our (physical) products. I have a pretty decent understanding of how code/programs works structurally, I just don't know the syntax/language (Python in this case). So I can translate things like "Create an array, then query this instrument for xyz measurements, then store those measurements in the array. Then store that array in the .csv file we created before" It works fantastic and saved us from outsourcing.
- lifeformed 2y agoI think people who are successfully using it to write code are just chaining APIs together to make the same web apps you see everywhere.
- imiric 2y agoThe vast majority of software is "just chaining APIs together". It makes sense that LLMs would excel at code they've been trained on the most, which means they can be useful to a lot of people. This also means that these people will be the first to be made redundant by LLMs, once the quality improves enough.
- elbear 2y agoI would say all software is chaining APIs together.
- imiric 2y agoWell, that depends on how you look at it. All software calls APIs, but some rely on literally "just chaining" these calls together more than writing custom behavior from scratch. After all, someone needs to write the APIs to begin with. That's not to say that these projects aren't useful or valuable, but there's a clear difference in the skill required for either. You could argue that it's all APIs down to the hardware level, but that's not a helpful perspective in this discussion.
- elbear 2y ago| You could argue that it's all APIs down to the hardware level, but that's not a helpful perspective in this discussion. Yes, that's what I'm arguing. Why isn't useful? I think it's useful, because it demystifies things. You know that in order to do something, you need to know how to use the particular API.
- NetOpWibby 2y ago> The fact everyone that say they've become more productive with LLMs won't say how exactly. Anecdotally, I no longer use StackOverflow. I don’t have to deal with random downvotes and feeling stupid because some expert with a 10k+ score on 15 SE sites each votes my question to be closed. I’m pretty tech savvy, been doing development for 15 years, but I’m always learning new things. I can describe a rough idea of what I want to an LLM and get just enough code for me to hit the ground running…or, I can ask a question in forum and twiddle my thumbs and look through 50 tabs to hopefully stumble upon a solution in the meantime. I’m productive af now. I was paying for ChatGPT but Claude has been my goto for the past few months.
- sumedh 2y ago> The fact everyone that say they've become more productive with LLMs won't say how exactly. I have python scripts which do lot of automation like downloading pdfs, bookmarking pdfs, processing them, etc. Thanks to LLMs I dont write a python code myself, I just ask an LLM to write it, I just provide the requirement. I just copy the code generated by the AI model and run it. If there any errors, I just ask AI to fix it.
- yuppiemephisto 2y ago> The fact everyone that say they've become more productive with LLMs won't say how exactly. But no one really show how they're actually solving problems with LLMs and how the alternatives were worse for them. A pretty literal response: https://www.youtube.com/@TheRevAlokSingh/streams https://www.youtube.com/@TheRevAlokSingh/streams Plenty of Lean 4 and Cursor.