8 ms·

I’m a little shocked at how much negativity there is around LLMs among developers. It’s a new tool that requires some learning, and it’s sometimes not so great,

by lukevp 2y ago

I’m a little shocked at how much negativity there is around LLMs among developers. It’s a new tool that requires some learning, and it’s sometimes not so great, but if you’ve used an IDE with real coding assistance built in (eg. VS Code in Edit with Copilot mode - NOT Chat mode, using Claude 3.5), it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10 seconds later. The amount of speed up I see as a very experienced dev is astronomical. And just like 6 months ago it was awful. How great is it gonna be in a year or two? It doesn’t even have access to running unit tests or reading console errors or IDE hints, and it still generates mostly correct code. Once it gets more deeply embedded it’s just going to improve more and more.

mlboss 2y ago
It is difficult to get a man to understand something, when his salary depends on his not understanding it.
- jayd16 2y ago
  On this site, the adage cuts both ways.
jayd16 2y ago
Unreliable tools are utterly exhausting. > not much worse than a junior dev and 100x faster. Is there a greater hell than this!?
- thfuran 2y ago
  Worse than an intern and 1000x faster?
- thayne 2y ago
  At least with a junior dev, I can teach them how to do it better next time. Not so much with generative "AI".
  - anon7000 2y ago
    Not totally. But you might be surprised at the things you can do. Cursor has some template-like files where you can basically teach the AI “when we do X, do it this way.” Or you can change the global prompt to add the things it should keep in mind when working with you. If you actually take the time to tell it “hey, don’t do it this way,” it can definitely do it differently the next time. On top of that, is anyone training models on their own codebase, and noting to AI which patterns are best practice and which aren’t? There are a ton of ways to make it better than the baseline copilot experience
- nxpnsv 2y ago
  That's 100x more bugs to fix. Moreover, increasingly complex models produce bugs that are increasingly hard to spot and fix.
- stavros 2y ago
  "It makes lots of mistakes, but at least it makes them quickly!"
- Lio 2y ago
  If the old metric is right, that it is ten times harder to debug code than to write it, having something that writes buggy code 100x faster than you can understand it is a problem. Especially given that you can ask an LLM to optimise code and on multiple runs it can not tell if it's is improving or degenerating.
- heresie-dabord 2y ago
  > a junior dev and 100x faster. > Is there a greater hell than this!? Yes — junior management using LLMs and 100x more cocksure.
- bigstrat2003 2y ago
  I am of the firm belief that unreliable help is worse than no help at all. LLMs are unreliable help, therefore they are useless to me.
- daveguy 2y ago
  LLMs are the "move fast and break things" of AI.
pton_xd 2y ago
I think the usefulness is just very domain specific. If you're writing some types of boilerplate or often-tutorialized code it can spit out something very reasonable. Other types of code, like say in game dev, it stumbles around and never produces anything usable. But like you said, in a few more years we'll see! It does feel like there's some missing pieces yet to be figured out to truly "reason" and generalize.
- ROM_Bombadil 2y ago
  > If you're writing some types of boilerplate or often-tutorialized code it can spit out something very reasonable. Other types of code, like say in game dev, it stumbles around and never produces anything usable. This makes me think of a quirk I discovered recently which is that ChatGPT simply won't generate a picture of a 'full glass of wine'. It generates pictures with all sorts of crazy waves/splashes in the glass but the glass is always half full no matter how you prompt it. I'm not enough of an expert to make any deductions from this, but I think it hints at what the limitations of the currently models are.
idonotknowwhy 2y ago
Yeah I'm surprised by all the negativity as well. I'm listening to the post right now (using xtts-v2 finetuned on a voice I like lol). Sounds like these companies are overvalued / over hyped. Maybe they are / some of these companies go the way of myspace, but LLMs are incredibly useful for me. I'm able to do a so much more using LLMs (Mistral-Large, Qwen2.5 and R1 locally, Claude via API) than without them. I have to get the IDE setup properly now.
- spaceman_2020 2y ago
  Personally, I've found DeepSeek R1 to be a profoundly good model for thinking through problems across fields. I had a complex finance situation that I was struggling with, both from a mathematical/taxation perspective and a personal psychological finance hangup. I spent a few good hours talking to it through everything and had a mental breakthrough. To get the same kind of insight, I would have to pay a financial advisor AND a psychologist for several hours. That all of this was free while someone calls it a "con" seems completely wrong (I got my CFA cousin to look over the numbers and he agreed with R1's advice, fwiw)
  - idonotknowwhy 2y ago
    Yeah, I've had similar experiences. I still hesitate if it's a field I don't know too well of course (never trust an LLM), but R1 has been able to solve things I've been stuck on. And watching it's <think></think> process has been insightful. Only issue is that it ties up all my GPUs while I run it. Hopefully Mistral can copy their technique and give us a 123b reasoning model.
  - cess11 2y ago
    Did you run this locally?
mitthrowaway2 2y ago
LLMs are pretty good at the aspects of coding that I consider to be "the fun part". Using them has made me more productive, but also made my job less fun, because I can't justify spending time using my own brain to do "the fun part" on my employer's dime. And that was something I was particularly good at, which is why I was able to be paid well to do it. So now my company makes more money, and the work gets done faster, but I can't say I feel appreciative. I'm sure it's great for founders though, for whom doing the work is merely an obstacle to having the finished product. For me, the work is the end goal, because I'm not hired to own the result.
- froidpink 2y ago
  It's kind of analogous to the old taxi drivers who took pride in having a sixth sense knowing which route to take you, vs uber drivers who just blindly follow their navigation
  - portaouflop 2y ago
    Some of them might have had a really good mental map; but the majority would just take inefficient routes (and charge you some random price that they put into their counter) — plenty of reasons to dislike Uber but having a pre-set price, vetted/rated drivers, and clear routing for a taxi service is a massive plus in my opinion.
- stevage 2y ago
  Huh, for me it's the opposite. It does the boring bit, writing pedestrian method bodies. Writing import statements. Closing tags. I do the fun bit: having creative ideas and trying them out.
  - Piskvorrr 2y ago
    Looks like you haven't used a decent IDE: these things have been standard for decades, locally and with minimal requirements. But wait, now it happens in the Cloud (meh, that's not gonna fly anymore, too last decade)...AND requires massive amounts of power AND cooling, PLUS it's FUBAR about 50/50. For an incremental improvement...not great, not terrible.
stuckkeys 2y ago
Honestyl. It has its drawbacks but I am usually at 50x with few different agents running side by side. What we need is better GPU competition with tons of ram.
- notpachet 2y ago
  Doesn't that just scream "bad design" at you? Shouldn't we be aiming for agents that require less GPU? And agents that are good enough that we don't have to shop around for "competing prices" on answers?
guyfromfargo 2y ago
We’re already here. Check out Cline or Roocode. The technology is incredible. I wrote a custom MCP to grab tickets from my Kanban board, Roo will pull down the tickets and start implementing them. I then have another agent that QAs, and either kicks the ticket back, or moves the ticket to human review. I’m doing this on a real world micro SaaS. It has about 50 paying customers, and I’m the sole developer. I did a complete rewrite and the AI was able to complete about 90% of the project. I estimate I can get about a week’s worth of coding done in a day with this setup. I haven’t even scratched the surface of optimizing this workflow. I’m also just one guy working nights and weekends, I’m sure there are many startups solving this same problem. It’s amazing to be shipping features this quick, but as a developer I’m terrified of what this is going to do to our careers.
xyzzy123 2y ago
For easy things, LLM assist has sped things up a lot for me. For medium complexity things, I can get them done quickly without manual coding if I have a clear understanding in mind of what the implementation should look like. I supply the requirements, design and strategy and it's fairly easy to "keep things on the rails". The "write a PRD first" hack (https://www.aiagentshub.net/blog/how-to-10x-your-development-speed-with-cursor-ai https://www.aiagentshub.net/blog/how-to-10x-your-development...) works pretty well. Agent with YOLO mode and terminal access rips, particularly if you have good tests. For tasks where I know the spec of the feature but don't clearly understand how I would design / implement the feature myself, it's hit-and-miss. Mostly miss for me. I also haven't had much success with niche libraries, have to stick to the most popular library/tool/framework choices or it will quickly get stuck.
- XorNot 2y ago
  Whereas I've been disabling AI assist features because I find them actively disruptive to the development process. When it ghost pops up text suggesting what I should do, it's sometimes right...but it breaks flow. It forces me to read and parse apparently correct code, and decide if it is correct or it's just a mirage which is valid but not actually what I'm doing at all.
danroblew 2y ago
The article is about how the economics of the LLM market is making all tech look bad. They need trillions of dollars in returns. VC's won't finance tech startups for decades. I use Cursor sometimes, and VSCode + Continue with llama.cpp, and it's great. That's not worth billions. It's definitely not worth trillions.
- automatic6131 2y ago
  This is the crux. A cool thing has been invented, with real usages. Unfortunately, it's cost hundreds of billions of dollars and it has absolutely zero hope of making the trillions needed to justify that. Now someone will respond about how it's just a stepping stone, and how the billions are justified by _something completely imaginary, and not invented yet, and maybe not ever_ e.g. agents.
  - trhway 2y ago
    >it's cost hundreds of billions of dollars and it has absolutely zero hope of making the trillions needed to justify that. The BigTech companies have been flush with liquidity and poured those hundreds of billions into the promising tech, and as result we got a wonderful new technology. There is not much need for those trillions in return - just look at liquidity positions of those companies, they are just fine. If those trillions come in eventually - even better.
    - automatic6131 2y ago
      >There is not much need for those trillions in return Whilst you are correct that big tech cos do not need the return to survive, that's not how public markets work at all, and thus not how the incentives for those in charge of the companies work, and so making you actually wrong.
      - trhway 2y ago
        If i were wrong, those companies would be distributing that cash to shareholders instead of chasing any promise of any big chance. If investment in AI don't pan out (i do think that it will pan out, and those trillions will come) then those companies would just pour even more billions into whatever big thing/promise would come next. Rinse and repeat. Because some of those things do generate tremendous returns, and thus not playing that game is what really constitute true loss of money.
SolarNet 2y ago
I feel like you missed the first third of this article that was quite clear they are not saying there are no uses cases. They are saying there doesn't seem to be an economic model that makes sense.
jemmyw 2y ago
I was thinking the same but it's not really what the post is about. They talk about there are use cases for LLMs and devs can be benefiting. What it goes into is how over hyped and over valued these companies are. They've blown through $5bn of compute each in a year and their revenue is abysmal. Microsoft won't report on ai separately, probably because it's abysmal. I'm positive on LLMs for coding. But I think I have to agree with their assessment. Coding seems like the best area for these tools and what we see now is great. It's probably even worth $10b to the IT industry maybe eventually. But they're not paying for it yet, clearly. And I also think it's just not going to have huge significance outside our industry. The people I rub shoulders with outside of work have not mentioned or asked about it once, which is not necessarily meaningful but it does reveal the limits of hype too.
roenxi 2y ago
> I’m a little shocked at how much negativity there is around LLMs among developers. While the timeline is unclear; it seems likely that LLMs will obsolete precisely the skills that developers use to earn their income. I imagine a lot of them feel rather threatened by the rapid rate of progress. Pointing out that it is already operating at junior dev quality and rapidly improving is unlikely to quiet the discontent.
- surgical_fire 2y ago
  I use LLMs in coding. There are Junior Devs in my team. If you think LLMs operate at "junior dev" capacity you either don't work with junior devs and is just bullshitting your way around here, or you just pick pretty awful junior devs. LLMs are alright. An okay productivity tool, although its inconsistencies many times nullify productivity gains - By design they often spit out wrong results that look and sound very plausible. A productivity blackhole. Its mistakes are sometimes hard to spot, but pervasive. Beyond that, if your think that all a dev does is spit out code, and since LLMs can spit out code it can replace devs in some imaginary timeline, you are sorely mistaken. The least part of my work is actually spitting out code, although it is the part I enjoy the most. I honestly feel way nore threatened by economic downturns and the looming threat of recession. The only way LLMs threaten me is by being a wasteful technology that may precipitate a downturn in tech companies, causing more layoffs, etc nd so forth.
  - trashtester 2y ago
    If you define "junior" based mostly on age, then LLM's aren't yet at the level of a good "junior". If you base it on ability, then an LLM can be be more useful to a good developer than 1 or more less competent "junior" team members (regardless of their age). Not because it can do all the things like any "junior" can (like make coffee), but because the things it can do on top of what a "junior" can do, more than makes up for it.
    - deleted 2y ago
      [deleted]
  - 2y ago
forgetfreeman 2y ago
Conversely I'm shocked the negativity hasn't graduated to naked hostility among developers. A group that tends to pride itself on clarity of thought entranced by bullshit generators? A group that tends to pride itself on correctness of work cheerfully adding tools to their workflow that provably fuck up in unpredictable ways and that have to be monitored constantly for just such behavior? Why not hire a few junior devs instead if that's your jam, at least you can train a human towards competence.
enpera 2y ago
No matter how widespread Copilot becomes, it won't make OpenAI profitable, nor will it enable Sam Altman or Jensen Huang to complete their apps.
Kappa90 2y ago
If you want to see an LLM tackling unit testing, check https://www.codebeaver.ai https://www.codebeaver.ai Disclaimer: I’m the developer behind CodeBeaver
agentultra 2y ago
That experience is heavily subsidized and is unprofitable for these companies providing it based on what we know. Even with all of the other developers who are also using the same work flow and espousing how great it is. Even with all of the monthly subscribers at various tiers. It has been unprofitable for several years and continues to be unprofitable and will likely remain unprofitable given current trends. The author spends a good amount of bytes telling us that they don't want to hear this argument even though they expect it.
- square_usual 2y ago
  It is unprofitable because they keep spending money developing new AI. Inference for existing AI is not unprofitable.
  - sgt101 2y ago
    For now. Unless closed models have significant advantage AI inference will be a commodity business - like server hosting. I'm not sure that closed models will maintain an advantage.
- apstls 2y ago
  I think these types of arguments need to at the very least acknowledge the distribution of cost between training and inference.
  - agentultra 2y ago
    Perhaps, and the externalities often unaccounted for or hand-waved away. Even the US Government is getting involved in subsidizing these companies and all of the infrastructure and resources needed to keep it expanding. We can look forward to even more methane power plants, more drilling, more fracking, more noisy data-centres sucking up fresh water from local reserves and increased damage to the environment that will come out of the pocket books of... ? Update: And for what? "Deep Research"? Apparently it's not that great or world-changing for the costs involved. It seems that the author is tired of the yearly promise that everything is just a year or two away as long as we keep shovelling more money and resources into the furnace.
dr-detroit 2y ago
[dead]
tehologist 2y ago
I personally think IDE makes for worse programmers vs just using a text editor.
dartos 2y ago
> it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10 seconds later. The amount of speed up I see as a very experienced dev is astronomical Personally, I find that waiting for the code to generate, then reviewing the code carefully, then deciding if I need to rewrite it to be more painful, more error prone, and much slower than writing the code correctly. Especially since this AI junior never learn from it’s mistakes. I think it speaks to different approaches to how individuals write code. > How great is it gonna be in a year or two? I would bet that it’s about the same (not great code, generally), but the tools fail to generate responses less often and likely would have more context. Hopefully they become fast enough to run offline or at least feel more instantaneous.
delusional 2y ago
Writing the code isn't the hard part, and wrangling the computer is the part of the work I enjoy most. My problem with AI bros is that they explicitly want to automate all the shit people like to do, such that we can all finally be free to work service jobs. I happen to value human creativity.
Piskvorrr 2y ago
So, in essence, it's now incrementally better than a templating script (except when worse), but Have Faith, it will be Better Soon. TBH, that's the same song that's been on repeat since the Dartmouth Workshop. In 1956. Jam yesterday and jam tomorrow, never any jam today.
vonneumannstan 2y ago
It's just status anxiety. Mid engineers go on and on claiming theres literally no value from LLMs even possible in principle while top tier people are using them as force multipliers.
- jcgrillo 2y ago
  > top tier people Who? How? This is not what I've seen where I work. There's a bunch of hubbub and generalized excitement, and lots of talk about what could be done, or what might be done, but not very much actual doing. I must just be a clueless "mid".
  - vonneumannstan 2y ago
    Yeah unironically. Guido van Rossum - "I use it every day. My biggest adjustment with using Copilot was that instead of writing code, my posture shifted to reviewing code." https://www.youtube.com/watch?v=-DVyjdw4t9I https://www.youtube.com/watch?v=-DVyjdw4t9I Here's Jeff Dean saying 25% of the characters in new PRs at Google are AI Generated. https://www.dwarkeshpatel.com/p/jeff-dean-and-noam-shazeer https://www.dwarkeshpatel.com/p/jeff-dean-and-noam-shazeer Andrej Karpathy - "I basically can't imagine going back to "unassisted" coding at this point" https://www.reddit.com/r/singularity/comments/1ezssll/andrej_karpathy_programming_is_changing_so_fast/#lightbox https://www.reddit.com/r/singularity/comments/1ezssll/andrej... Andrew Ng - "I run multiple models on my laptop — Mistral, Llama, Zefa. And I use ChatGPT quite often. " https://www.ft.com/content/2dc07f9e-d2a9-4d98-b746-b051f9352be3 https://www.ft.com/content/2dc07f9e-d2a9-4d98-b746-b051f9352... Simon Willison https://simonwillison.net/2023/Mar/27/ai-enhanced-development/ https://simonwillison.net/2023/Mar/27/ai-enhanced-developmen... I mean I can keep going. I doubt you would compare yourself to these people.
    - habinero 2y ago
      These kinds of responses are my favorite dark pattern rhetorical device, because you can assert literally _anything_ in this format and almost nobody will refute you, because the cost of refuting bullshit is 100x the cost of producing it. Anyways, here goes. 1. Guido uses Copilot like I do - as a StackOverflow replacement to write the dumb boilerplate code. A much less flattering quote is "It doesn't save me much thinking, but [it helps] because I'm a poor typist". Also it's literally a minute or two of a three hour podcast. 2. A lot of code is autogenerated lol. Again, it's all the boring boilerplate stuff. 3. The cofounder of OpenAI is a biased source lol 4. He's an AI researcher, of course he runs that stuff. 5. Again, similar to Guido. He's using it for the boilerplate. Nothing wrong with enjoying using it as a toy, as he is here. But he's not doing serious work with it. There's no virtue in hyping this stuff like a HODL bitcoin cultist.
bigstrat2003 2y ago
I disagree. Tools which need to be babysat, the way LLMs do, slow you down rather than speed you up. It's like having to mentor a junior team member, except a human will eventually learn and you can just let him work. LLMs are incapable of learning, so you can't ever leave the phase where they are a drain.
paulcole 2y ago
> I’m a little shocked at how much negativity there is around LLMs among developers There's a Quentin Tarantino quote where he says there are 2 kinds of film critics. There are those who love movies and there are those that love the movies they love. A lot of developers really seem to love the technology they love. These people are where most of the negativity is coming from. And my guess is that the people who are encouraged by LLMs and not negative (mostly) aren't taking time out of their days to write long blog posts or argue about it online.
- habinero 2y ago
  I use Copilot for autocompleting the boring boilerplate. I like it. I also think LLMs are mostly useless. It's not the technology, it's the stupid overhype. It really feels like all the HODL bitcoin cultists have finally gotten over their lost apes and found a new technogod. So many people in these threads are convinced it's about to gain sentience. That's not going to happen. You get the people outing themselves by saying "it does my job better than me!" If you say something honest and direct like "their output is mediocre and unreliable" or "the RNG text generator is not capable of thinking, you're just Clever Hans-ing yourself" or "if it does your job better than you, that says more about you than about it", you get people clutching their pearls like a Stanford parent whose kid got a D. arXiv has turned into a paper mill of AI startups uploading marketing hype cosplaying as "research".
synicalx 2y ago
I don't really get the hate; work is boring, if some tool can make it happen faster and with less effort I'm all for it. I don't hear the line cooks at McDonalds complaining that they have to use a semi-automated grill that beeps at them instead of an open fire.
michaelcampbell 2y ago
As a developer I'm seeing less hate than apathy from my colleagues, but rather the hatred I do see is from people trying to push LLM's ON developers. So it's from middle management levels riding the hype train, and possibly trying to save money and getting bonuses for it at the expense of other people. Just like when offshoring was in its same point on Gartner's hype curve. "Everyone has a model, but no one has a business".

by lukevp 2y ago

mlboss 2y ago

It is difficult to get a man to understand something, when his salary depends on his not understanding it.

jayd16 2y ago
On this site, the adage cuts both ways.

jayd16 2y ago

Unreliable tools are utterly exhausting. > not much worse than a junior dev and 100x faster. Is there a greater hell than this!?

thfuran 2y ago
Worse than an intern and 1000x faster?
thayne 2y ago
At least with a junior dev, I can teach them how to do it better next time. Not so much with generative "AI".
- anon7000 2y ago
  Not totally. But you might be surprised at the things you can do. Cursor has some template-like files where you can basically teach the AI “when we do X, do it this way.” Or you can change the global prompt to add the things it should keep in mind when working with you. If you actually take the time to tell it “hey, don’t do it this way,” it can definitely do it differently the next time. On top of that, is anyone training models on their own codebase, and noting to AI which patterns are best practice and which aren’t? There are a ton of ways to make it better than the baseline copilot experience
nxpnsv 2y ago
That's 100x more bugs to fix. Moreover, increasingly complex models produce bugs that are increasingly hard to spot and fix.
stavros 2y ago
"It makes lots of mistakes, but at least it makes them quickly!"
Lio 2y ago
If the old metric is right, that it is ten times harder to debug code than to write it, having something that writes buggy code 100x faster than you can understand it is a problem. Especially given that you can ask an LLM to optimise code and on multiple runs it can not tell if it's is improving or degenerating.
heresie-dabord 2y ago
> a junior dev and 100x faster. > Is there a greater hell than this!? Yes — junior management using LLMs and 100x more cocksure.
bigstrat2003 2y ago
I am of the firm belief that unreliable help is worse than no help at all. LLMs are unreliable help, therefore they are useless to me.
daveguy 2y ago
LLMs are the "move fast and break things" of AI.

pton_xd 2y ago

I think the usefulness is just very domain specific. If you're writing some types of boilerplate or often-tutorialized code it can spit out something very reasonable. Other types of code, like say in game dev, it stumbles around and never produces anything usable. But like you said, in a few more years we'll see! It does feel like there's some missing pieces yet to be figured out to truly "reason" and generalize.

ROM_Bombadil 2y ago
> If you're writing some types of boilerplate or often-tutorialized code it can spit out something very reasonable. Other types of code, like say in game dev, it stumbles around and never produces anything usable. This makes me think of a quirk I discovered recently which is that ChatGPT simply won't generate a picture of a 'full glass of wine'. It generates pictures with all sorts of crazy waves/splashes in the glass but the glass is always half full no matter how you prompt it. I'm not enough of an expert to make any deductions from this, but I think it hints at what the limitations of the currently models are.

idonotknowwhy 2y ago

Yeah I'm surprised by all the negativity as well. I'm listening to the post right now (using xtts-v2 finetuned on a voice I like lol). Sounds like these companies are overvalued / over hyped. Maybe they are / some of these companies go the way of myspace, but LLMs are incredibly useful for me. I'm able to do a so much more using LLMs (Mistral-Large, Qwen2.5 and R1 locally, Claude via API) than without them. I have to get the IDE setup properly now.

spaceman_2020 2y ago
Personally, I've found DeepSeek R1 to be a profoundly good model for thinking through problems across fields. I had a complex finance situation that I was struggling with, both from a mathematical/taxation perspective and a personal psychological finance hangup. I spent a few good hours talking to it through everything and had a mental breakthrough. To get the same kind of insight, I would have to pay a financial advisor AND a psychologist for several hours. That all of this was free while someone calls it a "con" seems completely wrong (I got my CFA cousin to look over the numbers and he agreed with R1's advice, fwiw)
- idonotknowwhy 2y ago
  Yeah, I've had similar experiences. I still hesitate if it's a field I don't know too well of course (never trust an LLM), but R1 has been able to solve things I've been stuck on. And watching it's <think></think> process has been insightful. Only issue is that it ties up all my GPUs while I run it. Hopefully Mistral can copy their technique and give us a 123b reasoning model.
- cess11 2y ago
  Did you run this locally?

mitthrowaway2 2y ago

LLMs are pretty good at the aspects of coding that I consider to be "the fun part". Using them has made me more productive, but also made my job less fun, because I can't justify spending time using my own brain to do "the fun part" on my employer's dime. And that was something I was particularly good at, which is why I was able to be paid well to do it. So now my company makes more money, and the work gets done faster, but I can't say I feel appreciative. I'm sure it's great for founders though, for whom doing the work is merely an obstacle to having the finished product. For me, the work is the end goal, because I'm not hired to own the result.

froidpink 2y ago
It's kind of analogous to the old taxi drivers who took pride in having a sixth sense knowing which route to take you, vs uber drivers who just blindly follow their navigation
- portaouflop 2y ago
  Some of them might have had a really good mental map; but the majority would just take inefficient routes (and charge you some random price that they put into their counter) — plenty of reasons to dislike Uber but having a pre-set price, vetted/rated drivers, and clear routing for a taxi service is a massive plus in my opinion.
stevage 2y ago
Huh, for me it's the opposite. It does the boring bit, writing pedestrian method bodies. Writing import statements. Closing tags. I do the fun bit: having creative ideas and trying them out.
- Piskvorrr 2y ago
  Looks like you haven't used a decent IDE: these things have been standard for decades, locally and with minimal requirements. But wait, now it happens in the Cloud (meh, that's not gonna fly anymore, too last decade)...AND requires massive amounts of power AND cooling, PLUS it's FUBAR about 50/50. For an incremental improvement...not great, not terrible.

stuckkeys 2y ago

Honestyl. It has its drawbacks but I am usually at 50x with few different agents running side by side. What we need is better GPU competition with tons of ram.

notpachet 2y ago
Doesn't that just scream "bad design" at you? Shouldn't we be aiming for agents that require less GPU? And agents that are good enough that we don't have to shop around for "competing prices" on answers?

guyfromfargo 2y ago

We’re already here. Check out Cline or Roocode. The technology is incredible. I wrote a custom MCP to grab tickets from my Kanban board, Roo will pull down the tickets and start implementing them. I then have another agent that QAs, and either kicks the ticket back, or moves the ticket to human review. I’m doing this on a real world micro SaaS. It has about 50 paying customers, and I’m the sole developer. I did a complete rewrite and the AI was able to complete about 90% of the project. I estimate I can get about a week’s worth of coding done in a day with this setup. I haven’t even scratched the surface of optimizing this workflow. I’m also just one guy working nights and weekends, I’m sure there are many startups solving this same problem. It’s amazing to be shipping features this quick, but as a developer I’m terrified of what this is going to do to our careers.

xyzzy123 2y ago

For easy things, LLM assist has sped things up a lot for me. For medium complexity things, I can get them done quickly without manual coding if I have a clear understanding in mind of what the implementation should look like. I supply the requirements, design and strategy and it's fairly easy to "keep things on the rails". The "write a PRD first" hack (https://www.aiagentshub.net/blog/how-to-10x-your-development-speed-with-cursor-ai https://www.aiagentshub.net/blog/how-to-10x-your-development...) works pretty well. Agent with YOLO mode and terminal access rips, particularly if you have good tests. For tasks where I know the spec of the feature but don't clearly understand how I would design / implement the feature myself, it's hit-and-miss. Mostly miss for me. I also haven't had much success with niche libraries, have to stick to the most popular library/tool/framework choices or it will quickly get stuck.

XorNot 2y ago
Whereas I've been disabling AI assist features because I find them actively disruptive to the development process. When it ghost pops up text suggesting what I should do, it's sometimes right...but it breaks flow. It forces me to read and parse apparently correct code, and decide if it is correct or it's just a mirage which is valid but not actually what I'm doing at all.

danroblew 2y ago

The article is about how the economics of the LLM market is making all tech look bad. They need trillions of dollars in returns. VC's won't finance tech startups for decades. I use Cursor sometimes, and VSCode + Continue with llama.cpp, and it's great. That's not worth billions. It's definitely not worth trillions.

automatic6131 2y ago
This is the crux. A cool thing has been invented, with real usages. Unfortunately, it's cost hundreds of billions of dollars and it has absolutely zero hope of making the trillions needed to justify that. Now someone will respond about how it's just a stepping stone, and how the billions are justified by _something completely imaginary, and not invented yet, and maybe not ever_ e.g. agents.
- trhway 2y ago
  >it's cost hundreds of billions of dollars and it has absolutely zero hope of making the trillions needed to justify that. The BigTech companies have been flush with liquidity and poured those hundreds of billions into the promising tech, and as result we got a wonderful new technology. There is not much need for those trillions in return - just look at liquidity positions of those companies, they are just fine. If those trillions come in eventually - even better.
  - automatic6131 2y ago
    >There is not much need for those trillions in return Whilst you are correct that big tech cos do not need the return to survive, that's not how public markets work at all, and thus not how the incentives for those in charge of the companies work, and so making you actually wrong.
    - trhway 2y ago
      If i were wrong, those companies would be distributing that cash to shareholders instead of chasing any promise of any big chance. If investment in AI don't pan out (i do think that it will pan out, and those trillions will come) then those companies would just pour even more billions into whatever big thing/promise would come next. Rinse and repeat. Because some of those things do generate tremendous returns, and thus not playing that game is what really constitute true loss of money.

SolarNet 2y ago

I feel like you missed the first third of this article that was quite clear they are not saying there are no uses cases. They are saying there doesn't seem to be an economic model that makes sense.

jemmyw 2y ago

I was thinking the same but it's not really what the post is about. They talk about there are use cases for LLMs and devs can be benefiting. What it goes into is how over hyped and over valued these companies are. They've blown through $5bn of compute each in a year and their revenue is abysmal. Microsoft won't report on ai separately, probably because it's abysmal. I'm positive on LLMs for coding. But I think I have to agree with their assessment. Coding seems like the best area for these tools and what we see now is great. It's probably even worth $10b to the IT industry maybe eventually. But they're not paying for it yet, clearly. And I also think it's just not going to have huge significance outside our industry. The people I rub shoulders with outside of work have not mentioned or asked about it once, which is not necessarily meaningful but it does reveal the limits of hype too.

roenxi 2y ago

> I’m a little shocked at how much negativity there is around LLMs among developers. While the timeline is unclear; it seems likely that LLMs will obsolete precisely the skills that developers use to earn their income. I imagine a lot of them feel rather threatened by the rapid rate of progress. Pointing out that it is already operating at junior dev quality and rapidly improving is unlikely to quiet the discontent.

surgical_fire 2y ago
I use LLMs in coding. There are Junior Devs in my team. If you think LLMs operate at "junior dev" capacity you either don't work with junior devs and is just bullshitting your way around here, or you just pick pretty awful junior devs. LLMs are alright. An okay productivity tool, although its inconsistencies many times nullify productivity gains - By design they often spit out wrong results that look and sound very plausible. A productivity blackhole. Its mistakes are sometimes hard to spot, but pervasive. Beyond that, if your think that all a dev does is spit out code, and since LLMs can spit out code it can replace devs in some imaginary timeline, you are sorely mistaken. The least part of my work is actually spitting out code, although it is the part I enjoy the most. I honestly feel way nore threatened by economic downturns and the looming threat of recession. The only way LLMs threaten me is by being a wasteful technology that may precipitate a downturn in tech companies, causing more layoffs, etc nd so forth.
- trashtester 2y ago
  If you define "junior" based mostly on age, then LLM's aren't yet at the level of a good "junior". If you base it on ability, then an LLM can be be more useful to a good developer than 1 or more less competent "junior" team members (regardless of their age). Not because it can do all the things like any "junior" can (like make coffee), but because the things it can do on top of what a "junior" can do, more than makes up for it.
  - deleted 2y ago
    [deleted]
- 2y ago

forgetfreeman 2y ago

Conversely I'm shocked the negativity hasn't graduated to naked hostility among developers. A group that tends to pride itself on clarity of thought entranced by bullshit generators? A group that tends to pride itself on correctness of work cheerfully adding tools to their workflow that provably fuck up in unpredictable ways and that have to be monitored constantly for just such behavior? Why not hire a few junior devs instead if that's your jam, at least you can train a human towards competence.

enpera 2y ago

No matter how widespread Copilot becomes, it won't make OpenAI profitable, nor will it enable Sam Altman or Jensen Huang to complete their apps.

Kappa90 2y ago

If you want to see an LLM tackling unit testing, check https://www.codebeaver.ai https://www.codebeaver.ai Disclaimer: I’m the developer behind CodeBeaver

agentultra 2y ago

That experience is heavily subsidized and is unprofitable for these companies providing it based on what we know. Even with all of the other developers who are also using the same work flow and espousing how great it is. Even with all of the monthly subscribers at various tiers. It has been unprofitable for several years and continues to be unprofitable and will likely remain unprofitable given current trends. The author spends a good amount of bytes telling us that they don't want to hear this argument even though they expect it.

square_usual 2y ago
It is unprofitable because they keep spending money developing new AI. Inference for existing AI is not unprofitable.
- sgt101 2y ago
  For now. Unless closed models have significant advantage AI inference will be a commodity business - like server hosting. I'm not sure that closed models will maintain an advantage.
apstls 2y ago
I think these types of arguments need to at the very least acknowledge the distribution of cost between training and inference.
- agentultra 2y ago
  Perhaps, and the externalities often unaccounted for or hand-waved away. Even the US Government is getting involved in subsidizing these companies and all of the infrastructure and resources needed to keep it expanding. We can look forward to even more methane power plants, more drilling, more fracking, more noisy data-centres sucking up fresh water from local reserves and increased damage to the environment that will come out of the pocket books of... ? Update: And for what? "Deep Research"? Apparently it's not that great or world-changing for the costs involved. It seems that the author is tired of the yearly promise that everything is just a year or two away as long as we keep shovelling more money and resources into the furnace.

dr-detroit 2y ago

[dead]

tehologist 2y ago

I personally think IDE makes for worse programmers vs just using a text editor.

dartos 2y ago

> it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10 seconds later. The amount of speed up I see as a very experienced dev is astronomical Personally, I find that waiting for the code to generate, then reviewing the code carefully, then deciding if I need to rewrite it to be more painful, more error prone, and much slower than writing the code correctly. Especially since this AI junior never learn from it’s mistakes. I think it speaks to different approaches to how individuals write code. > How great is it gonna be in a year or two? I would bet that it’s about the same (not great code, generally), but the tools fail to generate responses less often and likely would have more context. Hopefully they become fast enough to run offline or at least feel more instantaneous.

delusional 2y ago

Writing the code isn't the hard part, and wrangling the computer is the part of the work I enjoy most. My problem with AI bros is that they explicitly want to automate all the shit people like to do, such that we can all finally be free to work service jobs. I happen to value human creativity.

Piskvorrr 2y ago

So, in essence, it's now incrementally better than a templating script (except when worse), but Have Faith, it will be Better Soon. TBH, that's the same song that's been on repeat since the Dartmouth Workshop. In 1956. Jam yesterday and jam tomorrow, never any jam today.

vonneumannstan 2y ago

It's just status anxiety. Mid engineers go on and on claiming theres literally no value from LLMs even possible in principle while top tier people are using them as force multipliers.

jcgrillo 2y ago
> top tier people Who? How? This is not what I've seen where I work. There's a bunch of hubbub and generalized excitement, and lots of talk about what could be done, or what might be done, but not very much actual doing. I must just be a clueless "mid".
- vonneumannstan 2y ago
  Yeah unironically. Guido van Rossum - "I use it every day. My biggest adjustment with using Copilot was that instead of writing code, my posture shifted to reviewing code." https://www.youtube.com/watch?v=-DVyjdw4t9I https://www.youtube.com/watch?v=-DVyjdw4t9I Here's Jeff Dean saying 25% of the characters in new PRs at Google are AI Generated. https://www.dwarkeshpatel.com/p/jeff-dean-and-noam-shazeer https://www.dwarkeshpatel.com/p/jeff-dean-and-noam-shazeer Andrej Karpathy - "I basically can't imagine going back to "unassisted" coding at this point" https://www.reddit.com/r/singularity/comments/1ezssll/andrej_karpathy_programming_is_changing_so_fast/#lightbox https://www.reddit.com/r/singularity/comments/1ezssll/andrej... Andrew Ng - "I run multiple models on my laptop — Mistral, Llama, Zefa. And I use ChatGPT quite often. " https://www.ft.com/content/2dc07f9e-d2a9-4d98-b746-b051f9352be3 https://www.ft.com/content/2dc07f9e-d2a9-4d98-b746-b051f9352... Simon Willison https://simonwillison.net/2023/Mar/27/ai-enhanced-development/ https://simonwillison.net/2023/Mar/27/ai-enhanced-developmen... I mean I can keep going. I doubt you would compare yourself to these people.
  - habinero 2y ago
    These kinds of responses are my favorite dark pattern rhetorical device, because you can assert literally _anything_ in this format and almost nobody will refute you, because the cost of refuting bullshit is 100x the cost of producing it. Anyways, here goes. 1. Guido uses Copilot like I do - as a StackOverflow replacement to write the dumb boilerplate code. A much less flattering quote is "It doesn't save me much thinking, but [it helps] because I'm a poor typist". Also it's literally a minute or two of a three hour podcast. 2. A lot of code is autogenerated lol. Again, it's all the boring boilerplate stuff. 3. The cofounder of OpenAI is a biased source lol 4. He's an AI researcher, of course he runs that stuff. 5. Again, similar to Guido. He's using it for the boilerplate. Nothing wrong with enjoying using it as a toy, as he is here. But he's not doing serious work with it. There's no virtue in hyping this stuff like a HODL bitcoin cultist.

bigstrat2003 2y ago

I disagree. Tools which need to be babysat, the way LLMs do, slow you down rather than speed you up. It's like having to mentor a junior team member, except a human will eventually learn and you can just let him work. LLMs are incapable of learning, so you can't ever leave the phase where they are a drain.

paulcole 2y ago

> I’m a little shocked at how much negativity there is around LLMs among developers There's a Quentin Tarantino quote where he says there are 2 kinds of film critics. There are those who love movies and there are those that love the movies they love. A lot of developers really seem to love the technology they love. These people are where most of the negativity is coming from. And my guess is that the people who are encouraged by LLMs and not negative (mostly) aren't taking time out of their days to write long blog posts or argue about it online.

habinero 2y ago
I use Copilot for autocompleting the boring boilerplate. I like it. I also think LLMs are mostly useless. It's not the technology, it's the stupid overhype. It really feels like all the HODL bitcoin cultists have finally gotten over their lost apes and found a new technogod. So many people in these threads are convinced it's about to gain sentience. That's not going to happen. You get the people outing themselves by saying "it does my job better than me!" If you say something honest and direct like "their output is mediocre and unreliable" or "the RNG text generator is not capable of thinking, you're just Clever Hans-ing yourself" or "if it does your job better than you, that says more about you than about it", you get people clutching their pearls like a Stanford parent whose kid got a D. arXiv has turned into a paper mill of AI startups uploading marketing hype cosplaying as "research".

synicalx 2y ago

I don't really get the hate; work is boring, if some tool can make it happen faster and with less effort I'm all for it. I don't hear the line cooks at McDonalds complaining that they have to use a semi-automated grill that beeps at them instead of an open fire.

michaelcampbell 2y ago

As a developer I'm seeing less hate than apathy from my colleagues, but rather the hatred I do see is from people trying to push LLM's ON developers. So it's from middle management levels riding the hype train, and possibly trying to save money and getting bonuses for it at the expense of other people. Just like when offshoring was in its same point on Gartner's hype curve. "Everyone has a model, but no one has a business".