13 ms·

Everything around LLMs is still magical and wishful thinking

• atemerev 1y ago
  "It's crypto all over again" Crypto is a lifeline for me, as I cannot open a bank account in the country I live in, for reasons I can neither control nor fix. So I am happy if crypto is useless for you. For me and for millions like me, it is a matter of life and death. As for LLMs — once again, magic for some, reliable deterministic instrument for others (and also magic). Just classified and sorted a few hundreds of invoices. Yes, magic.
  • harel 1y ago
    Can you elaborate on your situation? Which country are you in? How is crypto used there?
    • atemerev 1y ago
      I am a Russian immigrant in Switzerland. As of right now, all Swiss banks block all Russian bank accounts until their owners can provide a valid physical residence permit card, due to sweeping sanctions (meanwhile, Russian-owned companies continue to freely trade crude oil from here, as they use Swiss nominal directors — the hypocrisy is through the roof). My residence permit is on renewal now and the case is being dragged for 7 months already — so, no bank account.
      • harel 1y ago
        Thank you for indulging my curiosity. I really hope your (and our world's) situation improves soon.
      • assuagering 1y ago
        Are you paid for work and able to pay for rent and groceries with cryptocurrency?
        • atemerev 1y ago
          Yes, I am paid for work in crypto (I am a consultant), and I can exchange it for cash and pay for rent and groceries in cash, just like people did in the previous century. In some other Swiss cantons like Zug, you can pay some of the bills and even some taxes with crypto directly, but not here yet.
  • mumbisChungo 1y ago
    Said this in another thread and I'll repeat it here: It's the same problem that crypto experiences. Almost everyone is propagating lies about the technology, even if a majority of those doing so don't understand enough to realize they're lies (naivety vs malice). I'd argue there's more intentional lying in crypto and less value to be gained, but in both cases people who might derive real benefit from the hard truth of the matter are turning away before they enter the door due to dishonesty/misrepresentation- and in both cases there are examples of people deriving real value today.
    • o11c 1y ago
      > I'd argue there's more intentional lying in crypto I disagree. Crypto sounds more like intentional lying because it's primarily hyped in contexts typical for scams/gambling. Yes, there are businesses involved (anybody can start one), but they're mostly new businesses or a tiny tack-on to an existing business. AI is largely being hyped within the existing major corporate structures, therefore its lies just get tagged as as "business as usual". That doesn't make them any less of a lie though.
      • mumbisChungo 1y ago
        I think crypto companies and AI companies probably intentionally mislead approximately the same amount as one another, but in crypto the average participant is often bagholding a very short term investment and has a direct and tangible incentive to mislead as many people about it as quickly as possible- whereas in AI people mostly just get lost in the sauce with anthropomorphism. Anecdotally, I see a lot more bold-facing lies by crypto traders or NFT "collectors" than by LLM enthusiasts.
  • tehjoker 1y ago
    This is basically the only use case for crypto, and one for which it was explicitly designed: censorship resistance. This is why people have so much trouble finding useful things for it to do in the legal economy, it was explicitly designed to facilitate transactions the government doesn't want or can't facilitate. In some cases, there are humanitarian applications, there are also a lot of illicit applications.
  • foobarchu 1y ago
    I don't think you actually disagree with the authors quip. You seem to want to use crypto as a currency, while OP was most likely referring to the grifting around crypto as an investment. If you're using it as a currency, then the people trying to pump and dump coins and use it for a money making vehicle are your adversaries. You are best served if it's stable instead of a rollercoaster of booms and busts.
    • atemerev 1y ago
      Stablecoins are a thing. But yes, I hate the current state of affairs, with "memecoins" and whatsnot. Particularly the government push from one particular country. We created crypto to be independent from governments, not to enable them.
  • troupo 1y ago
    It's a valid use case in the sea of nonsensical hype where "you are a moron if you don't believe in some true meaning of crypto". "You had to be there to believe it" https://x.com/0xbags/status/1940774543553146956 https://x.com/0xbags/status/1940774543553146956 AI craze is currently going through a similar period: any criticism is brushed away as being presented by morons who know nothing
• dcre 1y ago
  Similar argument to https://www.baldurbjarnason.com/2025/trusting-your-own-judgement-on-ai/ https://www.baldurbjarnason.com/2025/trusting-your-own-judge..., but I like this one better because at least it doesn’t try to pull the rhetorical trick of slipping from “we can’t know whether LLMs are helping because we haven’t studied the question systematically” to “actually we do know, and they’re shit”.
  • troupo 1y ago
    Wow. Quite a conclusion from an article that actually doesn't reach for that conclusion
    • dcre 1y ago
      It’s a bad and incoherent essay. It simultaneously wants to say “we can’t know”, “we do know and it doesn’t work”, and “we know it makes engineering faster and that’s bad because most engineering is bad.” This isn’t nuance, it’s confusion. Sorry!
      • troupo 1y ago
        It simultaneously wants none of those things
• careful_ai 1y ago
  [flagged]
  • Arainach 1y ago
    Amen. At my job right now there is an imminent threat from a team empowered to say "what if we asked an AI to just build X instead of having a team build and maintain it?" X is something where it's straightforward when N is below 50 but deeply complex when N is in the thousands, which for our team it is, and there is a huge risk that this team will get a demo with N=15 that attracts leadership attention and trying to explain why the AI-generated solution does not scale is a career-limiting move framing me as a naysayer, but this AI team would deliver the demo and go away and the inevitable failure of their solution at scale would ALSO be my team's problem, so..... I hate the future.
    • rdgthree 1y ago
      FWIW the comment you are responding to was authored by AI.
      • careful_ai 1y ago
        [dead]
        • jules-jules 1y ago
          This is so chatGpt it hurts. Can we petition hn to ban ai generated comments? I see more Reddit communities actively putting a ban on ai, hn should follow if it can be done with the available resources.
          • careful_ai 1y ago
            [dead]
          • ThrowawayR2 1y ago
            Generated comments of any kind are already banned according to dang (https://news.ycombinator.com/item?id=33945628 https://news.ycombinator.com/item?id=33945628) even though it hasn't been added to the HN guidelines page. So you could legitimately contact the moderators to ask them to investigate.
• notphilipmoran 1y ago
  I think that the disparity comes between people that are too in the weeds believing that their use cases apply to everyone. The reality is this world is made up of people with a wide array of different needs and AI is yet to proliferate into all usage applications. Sure some of this comes from a lack of education. But similar to crypto these movements only have value if the value is widely perceived. We have to work to continue to educate, continue to question, continue to understand different perspectives. All in favor of advancing the movement and coming out with better tech. I am a supporter of both but I agree with the reference in the article to both becoming echo chambers at times. This is a setback we need to avoid.
  • tempodox 1y ago
    Was this text generated by an LLM?
  • antonvs 1y ago
    > But similar to crypto these movements only have value if the value is widely perceived. The difference in the AI case is that companies that are actually able to use it to boost productivity significantly will start to outcompete those who don't. That's why, unlike crypto/blockchain, so many mainstream companies are pouring money into AI. It's not FOMO so much as fear of extinction.
• blueboo 1y ago
  Ok. Claude Code produces most code at Anthropic. Theres an enterprise code base, with acute real needs. There are real, experienced SWEs. How much babysitting and reviewing is undetermined; but the Ants seem to tremendously prefer the workflow. Even crypto people didn’t dogfood their crypto like that, on their own critical path.
  • taurath 1y ago
    Truly, how could they have the valuation they have and do anything else?
  • rightbyte 1y ago
    > but the Ants Is that the official cutsie name people working there are called? Those feels so 2020 ...
  • peter422 1y ago
    In my codebase in my proprietary project, it’s possible that LLMs write around half the code, but that’s just because a lot of the trivial parts of the project are quite verbose. The really difficult and valuable parts of the codebase are very very far beyond what the current LLMs are capable of, and believe me, I’ve tried! Writing the majority of the code is very different from creating the majority of the value. And I really use and value LLMs, but they are not replacing me at the moment.
  • thisoneworks 1y ago
    Regardless of the usefulness of llms, if you don't work at anthropic, how gullible are you to believe that claim at face value?
  • troupo 1y ago
    > Ok. Claude Code produces most code at Anthropic.m Does it? Or does their marketing tell you that? Strange that "most code is written by Claude" and they still hire for actual humans for all the positions from backend to API to desktop to mobile clients. > How much babysitting and reviewing is undetermined; but the Ants seem to tremendously prefer the workflow. So. We know nothing about their codebase, actual flows, programming languages, depth and breadth of usage, how much babysitting is required...
• sherdil2022 1y ago
  I follow Emily Bender on LinkedIn. She cuts through the AI hype and is also the author of The AI Con book - https://thecon.ai/ https://thecon.ai/ Of course people will either love AI or hate AI - and some don’t care. I am cautious especially when people say ‘AI is here to stay’. It takes away agency.
  • farts_mckensy 1y ago
    AI is here to stay, and you do not have agency over that. You can choose not to use it, but that has zero impact on the broader adoption rate. Just like when the automobile was introduced and society as a whole evolved.
    • goatlover 1y ago
      There was agency in policies promoting automobile adoption over rail travel. People who act like market forces are inevitable trends in nature tend to forget the political part where companies convince politicians to make policies in their favor, while bombarding the public with advertising. Like fossil fuel companies for example. Continued climate change wasn't inevitable. It was a choice humans with lots of money and power made and are still making.
      • farts_mckensy 1y ago
        And that's what's going to happen now. No amount of complaining about it online is going to stop it. They literally just passed a bill that prevents regulating AI. Rich people and their puppets in congress have some degree of agency. We don't. Sorry.
        • sciencejerk 1y ago
          I heard the AI clause didn't make it into the final version of the bill that was passed (?)
        • goatlover 1y ago
          99-1 vote in the Senate removed that from the bill before it went to the House.
• jm20 1y ago
  The best way I’ve heard this described: AI (LLMs) is probably 90% of the way to human levels of reasoning. We can probably get to about 95% optimizing current technology. Whether or not we can get to 100% using LLMs is an open research problem and far from guaranteed. If we can’t, it’s unclear if it will ever really proliferate the way things hope. That 5% makes a big difference in most non-niche use cases…
  • krapp 1y ago
    >The best way I’ve heard this described: AI (LLMs) is probably 90% of the way to human levels of reasoning. We can probably get to about 95% optimizing current technology. We don't know enough about how LLMs work or about how human reasoning works for this to be at all meaningful. These numbers quantify nothing but wishes and hype.
  • ath3nd 1y ago
    > AI (LLMs) is probably 90% of the way to human levels of reasoning Considering LLMs have 0 level of reasoning, I can't decide if it's a bad take, or a stab at the average human's level of reasoning. In all seriousness, the actual numbers vary from 13% to 26%: https://fortune.com/2025/02/12/openai-deepresearch-humanity-last-exam/ https://fortune.com/2025/02/12/openai-deepresearch-humanity-... My take is that there are fundamental limitations to try to pigeon-hole reasoning to LLMs, which are essentially a very very advanced autocomplete, and that's why those % won't jump too much too soon.
    • farts_mckensy 1y ago
      Whenever people claim that LLMs are not capable of reasoning, I put them into a category of people who are themselves not capable of reasoning.
      • ath3nd 1y ago
        Whenever people claim that LLMs are capable of reasoning, I put them into a category of people who are themselves able to reason as much as an LLM.
• jjtheblunt 1y ago
  https://en.wikipedia.org/wiki/Clarke%27s_three_laws https://en.wikipedia.org/wiki/Clarke%27s_three_laws includes the 3rd law, which reads, and seems on topic, "Any sufficiently advanced technology is indistinguishable from magic."
  • readthenotes1 1y ago
    And of course it says first law that applies here. The people I have talked to at length about using AI tools claim that it has been a boon for productivity: a nurse, a doctor, three (old) software developers, a product manager, and a graduate student in Control Systems. It is entirely believable that it may not, on average, help the average developer. I'm reminded of the old joke that ends with "who are you going to believe, me or you're lying eyes?"
• tasty_freeze 1y ago
  One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't most of the time) overall my productivity is 10-15% better. That is nothing to sneeze at, but it isn't 10x.
  • louthy 1y ago
    > overall my productivity is 10-15% better. That is nothing to sneeze at, but it isn't 10x. It is something to sneeze at if you are 10-15% more expensive to employ due to the cost of the LLM tools. The total cost of production should always be considered, not just throughput.
    • bravesoul2 1y ago
      That's a good insight be because with perfect competition it means you need to share your old salary with an LLM!
    • votepaunchy 1y ago
      > if you are 10-15% more expensive to employ due to the cost of the LLM tools How is one spending anywhere close to 10% of total compensation on LLMs?
    • CharlesW 1y ago
      > It is something to sneeze at if you are 10-15% more expensive to employ due to the cost of the LLM tools. Claude Max is $200/month, or ~2% of the salary of an average software engineer.
      • m4rtink 1y ago
        Does anyone actually know what the real cost for the customers will be once the free AI money no longer floods those companies?
• labrador 1y ago
  I'm a retired programmer. I can't imagine trusting code generated by probablities for anything mission critical. If it were close and just needed minor tweaks I could understand that. But I don't have experience with it. My comment is mainly to say LLMs are amazing in areas that are not coding, like brainstorming, blue sky thinking, filling in research details, asking questions that make me reflect. I treat the LLM like a thinking partner. It does make mistakes, but those can be caught easily by checking other sources, or even having another LLM review the conclusions.
  • garciasn 1y ago
    Well; I can't speak to your specific experience (current or past) but I'm telling you that while I'm skeptical as hell about EVERYTHING, it's blowing my expectations away in every conceivable way. I built something in less than 24h that I'm sure would have taken us MONTHS to just get off the ground, let alone to the polished version it's at right now. The most impressive thing is that it can do all of the things that I absolutely can do, just faster. But the most impressive thing is that it can do all the things I cannot possibly do and would have had to hire up/contract out to accomplish--for far less money, time, and with faster iterations than if I had to communicate with another human being. It's not perfect and it's incredibly frustrating at times (hardcoding values into the code when I have explicitly told it not to; outright lying that it made a particular fix, when it actually changed something else entirely unrelated), but it is a game changer IMO.
    • gyomu 1y ago
      > I built something in less than 24h that I'm sure would have taken us MONTHS to just get off the ground, let alone to the polished version it's at right now Would love to see it!
    • 98eb1d0ff7fb96 1y ago
      See, your comment is a good example of what's going wrong. The OP specifically mentioned "mission critical things" - My interpretation of that would be things that are not allowed to break, because otherwise people might die, in the worst case - and you were talking about just SOMETHING that got "done" faster. No mention about anything critical. Of course, I was playing around with claude code, too, and I was fascinated how fun it can be and yes, you can get stuff done. But I have absolutely no clue what the code is doing and if there are some nasty mistakes. So it kinda worked, but I would not use that for anything "mission critical" (whatever this means).
• alittlebee 1y ago
  [dead]
• yahoozoo 1y ago
  The thing is, the questions such as “are they an expert in the domain” … “are they good at coding to being with” … and so on only really apply to the folks claiming positive results from LLMs. On the flip side, someone not getting much value - or dare I say, a skeptic - pushes back because they _can see_ what the LLM gave them is wrong. I’m not providing any revelatory comment here, but the simple truth is: people who are shit to begin with think this is all amazing/magic/the future.
• deleted 1y ago
  [deleted]
• hotpotat 1y ago
  I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better and claude 4 is out, I barely ever write code, and I don’t mind. I’ve leaned into the Architect/Manager role and direct the agent with my specialized knowledge if I need to. I started a job at a demanding startup and it’s been several months and I have still not written a single line of code by hand. I audit everything myself before making PRs and test rigorously, but Cursor + Sonnet is just insane with their codebase. I’m convinced I’m their most productive employee and that’s not by measuring lines of code, which don’t matter; people who are experts in the codebase ask me for help with niche bugs I can narrow in on in 5-30 minutes as someone whose fresh to their domain. I had to lay off taking work away from the front end dev (which I’ve avoided my whole career) because I was stepping on his toes, fixing little problems as I saw them thanks to Claude. It’s not vibe coding - there’s a process of research and planning and perusing in careful steps, and I set the agent up for success. Domain knowledge is necessary. But I’m just so floored how anyone could not be extracting the same utility from it. It feels like there’s two articles like this every week now.
  • the__alchemist 1y ago
    Web dev CRUD in node?
    • hotpotat 1y ago
      Multi platform web+native consumer application with lots of moving parts and integration. I think to call it a CRUD app would be oversimplifying it.
  • gyomu 1y ago
    > I’ve shipped non trivial greenfield products Links please
    • 1y ago
• DiscourseFan 1y ago
  ChatGPT can write research papers in about 20 minutes—its the “Deep Research” tool. These are not original papers, but it can perform complex tasks that require multiple steps that would normally take a person hours. No its not a magic superintelligence, but it will transform a lot of white collar labor.
• martinald 1y ago
  I personally don't really get this. _So much_ work in the 'services' industries globally comes down to really a human transposing data from one Excel sheet to another (or from a CRM/emails to Excel), manually. Every (or nearly every) enterprise scale company will have hundreds if not thousands of FTEs doing this kind of work day in day out - often with a lot of it outsourced. I would guess that for every 1 software engineer there are 100 people doing this kind of 'manual data pipelining'. So really for giant value to be created out of LLMs you do not need them to be incredible at OCaml. They just need to ~outperform humans on Excel. Where I do think MCP really helps is that you can connect all these systems together easily, and a lot of the errors in this kind of work came from trying to pass the entire 'task' in context. If you can take an email via MCP, extract some data out and put it into a CRM (again via MCP) a row at a time the hallucination rate is very low IME. I would say at least a junior overworked human level. Perhaps this was the point of the article, but non-determinism is not an issue for these kind of use cases, given all the humans involved are not deterministic either. We can build systems and processes to help enforce quality on non deterministic (eg: human) systems. Finally, I've followed crypto closely and also LLMs closely. They do not seem to be similar in terms of utility and adoption. The closest thing I can recall is smartphone adoption. A lot of my non technical friends didn't think/want a smartphone when the iPhone first came out. Within a few years, all of them have them. Similar with LLMs. Virtually all of my non technical friends use it now for incredibly varied use cases.
  • deepsquirrelnet 1y ago
    Making a comparison to crypto is lazy criticism. It’s not even worth validating. It’s people who want to take the negative vibe from crypto and repurpose it. The two technologies have nothing to do with each other, and therefore there’s clearly no reason to make comparative technical assessments between them. That said, the social response is a trend of tech worship that I suspect many engineers who have been around the block are weary of. It’s easy to find unrealistic claims, the worst coming from the CEOs of AI companies. At the same time, a LOT of people are practically computer illiterate. I can only imagine how exciting it must seem to people who have very limited exposure to even basic automation. And the whole “talking computer” we’ve all become accustomed to seeing in science fiction is pretty much becoming reality. There’s a world of takes in there. It’s wild. I worked in ML and NLP several years before AI. What’s most striking to me is that this is way more mainstream than anything that has ever happened in the field. And with that comes a lot of inexperience in designing with statistical inference. It’s going to be the Wild West for a while — in opinions, in successful implementation, in learning how to form realistic project ideas. Look at it this way: now your friend with a novel app idea can be told to do it themselves. That’s at least a win for everyone.
• tiahura 1y ago
  Everything? As a lawyer, I’m producing 2x - with fewer errors. Admittedly, law is a field that mostly involves shuffling words around so it may be the best case scenario, but much of the skepticism comes off as cope.
  • KerrAvon 1y ago
    uh https://www.technologyreview.com/2025/05/20/1116823/how-ai-is-introducing-errors-into-courtrooms/ https://www.technologyreview.com/2025/05/20/1116823/how-ai-i... https://hai.stanford.edu/news/hallucinating-law-legal-mistakes-large-language-models-are-pervasive https://hai.stanford.edu/news/hallucinating-law-legal-mistak... https://www.reuters.com/technology/artificial-intelligence/ai-hallucinations-court-papers-spell-trouble-lawyers-2025-02-18/ https://www.reuters.com/technology/artificial-intelligence/a... There are more of these stories every week. Are you using AI in a way that doesn’t allow you to be entrapped by this sort of thing?
    • tiahura 1y ago
      ChatGPT links to the actual text in a case now. Also, take the output from Claude, and put it into Gemini and tell it to verify holdings. Furthermore, spot checking 10 cases doesn’t take long.
      • troupo 1y ago
        > ChatGPT links to the actual text in a case now. Or to text in a law is hallucinated: https://www.timesofisrael.com/judge-slams-police-for-using-ai-to-write-legal-argument-citing-nonexistent-laws/ https://www.timesofisrael.com/judge-slams-police-for-using-a...
  • logsr 1y ago
    computer programming and law are very similar. computer code is called code because it is the law that dictates the behavior of the computer. law is a bit different because it is a program that runs on people, who aren't as deterministic as machines, but in theory law and the interpretation of law are also supposed to be completely logical, and you can translate back and forth directly from the logic of law to a logical expression in a computer program. i specialize in programming, and LLMs are very good right now, if you set them up with the right tooling, feedback based learning methods, and efficient ways of capturing human input (review/approve/suggest/correct/etc). with programming you have compilers and other static analysis tools that you can use to verify output. for law you need similar static analysis tooling, to verify things like citations, procedural scheduling, electronic filing, etc, but if you loop that tooling in with an llm, the llm will be able to correct errors automatically, and you will get to an agent that can take a statement of fact, find a cause of action, and file a pro se lawsuit for someone. courts are going to be flooded with lawsuits, on a scale of 10-100X current case loads. criminal defendants will be able to use a smart phone app to represent themselves, with an AI handling all of the filings and motions, monitoring the trial in real time, giving advice to the defendant on when to make motions and what to say, maximizing delay and cost for the state with maximum efficiency. with 98% of convictions coming from guilty pleas (https://www.npr.org/2023/02/22/1158356619/plea-bargains-criminal-cases-justice https://www.npr.org/2023/02/22/1158356619/plea-bargains-crim...) which are largely driven by not being able to afford the cost of legal services the number of criminal defendants electing to go to full jury trial could easily explode 10-20X or more very quickly. fun times!
• larve 1y ago
  Software methodologies and workflows are not engineering either, yet we spend a fair amount of time iterating and refining those. You can very much become better at prompt engineering. There is a huge differential between individuals, for example. The code coming out of LLMs is just as deterministic as code coming out of humans, and despite humans being feckle beings, we still talk of software engineering. As for LLMs, they are and will forever be "unknowable". The human mind just can't comprehend what a billion parameters trained on trillions of tokens under different regimes for months corresponds to. While science has to do microscopic steps towards understanding the brain, we still have methods to teach, learn, be creative, be rigorous, communicate that do work despite it being this "magical" organ. With LLMs, you can be pretty rigorous. Benchmarks, evals, and just the vibes of day to day usage if you are a programmer, are not "wishful thinking", they are reasonably effective methods and the best we have.
• sureglymop 1y ago
  Loosely related, but I find the use of AGI (and sometimes even AI) as terms annoying lately. Especially in scientific papers, where I would imagine everything to be well defined. If at least in how it is used in that paper. So, why can't we just come up with some definition for what AGI is? We could then, say, logically prove that some AI fits that definition. Even if this doesn't seem practically useful, it's theoretically much more useful than just using that term with no meaning. Instead it kind of feels like it's an escape hatch. On wikipedia we have "a type of ai that would match or surpass human capabilities across virtually all cognitive tasks". How could we measure that? What good is this if we can't prove that a system has this property? Bit of a rant but I hope it's somewhat legible still.
  • AlienRobot 1y ago
    We have a definition. "AI is whatever hasn't been done yet."[1] 1. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
    • CharlesW 1y ago
      "AI" and "AGI" are very different things.
  • selfhoster11 1y ago
    You don't need consensus on the meaning across the board. I maintain my own, more permissive milestone for what constitutes "AGI", but I have no expectations that others will share it. Much like "crypto" to me is still cryptography, not cryptocurrency - sometimes the mainstream will just have a different opinion.
    • sureglymop 1y ago
      My point is, it's not about the mainstream or marketing. In science, some rigor is expected. There doesn't need to be consensus if a definition is established within some context. It's perfectly fine to redefine something as needed for the research but only if that definition is declared.
• AbrahamParangi 1y ago
  This reads like the author is mad about imprecision in the discourse which is real but to be quite frank more rampant amongst detractors than promoters, who often have to deal with the flaws and limitations on a day to day basis. The conclusion that everything around LLMs is magical thinking seems to be fairly hubristic to me given that in the last 5 years a set of previously borderline intractable problems have become completely or near completely solved, translation, transcription, and code generation (up to some scale), for instance.
  • troupo 1y ago
    > but to be quite frank more rampant amongst detractors than promoters, who often have to deal with the flaws and limitations on a day to day basis. "detractors" usually point to actual flaws. "promoters" usually uncritically hail LLMs as miracles capable of solving any problem in one go, without giving any specific details.
  • DavidPiper 1y ago
    Translation, transcription, and code generation (up to some scale) were borderline intractable problems? Google Translate, Whisper and Code Generators (up to some scale) have existed for quite some time without using LLMs.
    • famouswaffles 1y ago
      Google Translate just spits out nonsense for distant language pairs (English<->Korean etc) and doesn't compare to Sota LLMs, Whisper is a Transformer (Architecture used for LLMs) and Code Generators have nothing on LLMs.
• arendtio 1y ago
  I think it is more like googling: when the search engine appeared, everybody had to learn how to write a good query, even though the expectation was that everybody could use them. With LLMs, it's quite similar: you have to learn how to use them. Yes, they are non-deterministic, but if you know how to use them, you can increase your chances of getting a good result dramatically. Often, this not only means articulating a task, but also looking at the bigger picture and asking yourself what tasks you should assign in the first place. For example, I can ask the LLM to write software directly, or I can ask it to write user stories or prototypes and then take a multi-step approach to develop the software. This can make a huge difference in reliability. And to be clear, I don't mean that every bad result is caused by not correctly handling the LLM (some models are simply poor at specific tasks), but rather that it is a significant factor to consider when evaluating results.
  • leptons 1y ago
    >Yes, they are non-deterministic The LLM is more like a Ouija board than a reliable tool. >I can ask it to write user stories or prototypes By the time I write enough to explain thoroughly to an LLM what to write in "user stories" or "prototypes", I could have just written it myself, without the middleman(bot), and without the LLM hallucinating. If half the time I spend with an LLM is telling it what to do, and then another half is correcting what it did, then I'm not really saving any time at all by using it.
    • selfhoster11 1y ago
      Writing the prompt takes time, but in many cases it's like rubber-duck debugging - it helps you to figure out the actual requirements, flow, or even incorrect/missing assumptions. Is it worth the time? Subjective.
    • arendtio 1y ago
      For me, it is a different experience. Yes, it takes time to provide the LLM with the necessary context, but overall it can save me 3-4 times the time I would have invested. Maybe I am just slower at software development myself ;-) However, the work you do is indeed more that of a product owner than that of a developer. To avoid hallucinations, providing extensive documentation and allowing the LLM to perform test-driven development can be a game-changer. When you do that, generation time is highly correlated (negatively) with code quality. So when the AI solves the tasks quickly and easily, you have a good chance of it generating good code. As soon as the AI has to try and try again to build something working, you should be very skeptical of the result. Over the past months and years, I have used this method, and because it is somewhat reproducible, you can see the progress that LLMs are making. Sessions where the model does stupid things are becoming fewer, and sessions where the model finds a good solution become more frequent.
• alganet 1y ago
  LLM tech probably will find some legitimate use, but by then, everything will be filled with people misusing it. Millions of beginner developers running with scissors in their hands, millions of investment in the garbage. I don't think this can be reversed anymore, companies are all-in and pot commited.
• Sherveen 1y ago
  This is completely incoherent. 3 reasons: 1. he talks about what he's shipped, and yet compares it to crypto – already, you're in a contradiction as to your relative comparison – you straight up shouldn't blog if you can't conceive that these two are opposing thoughts 2. this whole refrain from people of like, "SHOW ME your enterprise codebase that includes lots of LLM code" – HELLO, people who work at private companies CANNOT just reveal their codebase to you for internet points 3. anyone who has actually used these tools has now integrated them into their daily life on the order of millions of people and billions of dollars – unless you think all CEOs are in a grand conspiracy, lying about their teams adopting AI
  • infp_arborist 1y ago
    Re. 2 - I find it fascinating that people using coding agents have seemingly no problem to reveal their full codebases to AI companies.
    • Sherveen 1y ago
      Can you clearly articulate me why they should have a problem? If you don't want training on your codebase, many AI companies offer this option. What's the issue?
  • Ferrus91 1y ago
    Some even integrating in such a way as to professionally humiliate himself or herself: https://www.hcrlaw.com/news-and-insights/in-house-lawyers-beware-barrister-rebuked-for-relying-on-hallucinating-ai/ https://www.hcrlaw.com/news-and-insights/in-house-lawyers-be...
• deleted 1y ago
  [deleted]
• ibaikov 1y ago
  Crypto and NFT situation happened because of our society, media and vc/startup landscape who hype things up a lot for their own reasons. We treat massive technologies as new brands of bottled water. Or, actually, as a new hype toy as fidget spinners or pop it toys. This tech is massively more complex and you have to invest time to learn about its abilities, limitations and potential developments. Almost nobody actually does this, it's easier to follow hype train and put money into something that grows up and looks cool without obvious cons. Crypto is cool for some stuff. On the other hand, where's your Stepn (and move to earn in general), decentraland cities, Apes that will make a multimedia universe? Where's "you'll be paying using crypto for everything"? Same for LLMs and AI: it is awesome for some things and absolutely sucks for other things. Curiously tho, it feels like UX was solved by making chats, but it actually still sucks enormously, as with crypto. It is mostly sufficient for doing basic stuff. It is difficult to predict where we'll land on the curve of difficult (or expensive) vs abilities. I'd bet AI will get way more capable, but even now you can't really deny its usefulness.
  • CharlesW 1y ago
    It makes no sense to compare the current AI hype to the tulip mania of crypto/NFTs. A much better parallel is to cloud computing hype in 2009.
    • ibaikov 1y ago
      I think it does. Startups, VCs and media still provoke hype trains. People cherry pick and extrapolate everything in AI as well.
      • selfhoster11 1y ago
        Except the benefits of LLM adoption can be measured empirically, with your own eyes. And don't require network effects to fully realise benefits, unlike cryptocurrency and NFTs. Ignoring the hype and doing your own thing with it is a good choice.
        • 1y ago
• deleted 1y ago
  [deleted]
• afiodorov 1y ago
  We've been visited by alien intelligence that is simultaneously fascinating and underwhelming. The real issue isn't the technology itself, but our complete inability to predict its competence. Our intuition for what should be hard or easy simply shatters. It can display superhuman breadth of knowledge, yet fail with a confident absurdity that, in a person, we'd label as malicious or delusional. The discourse is stuck because we're trying to map a familiar psychology onto a system that has none. We haven't just built a new tool; we've built a new kind of intellectual blindness for ourselves.
• localghost3000 1y ago
  I've developed the following methodology with LLM's and "agentic" (what a dumb fucking word...) workflows: I will use an LLM/agent if - I need to get a bunch of coding done and I keep getting booked into meetings. I'll give it a task on my todo list and see how it did when I get done with said meeting(s). Maybe 40% of the time it will have done something I'll keep or just need to do a few tweaks to. YMMV though. - I need to write up a bunch of dumb boilerplatey code. I've got my rules tuned so that it generally gets this kind of thing right. - I need a stupid one off script or a little application to help me with a specific problem and I don't care about code quality or maintainability. - Stack overflow replacement. - I need to do something annoying but well understood. An XML serializer in Java for example. - Unit tests. I'm questioning if this ones a good idea though outside of maybe doing some of the setup work though. I find I generally come to understand my code better through the exercise of writing up tests. Sometimes you're in a hurry though so...<shrug> With any of the above, if it doesn't get me close to what I want within 2 or 3 tries, I just back off and do the work. I also avoid building things I don't fully understand. I'm not going to waste 3 hours to save 1 hour of coding. I will not use an LLM if I need to do anything involving business logic and/or need to solve a novel problem. I also don't bother if I am working with novel tech. You'll get way more usable answers asking about Python then you will asking about Elm. TL;DR - use your brain. Understand how this tech works, its limitations, AND its strengths.
• hamilyon2 1y ago
  I am impressed by speed of the sound goalpost movement. Few days ago Google released very competent summary generator, interpreter between 10-s of languages, gpt-3 class general purpose assistant. Working locally on modest hardware. On 5 years old laptop, no discrete GPU. It alone potentially saves so much toil, so much stupid work. We also finally “solved computer vision”. Read from PDF, read diagrams and tables. Local vision models are much less impressive and need some care to use. Give it 2 years. I don't know if we can overhype it when it archives holy grail level on some important tasks.
  • assuagering 1y ago
    I have had SOTA models stray from factual content in documents I provided them with within 2-3 prompts. They haven't solved anything. They are just fast and look good doing what we ask them to do. But they corrupt data with a passion and to that hype just responds: "just give us 10x as much money and compute".
• ls-a 1y ago
  AI helped me find a bug that was going undetected and also helped me fix it. I was debugging a completely different bug, and decided to get a bit creative with what context i should share. AI pointed out the other bug. What I'm trying to say is people should stop talking about AI making them faster, that is not a good goal, unless your manager sucks. People should talk about concrete cases of how AI helped them instead. Another example I can give is AI helped me understand a part of a complicated protocol without reading the spec. It explained just the part I needed at the moment. AI made me dread my work less. I hope founders start getting creative with AI tools instead of copying each other.
• 3cats-in-a-coat 1y ago
  These polarized opinions that say AI is everything or nothing only reveal the emotional state of the writer, rather than any deep insights.
• kgeist 1y ago
  We recently started using LLMs at our company, and the first job I had was to transcribe 20k customer calls and extract the following info: 1) what products we're usually compared to 2) what problems users have with our software 3) what use cases users mention most often What used to take weeks of research took just a couple of hours. It helped us form a new strategy and brought real business value. I see LLMs as just a natural language processing engine, and they're great at that. Some people overhype it, sure, but that doesn't change the fact that it's been genuinely useful for our cases. Not sure what's up with all those "LLM bad" articles. If it doesn't work for you, just move on. Why should anyone have to prove anything to anyone? It's just a tool.
  • hx8 1y ago
    I think you are underestimating the negative impacts that overhype cause. It's distorting the market, causing over investment, preemptively slashing departments, and creating an expectation that will never be meet. These articles are important for cooling expectations. When people sell LLMs, they usually aren't talking about summarizing customer support calls, they are trying to sell the idea of firing customer support staff.
    • DebtDeflation 1y ago
      If anything, the LLM overhype is starting to die down........to make way for the AI Agent hype which is on trajectory to be 1000X worse. People are writing articles and making videos about how AI Agents will replace SaaS. What?
      • kgeist 1y ago
        The OP compares the current LLM hype to crypto, but I think it's more fair to compare it to the dotcom bubble. When a new, interesting technology appears, there's always a lot of hype around it - I think it's natural. People are still figuring out what works and what doesn't. Naturally, some overoptimistic people overhype it. The dotcom bubble burst; nonetheless, the internet is now an integral part of our lives. Despite the hype, crypto was always a very niche area, while currently even my grandmother uses ChatGPT - just like the internet.
• standardUser 1y ago
  > Like most skeptics and critics, I use these tools daily. And 50% of the time they work 50% of the time. I use LLMs nearly every day for my job as of about a year ago and they solve my issues about 90% of the time. I have a very hard time deciphering if these types of complaints about AI/LLMs should be taken seriously, or written off as irrational use patterns by some users. For example, I have never fed an LLM a codebase and expected it to work magic. I ask direct, specific questions at the edge of my understanding (not beyond it) and apply the solutions in a deliberate and testable manner. if you're taking a different approach and complaining about LLMs, I'm inclined to think you're doing it wrong. And missing out on the actual magic, which is small, useful and fairly consistent.
  • leptons 1y ago
    Your comment is no better than the comment in the article that the author is calling out. "90%" also seems a bit suspect.
    • richardw 1y ago
      I just went through the last 10 chat titles and all of them were spot on for me. Maybe the person you’re responding to has a different experience than you do and calling their perspective “suspect” is somewhat uncharitable. (There are times I do other kinds of work and it fails terribly. My main point stands.)
      • an0malous 1y ago
        Can you share the questions you asked?
        • richardw 1y ago
          It ranged from whether an epic v10 sport surf ski was a good fit for a newbie, to entra ID questions, to local data residency compliance laws, new jira alternatives, why schools ask for closed shoes, text to speech tool search. Many of these I use eg o4-mini-high for because I want it to ground itself: find material and compile something for me, but get me an answer fairly quickly. Ones that don’t work but weren’t in the last 10: voice. It sounds amazing but is dumb as rocks. Feels like most of the GPU compute is for the media, not smarts. A question about floating solar heaters for pools. It fed me scam material. A question about negotiating software pricing. Just useless, parroted my points back at me. I scale models up and down based on need. Very simple: gpt-40. Smarts: o4-mini-high. Research: deep research. I love Claude but at some point it kept running out of capacity so I’d move elsewhere. Although nothing beats it for artefacts. MS Copilot if I want a quick answer to something MS oriented. It’s terrible but it’s easy to access. Coding is generally Windsurf but honestly that’s been rare for the last month. Been too busy doing other things.
• awkwabear 1y ago
  This seems like a really bad take. I do PhD research for superconducting materials and right I've been adapting and scaling an existing segmentation model from a research paper for image processing to run multithreaded and took the training runtime per image from 55min to 2min. Yeah it was low hanging fruit but honestly its the type of thing that is just tedious and easy to make mistakes and spend forever debugging. Like sure I could have done it myself but it would have taken me days to figure out and I would have had to test and read a ton of docs. Claude got it working in like half an hour and generated every data plot I could need. If I wanted to test out different strategies and optimizations, I could iterate through various strategies rapidly. I don't really like to rely on AI a bunch but it indisputably is incredibly good at certain things. If I am just trying to get something done and don't need to worry about vulnerabilities as it is just data collection code that runs once, it saves a tremendous amount of time. I don't think it will outright replace developers but there is some room for it to expand the effectiveness of individual devs so long as they are actually providing oversight and not just letting it do stuff unchecked. I think the larger issue is more how economically viable it is for businesses to spend a ton on electricity and compute for me to be able to use it like this for 20 bucks a month. There will be an inevitable enshittification of services once a lot of the spaces investors are dumping money are figured out to be dead ends and people start calling for returns on their investment. Right now the cash is flowing cause business people don't fully understand what its good at or not but that's not gonna last forever.
  • phyzome 1y ago
    I don't think you got the author's point. They didn't say "AI is bad". Take another look.
    • awkwabear 1y ago
      Ah I'm sorry you're totally right, I was irresponsible and just skimmed the article last time. I retract my last statement about it being a bad take
      • troupo 1y ago
• weare138 1y ago
  Just wait until we get AI running on the blockchain in the metaverse. The future is here.
• deleted 1y ago
  [deleted]
• scrubs 1y ago
  Op? Take a standing ovation and a victory lap. Well written.
• geetee 1y ago
  I'm working with product managers that are almost certainly using LLMs to generate product requirement docs, complete with code samples, data type definitions, and diagrams. Everything looks good to the untrained eye, but it's complete and utter bullshit. LLM abuse is going to be the end of so many tech companies.
  • bwfan123 1y ago
    > Everything looks good to the untrained eye thats the trick with bullshit in general. > LLM abuse is going to be the end of so many tech companies and is also going to provide a lot of opportunities for experienced engineers to cleanup the mess.
• djoldman 1y ago
  Many well-trusted and reasonable tech folks who are known for sober takes on subjects have reported substantial improvements in their programming work by using various forms of generative AI. What does substantial mean? Somewhere between 5% and 100%. Something NOT insignificant. At a minimum, it is safe to say that GenAI is or could be a significantly beneficial tool for a significant number of people. It's not required that folks disclose how many CPUs, lines of code, numbers of bytes processed, or other details for the above to be a reasonable take.
  • troupo 1y ago
    "People claim to have productivity increases anywhere between a random number I invented to solve other number I invented. We should believe these claims uncritically"
• orbital-decay 1y ago
  The point about non-determinism is moot if you understand how it works. An accurate LLM always gives the same result where the same result is needed, no matter how many times you ask it. Try asking any LLM what is 2x2 on a temperature it's designed for, what are the chances to get 5 in a reply? In reality, modern LLMs trained with RL have terrible variance and mainly learn 1:1 mapping of ideas to ideas, which is a big issue for creative writing and parallel inference/majority voting techniques, so there's even less meaningful "non-determinism" available than you might think. It's usually either able or not able to give the correct answer, rerolling it doesn't work well. I think even a human has more non-determinism than a modern LLM (it's impossible to measure though).
  • troupo 1y ago
    Are those LLMs in the room with us now? ;) Actually, I did try asking ollama running locally. That should've reduced the amount of non-determinism and whatever layers providers add, and the uncertainty of computer availability. I asked it for a list of Javascript keywords ordered alphabetically. Within 5 minutes it produced a slightly different list Asking a model for 2x2 is moot because 2x2=5 is statically highly unlikely. Anything more complex though?
    • orbital-decay 1y ago
      >I asked it for a list of Javascript keywords ordered alphabetically. Within 5 minutes it produced a slightly different list That's not my point, the keyword here is "meaningful". How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones) If you spend some time with SotA LLMs, you'll see that on rerolling they express pretty much the same ideas in different ways, most of the time.
      • troupo 1y ago
        > That's not my point. How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones) > Yes. That's missing the forest for the trees, though. So, what is your point and forest? That there exists some non-deterministic accurate LLM with a 100% correct training set which will always produce the same answer if you ask it 2x2 (where asking is somehow different from prompting)? - Does it exist? - Where? (considering that the world at large uses chatgpt/claude/gemini) - what's its usefulness beyond 2x2?
• khazhoux 1y ago
  I'm honestly started to get pissed seeing these articles on HN every day. Aren't we technologists? I see these articles and it's just people who can't accept the future, or feel threatened by it, or are pissed they are not contributing to it. I'm an expert in my field with decades of experience, and a damn good programmer, and Cursor is like zipping down the street on a powered bike. There is no wishful thinking here! Can I use it for everything? No, and that applies to every tool. But ffs I have to read here every day on HN some self-important blogger telling me I'm imagining things.
  • troupo 1y ago
    > I'm honestly started to get pissed seeing these articles on HN every day. Aren't we technologists? Funnily enough, that's exactly what the article asks. > I'm an expert in my field with decades of experience, and a damn good programmer, and Cursor is like zipping down the street on a powered bike. Oh look, an unverified claim. Go through the first list in the article, and ask "what we, as technologists, know about this claim". You can also actually read the article and see that I actually use these tools every day.
    • khazhoux 1y ago
      Yeah, I read your article. It's the latest in the stream of "everyone is full of crap" about LLMs. You say you've used it to build multiple projects, but then title it "everything is wishful thinking". That makes no sense. You're either click-baiting or have an inconsistent stance. > This is crypto all over again No. The fundamental vision for BTC is that it grows to be a mainstream currency for a huge variety of use cases. If BTC continues to be used just for pirate sites, black market, etc, then it will be considered to have failed. With LLMs the success is immediate. We don't need LLMs to solve every software problem, for them to be massive accelerator of daily coding.
      • troupo 1y ago
        > That makes no sense. You're either click-baiting or have an inconsistent stance. Or you haven't actually understood a single thing from the article. Like the fact you say "aren't we technologists" immediately followed by a completely unverifiable claim that everyone is supposed to take at face value, uncritically. And any criticism is immediately labeled as "crap", "people who can't see the future" etc. Remind you of anyone?
• taylorallred 1y ago
  This is why I’d love to see some more examples of people sitting down and using these tools in day-to-day work rather than just hearing “they’re great!” or “they suck!”
• DeepYogurt 1y ago
  Nuance. You love to see it
• jnfno 1y ago
  [dead]
• eachro 1y ago
  "And 50% of the time they work 50% of the time." I think this is still an incredible outcome given how many dice rolls you can take in parallel with multiple claude/o3/gemini attempts at a problem with slightly different prompts. Granted, each rollout does not come for free given the babysitting you need to do but the cost is much lower than going down the path yourself/having junior colleagues make the attempt.
• deleted 1y ago
  [deleted]
• Narciss 1y ago
  Ok, I’ll take the bait and answer the questions you’ve set in the article. Who am I? Senior engineer with 5 years of experience, working within the AI team. Project: social platform with millions of MAU Codebase: Typescript, NextJS frontend, express backend, Prisma ORM, monorepo . Hundreds of thousands of lines of code. My expertise sits in the same domain, language, codebase that I apply AI to. When do I use AI? All the time. It has significantly sped me up. How do I use AI? Almost always in a targeted fashion. I line it up like a sniper rifle - I understand my problem, where in the codebase it should act in order to apply it, and what general structure I want. Then I create the prompt to get me there. Before, I needed to do the above anyway, minus the prompt, but then I needed to use my slow hands to code dozens or hundreds of lines of code to get there. Now, the AI does it for me, and it makes me much faster. To clarify, I do not plan everything in the prompt - I let the AI color between the lines, but I do give it the general direction. Of course, I also use AI for smaller tasks like writing SQL queries for example, and it’s saved tons of time there too. In my side projects, I have a different approach - I vibe code my way through with less “sniper targeting”, because I care less about the beauty of the code organization and more about results. If I do see AI slop, I then ask the AI to clean it up, or redo the prompt with more targeting. Overall, AI has significantly sped me up. I am still heavily involved as you can tell, but it is a phenomenal tool, and I can envision it taking over more and more of the code writing process, with the right rules in place (we are actively working on that at my work, with md files and rebuilding our FE from the ground up with AI in mind). Does this sort of answer give you a better idea of how AI helps?
• cedws 1y ago
  I still see little conversation about the two fundamental limitations of LLMs right now: context size, and prompt injection. * Computation does not scale linearly with context size, meaning the ‘memory’ of LLMs is limited and gets more expensive as it gets bigger. * Prompt injection limits the usability of LLMs in the real world. How can you put an LLM in the driving seat if malicious actors can talk it into doing something it’s not supposed to. Whenever I see a blog post by Anthropic or OpenAI I do a Ctrl+F for “prompt injection.” Never mentioned. They want people to forget this is a problem — because it’s a massive one.
• rickvidallon 1y ago
  In a brief nutshell, I'm personally not in favor of how AI is taking over the industry (all of industry, that is). First of all, LLMs aren't even really what I would call AI, but that's a digression. I think, mainly, people are using it for the wrong reasons. I've seen graphics designers at work use it to generate images—and I don't just mean Adobe Firefly generative fill, but all-out "create me this logo." That's just lazy, given their job titles, and has started making everything look samey. Worse than how Bootstrap, Tailwind, etc., has the tendency to make everything samey. For someone like me, who is self-taught on essentially all of his career skills, I have particular concern for a world in which people use AI to "learn things," when that tech doesn't allow them to make mistakes. It just does things for people. For that reason alone, I don't see AI as a viable way to learn at all. If your parents never take their hand from the back of the bicycle seat, do you really know how to ride a bike without falling over? Isn't the scraped knee how we truly mastered that skill? For SEO in particular, I'd probably defer to someone with more daily experience, like you. That said, I think I can extrapolate from what I've seen elsewhere that the sameyness may likely start to affect content itself (in fact, it already has, for so many formerly good news outlets). Google search kinda blows in recent years. The AI Overview feature means so many people aren't visiting the source website anyway. To me, none of this looks appetizing. It looks like a snake eating its own tail. I don't mean to sound so bummer about the topic, but I've begun to worry about my own place in this ecosystem for the next 15, 20 years until I retire. Most of the joy in development has been sucked out of the art. Today, it seems mostly about getting wrapped around the axle of countless frameworks (without even really understanding them) and manhandling those chocolates on Lucy's and Ethel's conveyor belt. It's a comedy of errors. I'll yell "Get off my lawn" with the best of them. Add the same issues to SEO, and I don't really know where we'll end up, but it doesn't look creative, to me. It looks like a sad cliché, like the rows upon rows of sad souvenir shops that all kind of sell the same thing—the tourist trap that travelers (at least, travelers like me) actually loathe and try to avoid.
  • Ferrus91 1y ago
    Like so much of everything produced now. Mainstream film and video games have been non-AI slop for almost 2 decades now.