3 ms·

Impressive numbers, but have you ever tried to use one of these things to get actual work done? They're terrible at it! Until these companies start demonstratin

Impressive numbers, but have you ever tried to use one of these things to get actual work done? They're terrible at it! Until these companies start demonstrating some actual value this is merely a hype bubble. Nvidia is slightly different categorically because they're selling the picks and shovels in this hype gold rush--but they were also in the last one (web3/crypto/nfts). At some point if these folks don't actually start delivering the value they're hyping it's all coming crashing down.

sensanaty 2y ago
The thing is, it's mostly managers and the like getting wow'd by how cool it seems initially, and they're the ones driving the hype. I can ask it in plain English and it answers with flowery language that reads like a professional wrote it, wow! Of course, once you actually try to use it for any real work that isn't a toy project with a trillion examples online, you very quickly run into the myriad of flaws. But they're not using it deeply like that, they're just interested in how they can cut costs or prop up their own product by wow-ing other managers with AI features even if they don't make a lick of sense from an end-user perspective. This isn't even to say it's completely useless, just that the hype and marketing is so insanely divergent from the reality, it's shocking. For me, it's at best a 5% productivity boost for very specific tasks like formatting structured text or something like that, but I wouldn't say that justifies a 500 billion dollar investment...
- snookeredpoot 2y ago
  Your opinion, like many others, is wildly misinformed. It’s not “just managers”. People are legit using copilot and it is saving time and helping develop better code. There are 100+ engineers in my company using it. And it’s not just copiloting, it is better than Wikipedia for explaining RFCs or academic papers. That is very important for ramping people when they can interactively ask hundreds of questions to learn about something without bothering top engineers. You really need to actually get some real world experience as not just pop off. Anyone who refers to LLMs as “stochastic parrots” has made their choice and is not interested in understand what areas can benefit and what areas cannot. They’re just scared and ignorant.
  - mattmanser 2y ago
    Copilot? I personally found Copilot pretty useless. And very distracting trying to constantly incorrectly guess what I was doing. I do use chatgpt/claude for quickly creating first draftw of frontend pages as I find it quicker, but I have to do extensive edits to get it right. But anything non-boiler plate I am faster writing it myself. I have also quickly found it is a stochastic parrot but that's ok for boilerplate stuff. It is useless at anything complicated and quickly starts hallucinating methods, parameters and functions when you point out problems with the code. I am beginning to believe that anything niche, where it can't steal from its training data, and it will always be utterly useless. I am curious how you feel it's so important to 100+ engineers, how do you use it that makes it effective for you?
    - 3vidence 2y ago
      Thank you! Maybe there is just a divide between engineers working on deep complex topics and those trying to setup their 17th Django app. LLMs are a totally useful tool! They definitely help me write all the bash scripts I consistently forget. But the deep stuff where the value lies? It starts to fall apart quickly.
      - jcgrillo 2y ago
        You put this better than I could have. I am really impressed by LLMs' ability to respond to a prompt with code which compiles and runs. That's a hugely impressive result. But it doesn't mean the technorapture is just around the corner. There's a vast gulf between what LLMs can do currently and what skilled people do, and it's not "obvious" that there's some path from here to a world in which LLMs become some kind of superintelligent engineer-gods. It could be physically impossible, for all we know. That's why the pseudoreligious twaddle gets under my skin so much--it's the naive assumption that progress is linear, instead of a jumpy "fits and starts" process of accidental discoveries building on one another (or encountering dead ends). There's no way to know whether these things can be improved enough to be truly useful for real work and pretending it's "right around the corner" won't necessarily make it happen.
  - 2y ago
edanm 2y ago
> Impressive numbers, but have you ever tried to use one of these things to get actual work done? They're terrible at it! If you mean "have you tried using an LLM like ChatGPT or Claude for doing development work", then you're wrong. I'm very confident in saying this, both because I personally use AI for programming and get a lot of value from it, and also because almost everyone else I know in real life also uses AI for programming. But also because there are dozens/hundreds of accounts of very good developers getting a lot of value from AI in their work. So at this point, insisting AI products are terrible doesn't say much about the AI products, so much as it says something about your ability to get value out of them.
- isbvhodnvemrwvn 2y ago
  you should probably disclose what you work on as this makes you not exactly impartial, it would be quite bad for you to criticize value of what you sell: > I'm the CEO of Hipposys Ltd, a boutique Data & AI Engineering shop (www.hipposys.com). We specialize in building RAG systems, Data Warehouses, and anything else related to the intersection between Data & AI. If you have any questions, please feel free to reach out to me at edan@hipposys.com, and we can see if we are a good fit!
  - edanm 2y ago
    First of all, I'm not sure what you mean by "disclose" - you took that from my HN profile which is a click away. Do you think I should mention this in every comment I make about LLMs? Secondly, you're confusing cause and effect. I've been in the industry for more than twenty years working on many different things, and my company has done mostly data engineering work for most of its existence, the AI engineering is a relatively-new (past year or so) shift. I don't think highly of AI because I work in AI - I work in AI because I think highly of it!
bdangubic 2y ago
there are MANY of us who have not only tried it but using it daily to do amazeballs shit. you can read numerous other posts here of people saying how indispensible they have become in their day-to-day work. just like with any other tool, some will learn its strengths and limitations and take them to the limit while others will say “this is no good…” if I was in the first cohort I would think long and hard how to move to the second…

sensanaty 2y ago

The thing is, it's mostly managers and the like getting wow'd by how cool it seems initially, and they're the ones driving the hype. I can ask it in plain English and it answers with flowery language that reads like a professional wrote it, wow! Of course, once you actually try to use it for any real work that isn't a toy project with a trillion examples online, you very quickly run into the myriad of flaws. But they're not using it deeply like that, they're just interested in how they can cut costs or prop up their own product by wow-ing other managers with AI features even if they don't make a lick of sense from an end-user perspective. This isn't even to say it's completely useless, just that the hype and marketing is so insanely divergent from the reality, it's shocking. For me, it's at best a 5% productivity boost for very specific tasks like formatting structured text or something like that, but I wouldn't say that justifies a 500 billion dollar investment...

snookeredpoot 2y ago
Your opinion, like many others, is wildly misinformed. It’s not “just managers”. People are legit using copilot and it is saving time and helping develop better code. There are 100+ engineers in my company using it. And it’s not just copiloting, it is better than Wikipedia for explaining RFCs or academic papers. That is very important for ramping people when they can interactively ask hundreds of questions to learn about something without bothering top engineers. You really need to actually get some real world experience as not just pop off. Anyone who refers to LLMs as “stochastic parrots” has made their choice and is not interested in understand what areas can benefit and what areas cannot. They’re just scared and ignorant.
- mattmanser 2y ago
  Copilot? I personally found Copilot pretty useless. And very distracting trying to constantly incorrectly guess what I was doing. I do use chatgpt/claude for quickly creating first draftw of frontend pages as I find it quicker, but I have to do extensive edits to get it right. But anything non-boiler plate I am faster writing it myself. I have also quickly found it is a stochastic parrot but that's ok for boilerplate stuff. It is useless at anything complicated and quickly starts hallucinating methods, parameters and functions when you point out problems with the code. I am beginning to believe that anything niche, where it can't steal from its training data, and it will always be utterly useless. I am curious how you feel it's so important to 100+ engineers, how do you use it that makes it effective for you?
  - 3vidence 2y ago
    Thank you! Maybe there is just a divide between engineers working on deep complex topics and those trying to setup their 17th Django app. LLMs are a totally useful tool! They definitely help me write all the bash scripts I consistently forget. But the deep stuff where the value lies? It starts to fall apart quickly.
    - jcgrillo 2y ago
      You put this better than I could have. I am really impressed by LLMs' ability to respond to a prompt with code which compiles and runs. That's a hugely impressive result. But it doesn't mean the technorapture is just around the corner. There's a vast gulf between what LLMs can do currently and what skilled people do, and it's not "obvious" that there's some path from here to a world in which LLMs become some kind of superintelligent engineer-gods. It could be physically impossible, for all we know. That's why the pseudoreligious twaddle gets under my skin so much--it's the naive assumption that progress is linear, instead of a jumpy "fits and starts" process of accidental discoveries building on one another (or encountering dead ends). There's no way to know whether these things can be improved enough to be truly useful for real work and pretending it's "right around the corner" won't necessarily make it happen.
- 2y ago

edanm 2y ago

> Impressive numbers, but have you ever tried to use one of these things to get actual work done? They're terrible at it! If you mean "have you tried using an LLM like ChatGPT or Claude for doing development work", then you're wrong. I'm very confident in saying this, both because I personally use AI for programming and get a lot of value from it, and also because almost everyone else I know in real life also uses AI for programming. But also because there are dozens/hundreds of accounts of very good developers getting a lot of value from AI in their work. So at this point, insisting AI products are terrible doesn't say much about the AI products, so much as it says something about your ability to get value out of them.

isbvhodnvemrwvn 2y ago
you should probably disclose what you work on as this makes you not exactly impartial, it would be quite bad for you to criticize value of what you sell: > I'm the CEO of Hipposys Ltd, a boutique Data & AI Engineering shop (www.hipposys.com). We specialize in building RAG systems, Data Warehouses, and anything else related to the intersection between Data & AI. If you have any questions, please feel free to reach out to me at edan@hipposys.com, and we can see if we are a good fit!
- edanm 2y ago
  First of all, I'm not sure what you mean by "disclose" - you took that from my HN profile which is a click away. Do you think I should mention this in every comment I make about LLMs? Secondly, you're confusing cause and effect. I've been in the industry for more than twenty years working on many different things, and my company has done mostly data engineering work for most of its existence, the AI engineering is a relatively-new (past year or so) shift. I don't think highly of AI because I work in AI - I work in AI because I think highly of it!

bdangubic 2y ago

there are MANY of us who have not only tried it but using it daily to do amazeballs shit. you can read numerous other posts here of people saying how indispensible they have become in their day-to-day work. just like with any other tool, some will learn its strengths and limitations and take them to the limit while others will say “this is no good…” if I was in the first cohort I would think long and hard how to move to the second…