6 ms·

Directionally I think this is right. Most LLM usage at scale tends to be filling the gaps between two hardened interfaces. The reliability comes not from the LL

Directionally I think this is right. Most LLM usage at scale tends to be filling the gaps between two hardened interfaces. The reliability comes not from the LLM inference and generation but the interfaces themselves only allowing certain configuration to work with them. LLM output is often coerced back into something more deterministic such as types, or DB primary keys. The value of the LLM is determined by how well your existing code and tools model the data, logic, and actions of your domain. In some ways I view LLMs today a bit like 3D printers, both in terms of hype and in terms of utility. They excel at quickly connecting parts similar to rapid prototyping with 3d printing parts. For reliability and scale you want either the LLM or an engineer to replace the printed/inferred connector with something durable and deterministic (metal/code) that is cheap and fast to run at scale. Additionally, there was a minute during the 3D printer Gardner hype cycle where there were notions that we would all just print substantial amounts of consumer goods when the reality is the high utility use case are much more narrow. There is a corollary here to LLM usage. While LLMs are extremely useful we cannot rely on LLMs to generate or infer our entire operational reality or even engage meaningfully with it without some sort of pre-existing digital modeling as an anchor.

abdulhaq 1y ago
this is a really good take
foobarbecue 1y ago
Hype cycle for drones and VR was similar -- at the peak, you have people claiming drones will take over package delivery and everyone will spend their day in VR. Reality is that the applicability is more narrow.
- jangxx 1y ago
  I mean both of these things are actually happening (drone deliveries and people spending a lot of time in VR), just at a much much smaller scale than it was hyped up to be.
  - giovannibonetti 1y ago
    Drones and VR require significant upfront hardware investment, which curbs adoption. On the other hand, adopting LLM-as-a-service has none of these costs, so no wonder so many companies are getting involved with it so quickly.
    - nativeit 1y ago
      Right, but abstract costs are still costs to someone, so how far does that go before mass adoption turns into a mass liability for whomever is ultimately on the hook? It seems like there is this extremely risky wager that everyone is playing--that LLM's will find their "killer app" before the real costs of maintaining them becomes too much to bear. I don't think these kinds of bets often pay off. The opposite actually, I think every truly revolutionary technological advance in the contemporary timeframe has arisen out of its very obvious killer app(s), they were in a sense inevitable. Speculative tech--the blockchain being one of the more salient and frequently tapped examples--tends to work in pretty clear bubbles, in my estimation. I've not yet been convinced this one is any different, aside from the absurd scale at which it has been cynically sold as the biggest thing since Gutenberg, but while that makes it somewhat distinct, it's still a rather poor argument against it being a bubble.
  - pxc 1y ago
    A parallel outcome for LLMs sounds realistic to me.
  - deadbabe 1y ago
    If it’s not happening at the scale it was pitched, then it’s not happening.
    - jangxx 1y ago
      This makes no sense, just because something didn't become as big as the hypemen said it would doesn't make the inventions or users of those inventions disappear.
- soulofmischief 1y ago
  That's the claim for AR, not VR, and you're just noticing how research and development cycles play out, you can draw comparisons to literally any technology cycle.
  - 65 1y ago
    That is in fact the claim for VR. Remember the Metaverse? Oculus headsets are VR headsets. The Apple Vision Pro is a VR headset.
    - mumbisChungo 1y ago
      The metaverse is and was a guess at how the children of today might interact as they age into active market participants. Like all these other examples, speculative mania preceded genuine demand and it remains to be seen whether it plays out over the coming 10-15 years.
      - sizzle 1y ago
        Ahh yes let’s get the next generation addicted to literal screens strapped to their eyeballs for maximum monetization, humanity be damned. Glad it’s a failing bet. Now sex bots might be onto something…
        mumbisChungo 1y ago
        It may or may not be a failing bet. Maybe smartphones are the ultimate form of human-data interface and we'll simply never do better.
        jrm4 1y ago
        I'll take your argument a bit further. The thing is -- "human-data" interfaces are not particularly important. Human-Human ones are. This is probably why it's going to be difficult, if not impossible, to beat the smartphone; VR or whatever doesn't fundamentally "bring people closer together" in a way the smartphone nearly absolutely did.
- threatofrain 1y ago
  Good drones are very Chinese atm, as is casual consumer drone delivery. Americans might be more than a decade away even with concerted bipartisan war-like effort to boost domestic drone competency. The reality is Chinese.
  - sarchertech 1y ago
    Aren’t people building DIY drones that are close to and in some cases superior to off the shelf Chinese drones?
    - threatofrain 1y ago
      Off the shelf Chinese drones is somewhat vague, we can just say DJI. Their full drone and dock system for the previous generation goes for around $20k. DJI iterates on this space on a yearly cadence and have just come out with the Dock 3. 54 minute flight time (47 min hover) for fully unmanned operations. If you're talking about fpv racing where tiny drones fly around 140+ mph, then yeah DJI isn't in that space.
      - sarchertech 1y ago
        That hardly seems like it would take the US 10 years to replicate on a war footing aside from the price. I mean if we’re talking dollar to dollar comparison, the US will likely never be able to produce something as cheaply as China (unless China drastically increases their average standard of living).
        tonyarkles 1y ago
        There’s a really weird phenomenon too with drones. I’ve used Chinese (non-drone) software for work a bunch in the past and it’s been almost universally awful. On the drone side, especially DJI, they’ve flipped this script completely. Every non-DJI drone I’ve flown has had miserable UX in comparison to DJI. Mission Planner (open source, as seen in the Ukraine attack videos) is super powerful but also looks like ass and functions similarly. QGC is a bit better, especially the vendor-customized versions (BSD licensed) but the vendors almost always neuter great features that are otherwise available in the open source version and at the same time modify things so that you can’t talk to the aircraft using the OSS version. The commercial offerings I’ve used are no better. Sure, we need to be working on being able to build the hardware components in North America, and I’ve seen a bunch of people jump on that in the last year. But wow is the software ever bad and I haven’t really seen anyone working to improve that.
- ivape 1y ago
  You checked out drone warfare? It’s all the rage in every conflict at the moment. The hype around drones is not fake, and I’d compare it more to autonomous cars because regulation is the only reason you don’t see a million private drones flying around.
  - dazed_confused 1y ago
    Yes, to an extent, but I would say that is an extension of artillery and long-range fire capabilities.
    - jmj 1y ago
      As is well known, AI is whatever hasn't been done yet.
  - regmaticalistic 1y ago
    "Drone warfare" isn't really a thing. Drones in the battlefield remain something that: a) Hegemon uses to avoid needing to deploy boots to the ground b) Resource strapped combatants use as a supplement of guerrilla tactics c) Specops have used to great publicity effect in a context of very heavy use of traditional intelligence assets The narrative that "this changes everything" is very much in line with how we see the story around AI developing as well. Highly publicized demos that hide the enormous efforts in traditional technologies and good old human labor surrounding them.
- golergka 1y ago
  People claimed that we would spend most of our day on the internet in the mid-90s, and then the dotcom bubble burst. And then people claimed that by 2015 robo-taxis would be around all the major cities of the planet. You can be right but too early. There was a hype wave for drones and VR (more than one for the latter one), but I wouldn't be so sure that it's peak of their real world usage yet.
  - skeeter2020 1y ago
    >> You can be right but too early. Unless opportunity cost is zero this is a varation on being wrong.
  - TeMPOraL 1y ago
    Which is why I think there are two distinct kinds of perspective, and for one of them, AI hype is just about at the right levels - and being too early is not a problem, unless it delays things indefinitely. I wrote about it recently here: https://news.ycombinator.com/item?id=44208831 https://news.ycombinator.com/item?id=44208831. Quoting myself (sorry): > For me, one of the Beneficiaries, the hype seems totally warranted. The capability is there, the possibilities are enormous, pace of advancement is staggering, and achieving them is realistic. If it takes a few years longer than the Investor group thinks - that's fine with us; it's only a problem for them.
    - regmaticalistic 1y ago
      You're getting from it the one thing it can deliver. Dazzle and publicity.
      - TeMPOraL 1y ago
        Yes. That, and a shit ton of work done - work I wouldn't manage to do, or wouldn't even try, without AI tools.
- freetinker 1y ago
  Drones are delivering. Ukraine?
  - regmaticalistic 1y ago
    What exactly are they delivering in Ukraine, besides publicity?
    - pixl97 1y ago
      Ammunition? Supplies? Explosives to the enemy?
whiplash451 1y ago
Interesting take but too bearish on LLMs in my opinion. LLMs have already found large-scale usage (deep research, translation) which makes them more ubiquitous today than 3D printers ever will or could have been.
- dingnuts 1y ago
  And yet you didn't provide a single reference link! Every case of LLM usage that I've seen claimed about those things has been largely a lie -- guess you won't take the opportunity to be the first to present a real example. Just another rumor.
  - whiplash451 1y ago
    My reference is the daily usage of chatgpt around me (outside of tech circles). I don’t want to sound like a hard-core LLM believer. I get your point and it’s fair. I just wanted to point out that the current usage of chatgpt is a lot broader than that of 3D printers even at the peak hype of it.
    - dingnuts 1y ago
      Outside of tech circles it looks like NFTs: people following hype using tech they don't understand which will be popular until the downsides we're aware of that they are ignorant to have consequences, and then the market will reflect the shift in opinion.
      - whiplash451 1y ago
        I see it differently: people are switching to chatgpt like they switched to google back in 2005 (from whatever alternative existed back then) And I mean random people, not tech circles It’s very different from NFTs in that respect
      - basch 1y ago
        No way. Everybody under a certain age is using ChatGPT, where they were once using search and friendship/expertises. It’s the number 1 app in the App Store. Copilot use in the enterprise is so seamless, you just talk to PowerPoint or outlook and it formulated what you were supposed to make or write. It’s not a fad, it is a paradigm change. People don’t need to understand how it works for it to work.
hk1337 1y ago
> Directionally I think this is right. We have a term at work we use called, "directionally accurate", when it's not entirely accurate but headed in the right direction.

abdulhaq 1y ago

this is a really good take

foobarbecue 1y ago

Hype cycle for drones and VR was similar -- at the peak, you have people claiming drones will take over package delivery and everyone will spend their day in VR. Reality is that the applicability is more narrow.

jangxx 1y ago
I mean both of these things are actually happening (drone deliveries and people spending a lot of time in VR), just at a much much smaller scale than it was hyped up to be.
- giovannibonetti 1y ago
  Drones and VR require significant upfront hardware investment, which curbs adoption. On the other hand, adopting LLM-as-a-service has none of these costs, so no wonder so many companies are getting involved with it so quickly.
  - nativeit 1y ago
    Right, but abstract costs are still costs to someone, so how far does that go before mass adoption turns into a mass liability for whomever is ultimately on the hook? It seems like there is this extremely risky wager that everyone is playing--that LLM's will find their "killer app" before the real costs of maintaining them becomes too much to bear. I don't think these kinds of bets often pay off. The opposite actually, I think every truly revolutionary technological advance in the contemporary timeframe has arisen out of its very obvious killer app(s), they were in a sense inevitable. Speculative tech--the blockchain being one of the more salient and frequently tapped examples--tends to work in pretty clear bubbles, in my estimation. I've not yet been convinced this one is any different, aside from the absurd scale at which it has been cynically sold as the biggest thing since Gutenberg, but while that makes it somewhat distinct, it's still a rather poor argument against it being a bubble.
- pxc 1y ago
  A parallel outcome for LLMs sounds realistic to me.
- deadbabe 1y ago
  If it’s not happening at the scale it was pitched, then it’s not happening.
  - jangxx 1y ago
    This makes no sense, just because something didn't become as big as the hypemen said it would doesn't make the inventions or users of those inventions disappear.
soulofmischief 1y ago
That's the claim for AR, not VR, and you're just noticing how research and development cycles play out, you can draw comparisons to literally any technology cycle.
- 65 1y ago
  That is in fact the claim for VR. Remember the Metaverse? Oculus headsets are VR headsets. The Apple Vision Pro is a VR headset.
  - mumbisChungo 1y ago
    The metaverse is and was a guess at how the children of today might interact as they age into active market participants. Like all these other examples, speculative mania preceded genuine demand and it remains to be seen whether it plays out over the coming 10-15 years.
    - sizzle 1y ago
      Ahh yes let’s get the next generation addicted to literal screens strapped to their eyeballs for maximum monetization, humanity be damned. Glad it’s a failing bet. Now sex bots might be onto something…
      - mumbisChungo 1y ago
        It may or may not be a failing bet. Maybe smartphones are the ultimate form of human-data interface and we'll simply never do better.
        jrm4 1y ago
        I'll take your argument a bit further. The thing is -- "human-data" interfaces are not particularly important. Human-Human ones are. This is probably why it's going to be difficult, if not impossible, to beat the smartphone; VR or whatever doesn't fundamentally "bring people closer together" in a way the smartphone nearly absolutely did.
threatofrain 1y ago
Good drones are very Chinese atm, as is casual consumer drone delivery. Americans might be more than a decade away even with concerted bipartisan war-like effort to boost domestic drone competency. The reality is Chinese.
- sarchertech 1y ago
  Aren’t people building DIY drones that are close to and in some cases superior to off the shelf Chinese drones?
  - threatofrain 1y ago
    Off the shelf Chinese drones is somewhat vague, we can just say DJI. Their full drone and dock system for the previous generation goes for around $20k. DJI iterates on this space on a yearly cadence and have just come out with the Dock 3. 54 minute flight time (47 min hover) for fully unmanned operations. If you're talking about fpv racing where tiny drones fly around 140+ mph, then yeah DJI isn't in that space.
    - sarchertech 1y ago
      That hardly seems like it would take the US 10 years to replicate on a war footing aside from the price. I mean if we’re talking dollar to dollar comparison, the US will likely never be able to produce something as cheaply as China (unless China drastically increases their average standard of living).
      - tonyarkles 1y ago
        There’s a really weird phenomenon too with drones. I’ve used Chinese (non-drone) software for work a bunch in the past and it’s been almost universally awful. On the drone side, especially DJI, they’ve flipped this script completely. Every non-DJI drone I’ve flown has had miserable UX in comparison to DJI. Mission Planner (open source, as seen in the Ukraine attack videos) is super powerful but also looks like ass and functions similarly. QGC is a bit better, especially the vendor-customized versions (BSD licensed) but the vendors almost always neuter great features that are otherwise available in the open source version and at the same time modify things so that you can’t talk to the aircraft using the OSS version. The commercial offerings I’ve used are no better. Sure, we need to be working on being able to build the hardware components in North America, and I’ve seen a bunch of people jump on that in the last year. But wow is the software ever bad and I haven’t really seen anyone working to improve that.
ivape 1y ago
You checked out drone warfare? It’s all the rage in every conflict at the moment. The hype around drones is not fake, and I’d compare it more to autonomous cars because regulation is the only reason you don’t see a million private drones flying around.
- dazed_confused 1y ago
  Yes, to an extent, but I would say that is an extension of artillery and long-range fire capabilities.
  - jmj 1y ago
    As is well known, AI is whatever hasn't been done yet.
- regmaticalistic 1y ago
  "Drone warfare" isn't really a thing. Drones in the battlefield remain something that: a) Hegemon uses to avoid needing to deploy boots to the ground b) Resource strapped combatants use as a supplement of guerrilla tactics c) Specops have used to great publicity effect in a context of very heavy use of traditional intelligence assets The narrative that "this changes everything" is very much in line with how we see the story around AI developing as well. Highly publicized demos that hide the enormous efforts in traditional technologies and good old human labor surrounding them.
golergka 1y ago
People claimed that we would spend most of our day on the internet in the mid-90s, and then the dotcom bubble burst. And then people claimed that by 2015 robo-taxis would be around all the major cities of the planet. You can be right but too early. There was a hype wave for drones and VR (more than one for the latter one), but I wouldn't be so sure that it's peak of their real world usage yet.
- skeeter2020 1y ago
  >> You can be right but too early. Unless opportunity cost is zero this is a varation on being wrong.
- TeMPOraL 1y ago
  Which is why I think there are two distinct kinds of perspective, and for one of them, AI hype is just about at the right levels - and being too early is not a problem, unless it delays things indefinitely. I wrote about it recently here: https://news.ycombinator.com/item?id=44208831 https://news.ycombinator.com/item?id=44208831. Quoting myself (sorry): > For me, one of the Beneficiaries, the hype seems totally warranted. The capability is there, the possibilities are enormous, pace of advancement is staggering, and achieving them is realistic. If it takes a few years longer than the Investor group thinks - that's fine with us; it's only a problem for them.
  - regmaticalistic 1y ago
    You're getting from it the one thing it can deliver. Dazzle and publicity.
    - TeMPOraL 1y ago
      Yes. That, and a shit ton of work done - work I wouldn't manage to do, or wouldn't even try, without AI tools.
freetinker 1y ago
Drones are delivering. Ukraine?
- regmaticalistic 1y ago
  What exactly are they delivering in Ukraine, besides publicity?
  - pixl97 1y ago
    Ammunition? Supplies? Explosives to the enemy?

whiplash451 1y ago

Interesting take but too bearish on LLMs in my opinion. LLMs have already found large-scale usage (deep research, translation) which makes them more ubiquitous today than 3D printers ever will or could have been.

dingnuts 1y ago
And yet you didn't provide a single reference link! Every case of LLM usage that I've seen claimed about those things has been largely a lie -- guess you won't take the opportunity to be the first to present a real example. Just another rumor.
- whiplash451 1y ago
  My reference is the daily usage of chatgpt around me (outside of tech circles). I don’t want to sound like a hard-core LLM believer. I get your point and it’s fair. I just wanted to point out that the current usage of chatgpt is a lot broader than that of 3D printers even at the peak hype of it.
  - dingnuts 1y ago
    Outside of tech circles it looks like NFTs: people following hype using tech they don't understand which will be popular until the downsides we're aware of that they are ignorant to have consequences, and then the market will reflect the shift in opinion.
    - whiplash451 1y ago
      I see it differently: people are switching to chatgpt like they switched to google back in 2005 (from whatever alternative existed back then) And I mean random people, not tech circles It’s very different from NFTs in that respect
    - basch 1y ago
      No way. Everybody under a certain age is using ChatGPT, where they were once using search and friendship/expertises. It’s the number 1 app in the App Store. Copilot use in the enterprise is so seamless, you just talk to PowerPoint or outlook and it formulated what you were supposed to make or write. It’s not a fad, it is a paradigm change. People don’t need to understand how it works for it to work.

hk1337 1y ago

> Directionally I think this is right. We have a term at work we use called, "directionally accurate", when it's not entirely accurate but headed in the right direction.