5 ms·

They want to throw LLMs at everything even if it does not make sense. Same is true for all the AI agent craze: https://medium.com/thoughts-on-machine-learning/l

by fsndz 2y ago

They want to throw LLMs at everything even if it does not make sense. Same is true for all the AI agent craze: https://medium.com/thoughts-on-machine-learning/langchains-state-of-ai-agents-report-is-mostly-bullshit-c689d0021a19 https://medium.com/thoughts-on-machine-learning/langchains-s...

marcosdumay 2y ago
If feels like the entire world has gone crazy. Even the serious idea that the article thinks could work is throwing the unreliable LLMs at verification! If there's any place you can use something that doesn't work most of the time, I guess it's there.
- deadbabe 2y ago
  This is typical of any hype bubble. Blockchain used to be the answer to everything.
  - Mistletoe 2y ago
    What's after this? Because I really do feel the economy is standing on a cliff right now. I don't see anything after this that can prop stocks up.
    - deadbabe 2y ago
      The post-quantum age. Companies will go post-quantum.
      - namaria 2y ago
        I think the operators are learning how to hype-edge. You find that sweet spot between promising and 'not just there yet' where you can take lots of investments and iterate forward just enough to keep it going. It doesn't matter if it can't actually 'get there' as long as people still believe it can. Come to think about it, a socioeconomic system dependent on population and economic growth is at a fundamental level driven by this balancing act: "We can solve every problem if we just forge ahead and keep enlarging the base of the pyramid - keep reproducing, keep investing, keep expanding the infrastructure".
    - dgfitz 2y ago
      That’s because we are still waiting for the 2008 bubble to pop, which was inflated by the 2020 bubble. It’s going to be bad. People will blame trump, Harris would be eating the same shit sandwich. It’s gonna be bad.
      - marcosdumay 2y ago
        What makes you think he won't just inflate the bubble again? Should we expect money pumps to generate inflation quicker on this cycle than on the last ones? If so, why?
        dgfitz 2y ago
        I think only an ignorant person doesn’t see the train wreck coming, and how making more money won’t fix fuck all.
- ajuc 2y ago
  It's similar in regular programming - LLMs are better at writing test code than actual code. Mostly because it's simpler (P vs NP etc), but I think also because it's less obvious when test code doesn't work. Replace all asserts with expected ==expected and most people won't notice.
  - jeltz 2y ago
    > Replace all asserts with expected ==expected and most people won't notice. Those tests were very common back when I used to work in Ruby on Rails and automatically generating test stubs was a popular practice. These stubs were often just converted into expected == expected tests so that they passed and then left like that.
  - majormajor 2y ago
    LLMs are pretty damn useful for generating tests, getting rid of a lot of tedium, but yeah, it's the same as human-written tests: if you don't check that your test doesn't work when it shouldn't (not the same thing as just writing a second test for that case - both those tests need to fail if you intentionally screw with their separate fixtures), then you shouldn't have too much confidence in your test.
    - marcosdumay 2y ago
      If LLMs can generate a test for you, it's because it's a test that you shouldn't need to write. They can't test what is really important, at all. Some development stacks are extremely underpowered for code verification, so they do patch the design issue. Just like some stacks are underpowered for abstraction and need patching by code generation. Both of those solve an immediate problem, in a haphazard and error-prone way, by adding burden on maintenance and code evolution linearly to how much you use it. And worse, if you rely too much on them they will lead your software architecture and make that burden superlinear.
      - williamcotton 2y ago
        Claude wrote the harness and pretty much all of these tests, eg: https://github.com/williamcotton/search-input-query/blob/main/search-input-query-parser/src/parser.test.ts#L187 https://github.com/williamcotton/search-input-query/blob/mai... It is a good test suite and it saved me quite a bit of typing! In fact, Claude did most of the typing for the entire project: https://github.com/williamcotton/search-input-query https://github.com/williamcotton/search-input-query BTW, I obviously didn't just type "make a lexer and multi-pass parser that returns multiple errors and then make a single-line instance of a Monaco editor with error reporting, type checking, syntax highlighting and tab completion". I put it together piece-by-piece and with detailed architectural guidance.
- edmundsauto 2y ago
  Only if it fails in the same way. LLMs and the multi-agent approach operate under the assumption that they are programmable agents and each agent is more of a trade off against failure modes. If you can string them together, and if the output is easily verified, it can be a great fit for the problem.
  - astrange 2y ago
    If you're going to do that you need completely different LLMs to base the agents on. The ones I've tried have "mode collapse" - ask them to emulate different agents and they'll all end up behaving the same way. Simple example, if you ask it to write different stories they'll usually end up having the same character names.
    - edmundsauto 2y ago
      It may depend on the domain. I tend to use LLMs for things that are less open ended, more categorization and summarization response than pure novel creation. In these situations, I’ve been able to sufficiently program the agent that I haven’t seen too much of an issue as you described. Consistency is a feature.
- FredPret 2y ago
  This happens all the time. Once it was spices. Then poppies. Modern art. The .com craze. Those blockchain ape images. Blockchain. Now LLM. All of these had a bit of true value and a whole load of bullshit. Eventually the bullshit disappears and the core remains, and the world goes nuts about the next thing.
  - vishnugupta 2y ago
    Exactly. I’ve seen this enough now to appreciate that oft repeated tech adoption curve. It seems like we are in “peak expectations” phase which is immediately followed by the disillusionment and then maturity phase.
- cwzwarich 2y ago
  If your LLM is producing a proof that can be checked by another program, then there’s nothing wrong with their reliability. It’s just like playing a game whose rules are a logical system.
ReptileMan 2y ago
Isn't that the case with every new tech. There was a time in which people tried to cook everything in a microwave
- dangsux 2y ago
  [dead]
- ksynwa 2y ago
  Microwave sellers did not become trillion dollar companies off that hype
  - ReptileMan 2y ago
    Mostly because the marginal cost of microwaves was not close to zero.
    - ksynwa 2y ago
      Mostly because they were not making claims that sentient microwaves that would cook your food for you were just around the corner which then the most respected media outlets parroted uncritically.
      - rsynnott 2y ago
        I mean, they were at one point making pretty extravagant claims about microwaves, but to a less credulous audience. Trouble with LLMs is that they look like magic if you don’t look too hard, particularly to laypeople. It’s far easier to buy into a narrative that they actually _are_ magic, or will become so.
        lxgr 2y ago
        I feel like what makes this a bit different from just regular old sufficiently advanced technology is the combination of two things: - LLMs are extremely competent at surface-level pattern matching and manipulation of the type we'd previously assumed that only AGI would be able to do. - A large fraction of tasks (and by extension jobs) that we used to, and largely still do, consider to be "knowledge work", i.e. requiring a high level of skill and intelligence, are in fact surface-level pattern matching and manipulation. Reconciling these facts raises some uncomfortable implications, and calling LLMs "actually intelligent" lets us avoid these.
wslh 2y ago
This makes complete sense from an investor’s perspective, as it increases the chances of a successful exit. While we focus on the technical merits or critique here on HN/YC, investors are playing a completely different game. To be a bit acerbic, and inspired by Arthur C. Clarke, I might say: "Any sufficiently complex business could be indistinguishable from Theranos".
- spencerchubb 2y ago
  Theranos was not a "complex business". It was deliberate fraud and deception, and investors that were just gullible. The investors should have demanded to see concrete results
  - wslh 2y ago
    I expected you to take this with a grain of salt but also to read between the lines: while some projects involve deliberate fraud, others may simply lack coherence and inadvertently follow the principles of the greater fool theory [1]. The use of ambiguous or indistinguishable language often blurs the distinction, making it harder to differentiate outright deception from an unsound business model. [1] https://en.wikipedia.org/wiki/Greater_fool_theory https://en.wikipedia.org/wiki/Greater_fool_theory
xbmcuser 2y ago
yes thats how we progress this is how the internet boom happened as well everything became . com then the real workable businesses were left and all the unworkable things were gone. Recently I came across some one advertising an LLM to generate fashion magazine shoot in Pakistan at 20-25% of the cost. It hit me then that they are undercutting the fashion shoot of country like Pakistan which is already cheaper by 90-95% from most western countries. This AI is replacing the work of 10-20 people.
- startupsfail 2y ago
  The annoying part, a lot of money could be funneled into these unworkable businesses in the process, crypto being a good example. And these unworkable businesses tend to try to continue getting their way into the money somehow regardless. Most recent example was funneling money from Russia into Trump’s campaign.
  - bubaumba 2y ago
    > The annoying part, a lot of money could be funneled into these unworkable businesses in the process, crypto being a good example There was a thread here about why ycombinator invests into several competing startups. The answer is success is often more about connections and politics than the product itself. And crypto, yes, is a good example of this. Musk will get his $1B in bitcoins back for sure. > Most recent example was funneling money from Russia into Trump’s campaign. Musk again?
alw4s 2y ago
please dont post a link that is behind a paywall !!
- ksynwa 2y ago
  https://archive.is/dLp6t https://archive.is/dLp6t It is a registration wall I think.
  - tomrod 2y ago
    Same result. Information locks are verboten.
    - lxgr 2y ago
      As annoying as I find them, on this site they're in fact not: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989
- lxgr 2y ago
  Please don't complain about paywalls: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989
trolan 2y ago
https://archive.ph/dLp6t https://archive.ph/dLp6t
rsynnott 2y ago
It really feels like we’re close to the end of the current bubble now; the applications being trotted out are just increasingly absurd.
spencerchubb 2y ago
LLMs have powered products used by hundreds of millions, maybe billions. Most experiments will fail and that's okay, arguably even a good thing. Only time will tell which ones succeed
logifail 2y ago
> They want to throw LLMs at everything [..] Oh yes. I had a discussion with a manager at a client last week and was trying to run him through some (technical) issues relating to challenges an important project faces. His immediate response was that maybe we should just let ChatGPT help us decide the best option. I had to bite my tongue. OTOH, I'm more and more convinced that ChatGPT will replace managers long before it replaces technical staff.
isoprophlex 2y ago
> I knew it was bullshit from the get-go as soon as I read their definition of AI agents. That is one spicy article, it got a few laughs out of me. I must agree 100% that Langchain is an abomination, both their APIs as well as their marketing.

by fsndz 2y ago

marcosdumay 2y ago

If feels like the entire world has gone crazy. Even the serious idea that the article thinks could work is throwing the unreliable LLMs at verification! If there's any place you can use something that doesn't work most of the time, I guess it's there.

deadbabe 2y ago
This is typical of any hype bubble. Blockchain used to be the answer to everything.
- Mistletoe 2y ago
  What's after this? Because I really do feel the economy is standing on a cliff right now. I don't see anything after this that can prop stocks up.
  - deadbabe 2y ago
    The post-quantum age. Companies will go post-quantum.
    - namaria 2y ago
      I think the operators are learning how to hype-edge. You find that sweet spot between promising and 'not just there yet' where you can take lots of investments and iterate forward just enough to keep it going. It doesn't matter if it can't actually 'get there' as long as people still believe it can. Come to think about it, a socioeconomic system dependent on population and economic growth is at a fundamental level driven by this balancing act: "We can solve every problem if we just forge ahead and keep enlarging the base of the pyramid - keep reproducing, keep investing, keep expanding the infrastructure".
  - dgfitz 2y ago
    That’s because we are still waiting for the 2008 bubble to pop, which was inflated by the 2020 bubble. It’s going to be bad. People will blame trump, Harris would be eating the same shit sandwich. It’s gonna be bad.
    - marcosdumay 2y ago
      What makes you think he won't just inflate the bubble again? Should we expect money pumps to generate inflation quicker on this cycle than on the last ones? If so, why?
      - dgfitz 2y ago
        I think only an ignorant person doesn’t see the train wreck coming, and how making more money won’t fix fuck all.
ajuc 2y ago
It's similar in regular programming - LLMs are better at writing test code than actual code. Mostly because it's simpler (P vs NP etc), but I think also because it's less obvious when test code doesn't work. Replace all asserts with expected ==expected and most people won't notice.
- jeltz 2y ago
  > Replace all asserts with expected ==expected and most people won't notice. Those tests were very common back when I used to work in Ruby on Rails and automatically generating test stubs was a popular practice. These stubs were often just converted into expected == expected tests so that they passed and then left like that.
- majormajor 2y ago
  LLMs are pretty damn useful for generating tests, getting rid of a lot of tedium, but yeah, it's the same as human-written tests: if you don't check that your test doesn't work when it shouldn't (not the same thing as just writing a second test for that case - both those tests need to fail if you intentionally screw with their separate fixtures), then you shouldn't have too much confidence in your test.
  - marcosdumay 2y ago
    If LLMs can generate a test for you, it's because it's a test that you shouldn't need to write. They can't test what is really important, at all. Some development stacks are extremely underpowered for code verification, so they do patch the design issue. Just like some stacks are underpowered for abstraction and need patching by code generation. Both of those solve an immediate problem, in a haphazard and error-prone way, by adding burden on maintenance and code evolution linearly to how much you use it. And worse, if you rely too much on them they will lead your software architecture and make that burden superlinear.
    - williamcotton 2y ago
      Claude wrote the harness and pretty much all of these tests, eg: https://github.com/williamcotton/search-input-query/blob/main/search-input-query-parser/src/parser.test.ts#L187 https://github.com/williamcotton/search-input-query/blob/mai... It is a good test suite and it saved me quite a bit of typing! In fact, Claude did most of the typing for the entire project: https://github.com/williamcotton/search-input-query https://github.com/williamcotton/search-input-query BTW, I obviously didn't just type "make a lexer and multi-pass parser that returns multiple errors and then make a single-line instance of a Monaco editor with error reporting, type checking, syntax highlighting and tab completion". I put it together piece-by-piece and with detailed architectural guidance.
edmundsauto 2y ago
Only if it fails in the same way. LLMs and the multi-agent approach operate under the assumption that they are programmable agents and each agent is more of a trade off against failure modes. If you can string them together, and if the output is easily verified, it can be a great fit for the problem.
- astrange 2y ago
  If you're going to do that you need completely different LLMs to base the agents on. The ones I've tried have "mode collapse" - ask them to emulate different agents and they'll all end up behaving the same way. Simple example, if you ask it to write different stories they'll usually end up having the same character names.
  - edmundsauto 2y ago
    It may depend on the domain. I tend to use LLMs for things that are less open ended, more categorization and summarization response than pure novel creation. In these situations, I’ve been able to sufficiently program the agent that I haven’t seen too much of an issue as you described. Consistency is a feature.
FredPret 2y ago
This happens all the time. Once it was spices. Then poppies. Modern art. The .com craze. Those blockchain ape images. Blockchain. Now LLM. All of these had a bit of true value and a whole load of bullshit. Eventually the bullshit disappears and the core remains, and the world goes nuts about the next thing.
- vishnugupta 2y ago
  Exactly. I’ve seen this enough now to appreciate that oft repeated tech adoption curve. It seems like we are in “peak expectations” phase which is immediately followed by the disillusionment and then maturity phase.
cwzwarich 2y ago
If your LLM is producing a proof that can be checked by another program, then there’s nothing wrong with their reliability. It’s just like playing a game whose rules are a logical system.

ReptileMan 2y ago

Isn't that the case with every new tech. There was a time in which people tried to cook everything in a microwave

dangsux 2y ago
[dead]
ksynwa 2y ago
Microwave sellers did not become trillion dollar companies off that hype
- ReptileMan 2y ago
  Mostly because the marginal cost of microwaves was not close to zero.
  - ksynwa 2y ago
    Mostly because they were not making claims that sentient microwaves that would cook your food for you were just around the corner which then the most respected media outlets parroted uncritically.
    - rsynnott 2y ago
      I mean, they were at one point making pretty extravagant claims about microwaves, but to a less credulous audience. Trouble with LLMs is that they look like magic if you don’t look too hard, particularly to laypeople. It’s far easier to buy into a narrative that they actually _are_ magic, or will become so.
      - lxgr 2y ago
        I feel like what makes this a bit different from just regular old sufficiently advanced technology is the combination of two things: - LLMs are extremely competent at surface-level pattern matching and manipulation of the type we'd previously assumed that only AGI would be able to do. - A large fraction of tasks (and by extension jobs) that we used to, and largely still do, consider to be "knowledge work", i.e. requiring a high level of skill and intelligence, are in fact surface-level pattern matching and manipulation. Reconciling these facts raises some uncomfortable implications, and calling LLMs "actually intelligent" lets us avoid these.

wslh 2y ago

This makes complete sense from an investor’s perspective, as it increases the chances of a successful exit. While we focus on the technical merits or critique here on HN/YC, investors are playing a completely different game. To be a bit acerbic, and inspired by Arthur C. Clarke, I might say: "Any sufficiently complex business could be indistinguishable from Theranos".

spencerchubb 2y ago
Theranos was not a "complex business". It was deliberate fraud and deception, and investors that were just gullible. The investors should have demanded to see concrete results
- wslh 2y ago
  I expected you to take this with a grain of salt but also to read between the lines: while some projects involve deliberate fraud, others may simply lack coherence and inadvertently follow the principles of the greater fool theory [1]. The use of ambiguous or indistinguishable language often blurs the distinction, making it harder to differentiate outright deception from an unsound business model. [1] https://en.wikipedia.org/wiki/Greater_fool_theory https://en.wikipedia.org/wiki/Greater_fool_theory

xbmcuser 2y ago

yes thats how we progress this is how the internet boom happened as well everything became . com then the real workable businesses were left and all the unworkable things were gone. Recently I came across some one advertising an LLM to generate fashion magazine shoot in Pakistan at 20-25% of the cost. It hit me then that they are undercutting the fashion shoot of country like Pakistan which is already cheaper by 90-95% from most western countries. This AI is replacing the work of 10-20 people.

startupsfail 2y ago
The annoying part, a lot of money could be funneled into these unworkable businesses in the process, crypto being a good example. And these unworkable businesses tend to try to continue getting their way into the money somehow regardless. Most recent example was funneling money from Russia into Trump’s campaign.
- bubaumba 2y ago
  > The annoying part, a lot of money could be funneled into these unworkable businesses in the process, crypto being a good example There was a thread here about why ycombinator invests into several competing startups. The answer is success is often more about connections and politics than the product itself. And crypto, yes, is a good example of this. Musk will get his $1B in bitcoins back for sure. > Most recent example was funneling money from Russia into Trump’s campaign. Musk again?

alw4s 2y ago

please dont post a link that is behind a paywall !!

ksynwa 2y ago
https://archive.is/dLp6t https://archive.is/dLp6t It is a registration wall I think.
- tomrod 2y ago
  Same result. Information locks are verboten.
  - lxgr 2y ago
    As annoying as I find them, on this site they're in fact not: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989
lxgr 2y ago
Please don't complain about paywalls: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989

trolan 2y ago

https://archive.ph/dLp6t https://archive.ph/dLp6t

rsynnott 2y ago

It really feels like we’re close to the end of the current bubble now; the applications being trotted out are just increasingly absurd.

spencerchubb 2y ago

LLMs have powered products used by hundreds of millions, maybe billions. Most experiments will fail and that's okay, arguably even a good thing. Only time will tell which ones succeed

logifail 2y ago

> They want to throw LLMs at everything [..] Oh yes. I had a discussion with a manager at a client last week and was trying to run him through some (technical) issues relating to challenges an important project faces. His immediate response was that maybe we should just let ChatGPT help us decide the best option. I had to bite my tongue. OTOH, I'm more and more convinced that ChatGPT will replace managers long before it replaces technical staff.

isoprophlex 2y ago

> I knew it was bullshit from the get-go as soon as I read their definition of AI agents. That is one spicy article, it got a few laughs out of me. I must agree 100% that Langchain is an abomination, both their APIs as well as their marketing.