8 ms·

Building AI products in the probabilistic era

192 points by sdan 1y ago

AIorNot 1y ago
From the article: “We have a class of products with deterministic cost and stochastic outputs: a built-in unresolved tension. Users insert the coin with certainty, but will be uncertain of whether they'll get back what they expect. This fundamental mismatch between deterministic mental models and probabilistic reality produces frustration — a gap the industry hasn't yet learned to bridge.” And all the news today around AI being a bubble - We’re still learning what we can do with these models and how to evaluate them but industry and capitalism forces our hand into building sellable products rapidly
- deleted 1y ago
  [deleted]
- CGMthrowaway 1y ago
  It's like putting money into a (potentially) rigged slot machine
  - ACCount37 1y ago
    It's like paying a human to do something. Anyone who thinks humans are reliable must have never met one.
- voidhorse 1y ago
  Precisely. There is nothing inherently wrong with LLMs and "agent" systems. There are certain classes of problems that they might be great for solving. The problem is that the tech industry has devolved into a late-capitalist clown market supported on pure wild speculation and absurd "everything machine" marketing. This not only leads to active damage (see people falling into delusional spirals thanks to chat bots) but also inhibits us from figuring out what the actual good uses are and investing into the right applications. Radical technology leads to behavior change. Smart phones led to behavior change, but you didn't have to beg people to buy them. LLMs are leading to behavior change but only because they are being forcibly shoved into everyone's faces and people are co-gaslighting each other into the collective hysteria that to not participate is to miss out on something big, but they can never articulate what that big thing actually is.
failiaf 1y ago
(unrelated) what's the font used for the cursive in the article? the heading is ibm plex serif and the content dm mono, but the cursive font is simply labeled as dm mono which isn't accurate
- nbbaier 1y ago
  Seems to be Dank Mono Regular Italic: https://philpl.gumroad.com/l/dank-mono https://philpl.gumroad.com/l/dank-mono
  - failiaf 1y ago
    oh! i mistook 'dm' to be 'dm mono', but this appears to be correct
- leutersp 1y ago
  Chrome Dev console shows that the italics font is indeed named "dm" just like the rest of the content. It is not really a cursive, only a few letters are stylized ("f", "s" and "l"). It is possible (and often desirable) to use different WOFF fonts for italics, and they can look quite different from the standard font.
hodgehog11 1y ago
This seems like an expression of The Bitter Lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson... https://news.ycombinator.com/item?id=19393432 https://news.ycombinator.com/item?id=19393432
- greymalik 1y ago
  How so?
  - chaos_emergent 1y ago
    Author advocates for building general purpose systems that can accomplish goals within some causal boundary given relevant constraints, versus highly deterministic logical flows that are created from priors like intuition or user research. Parallel with the bitter lesson being that general purpose algorithms that use search and learning leveraged by increasing computational capacity tend to beat out specialized methods that exploit intuition about how cognitive processes work.
- adidoit 1y ago
  No it isn't...the author is talking about products and building with a completely different mindset to deterministic software. The bitter lesson is about model level performance improvements and the futility of scaffolding in the face of search and scaling.
  - hodgehog11 1y ago
    It isn't clear to me why these are so different? The alternative mindset to deterministic software is to use probabilistic models. The common mentality is that deterministic software takes developer knowledge into account. This becomes less effective in the big data era, that's the bitter lesson, and that's why the shift is taking place. I'm not saying that the article is the bitter lesson restated, but I'm saying that this is a realisation of that lesson.
    - 1y ago
pdhborges 1y ago
I will believe this theory if someone shows me that the ratio of scientists to engineers of leading teams of the leading companies deploying AI products is bigger than 1.
- layer8 1y ago
  I don’t think the dichotomy between scientists and engineers that’s being established here is making much sense in the first place. Applied science is applied science.
mentalgear 1y ago
> After decades of technical innovation, the world has (rightfully) developed some anti-bodies to tech hype. Mainstream audiences have become naturally skeptical of big claims of “the world is changing”. Well, it took about 3 years of non-stop AI hype from the industry and press (and constant ignoring of actual experts) until finally the perception seems to have shifted in recognising it as another bubble. So I wouldn't say any lessons were learned. Get ready for the next bubble when the crypto grifters that moved to "AI" will soon move on the to the NEXT-BIG-THING!
- lacy_tinpot 1y ago
  Is it really "hype" if like 100s of millions of people are using llms on a daily basis?
  - pmg101 1y ago
    The dot-com bubble burst but I'm betting you visited at least one of those "websites" they were hyping today.
  - PaulRobinson 1y ago
    It’s not the usage that’s the problem. It’s the valuations.
  - nutjob2 1y ago
    Absolutely, it's just that some are just kidding themself as to what it can do now and in the future.
- ivape 1y ago
  ”… finally the perception seems to have shifted in recognising it as another bubble” Who recognized this exactly? The MIT article? Give me a break. NVDA was $90 this year, was that the world recognizing AI was a bubble? No one is privy to anything when it comes to this. Everyone is just going to get blindsided again and again every time they sleep on this stuff.
- ACCount37 1y ago
  There's been non-stop talk of "AI bubble" for 3 years now. Frontier AI systems keep improving in the meanwhile. Clearly, a lot of people very desperately want AI tech to fail. And if there is such a strong demand for "tell me that AI tech will fail", then there will be hacks willing to supply. I trust most of those "experts" as far as I can throw them.
therobots927 1y ago
This is pure sophistry and the use of formal mathematical notation just adds insult to injury here: “Think about it: we’ve built a special kind of function F' that for all we know can now accept anything — compose poetry, translate messages, even debug code! — and we expect it to always reply with something reasonable.” This forms the axiom from which the rest of this article builds its case. At each step further fuzzy reasoning is used. Take this for example: “Can we solve hallucination? Well, we could train perfect systems to always try to reply correctly, but some questions simply don't have "correct" answers. What even is the "correct" when the question is "should I leave him?".” Yes of course relationship questions don’t have a “correct” answer. But physics questions do. Code vulnerability questions do. Math questions do. I mean seriously? The most disturbing part of my tech career has been witnessing the ability that many highly intelligent and accomplished people have to apparently fool themselves with faulty yet complex reasoning. The fact that this article is written in defense of chatbots that ALSO have complex and flawed reasoning just drives home my point. We’re throwing away determinism just like that? I’m not saying future computing won’t be probabilistic but to say that LLMs are probabilistic, so they are the future of computing can only be said by someone with an incredibly strong prior on LLMs. I’d recommend Baudrillards work on hyperreality. This AI conversation could not be a better example of the loss of meaning. I hope this dark age doesn’t last as long as the last one. I mean just read this conclusion: “It's ontologically different. We're moving away from deterministic mechanicism, a world of perfect information and perfect knowledge, and walking into one made of emergent unknown behaviors, where instead of planning and engineering we observe and hypothesize.” I don’t actually think the above paragraph makes any sense, does anyone disagree with me? “Instead of planning we observe and hypothesize”? That’s called the scientific method. Which is a PRECURSOR to planning and engineering. That’s how we built the technology we have today. I’ll stop now because I need to keep my blood pressure low.
- falcor84 1y ago
  > Yes of course relationship questions don’t have a “correct” answer. But physics questions do. Code vulnerability questions do. Math questions do. I mean seriously? But as per Gödel's incompleteness theorem and the Halting Problem, math questions (and consequently physics and CS questions) don't always have an answer.
ankit219 1y ago
Building with non deterministic systems isnt new. It does not take a scientist. Though people who have experience with these systems are fewer in number today. You saw the same thing with TCP/IP development where we ended up developing systems that assumed the randomness and made sure that isnt passed on to the next layer. For every game, given the latency involved in previous networks, there is no way on the network games were deterministic.
- golergka 1y ago
  Isn't any kind of human in the loop make system non-deterministic?
ipdashc 1y ago
While this article is a little overenthusiastic for my taste, I think I agree with the general idea of it - and it's always kind of been my pet peeve when it comes to ML. It's a little depressing to think that's probably where the industry is heading. Does anyone feel the same way? A lot of the stuff the author says resonates deeply, but like, the whole deterministism thing is why I liked programming and computers in the first place. They are complicated but simple; they run on straightforward, man-made rules. As the article says: > Any good engineer will know how the Internet works: we designed it! We know how packets of data move around, we know how bytes behave, even in uncertain environments like faulty connections. I've always loved this aspect of it. We humans built the entire system, from protocols down to transistors (and the electronics/physics is so abstracted away it doesn't matter). If one wants to understand or tweak some aspect of it, with enough documentation or reverse engineering, there is nothing stopping you. Everything makes sense. The author is spot on; every time I've worked with ML it feels more like you're supposed to be a scientist than an engineer, running trials and collecting statistics and tweaking the black box until it works. And I hate that. Props to those who can handle real fields like biology or chemistry, right, but I never wanted to be involved with that kind of stuff. But it seems like that's the direction we're inevitably going.
- ACCount37 1y ago
  ML doesn't work like programming because it's not programming. It just happens to run on the same computational substrate. Modern ML is at this hellish intersection of underexplored math, twisted neurobiology and applied demon summoning. An engineer works with known laws of nature - but the laws of machine learning are still being written. You have to be at least a little bit of a scientist to navigate this landscape. Unfortunately, the nature of intelligence doesn't seem to yield itself to simple, straightforward, human-understandable systems. But machine intelligence is desirable. So we're building AIs anyway.
  - voidhorse 1y ago
    You should read some of the papers written in the 1940s and learn about the history of cybernetics. Your glowing perception of the "demon summoning" nature of ML might change a bit. People want others to think this tech is mysterious. It's not. We've known the theory of these systems since the mid 1900s, we just didn't fully work out the resource arrangements to make them tractable until recently. Yes, there are some unknowns and the end product is a black box insofar as you cannot simply inspect source code, but this description of the situation is pure fantasy.
deleted 1y ago
[deleted]
novok 1y ago
What exactly was rewritten in 3 weeks at replit? Literally everything? The agent part?
bithive123 1y ago
It became evident to me while playing with Stable Diffusion that it's basically a slot machine. A skinner box with a variable reinforcement schedule. Harmless enough if you are just making images for fun. But probably not an ideal workflow for real work.
- A4ET8a8uTh0_v2 1y ago
  << But probably not an ideal workflow for real work. Hmm. Ideal is rarely an option, so I have to assume you are being careful about phrasing. Still, despite it being a black box, one can still tip the odds on one's favor, so the real question is what is considered 'real work'? I personally would define that as whatever they you are being paid to do. If that premise is accepted, then the tool is not the issue, despite its obvious handicaps.
- diggan 1y ago
  > It became evident to me while playing with Stable Diffusion that it's basically a slot machine. It can be, and usually is by default. If you set the seeds to deterministic numbers, and everything else remains the same, you'll get deterministic output. A slot machine implies you keep putting in the same thing and get random good/bad outcomes, that's not really true for Stable Diffusion.
  - bithive123 1y ago
    Strictly speaking, yes, but there is so much variability introduced by prompting that even keeping the seed value static doesn't change the "slot machine" feeling, IMHO. While prompting is something one can get better at, you're still just rolling the dice and waiting to see whether the output is delightful or dismaying.
    - diggan 1y ago
      > IMHO. While prompting is something one can get better at, you're still just rolling the dice and waiting to see whether the output is delightful or dismaying. You yourself acknowledge someone can better than another on getting good results from Stable Diffusion, how is that in any way similar to slot machine or rolling the dice? The point of those analogies is precisely that it doesn't matter what skill/knowledge you have, you'll get a random outcome. The same is very much not true for Stable Diffusion usage, something you seem to know yourself too.
techlatest_net 1y ago
[dead]
patrickscoleman 1y ago
Great read. We've been seeing some wild emergent behavior at Rime (tts voice ai) too, e.g. training the model to <laugh> and it being able to <sigh>.
thorum 1y ago
I like this framing, but I don’t think it’s entirely new to LLMs. Humans have been building flexible, multi-purpose tools and using them for things the original inventor or manufacturer didn’t think of since before the invention of the wheel. It’s in our DNA. Our brains have been shaped by a world where that is normal. The rigidness and near-perfect reliability of computer software is the unusual thing in human history, an outlier we’ve gotten used to.
- therobots927 1y ago
  “The rigidness and near-perfect reliability of computer software is the unusual thing in human history, an outlier we’ve gotten used to.” Ordered approximately by recency: Banking? Clocks? Roman aqueducts? Mayan calendars? The sun rising every day? Predictable rainy and dry season? How is software the outlier here?
  - thorum 1y ago
    My point was more “humans are used to tools that don’t always work and can be used in creative ways” than “no human invention has ever been rigid and reliable”. People on HN regularly claim that LLMs are useless if they aren’t 100% accurate all the time. I don’t think this is true. We work around that kind of thing every day. With your examples: - Before computers, fraud and human error was common in the banking system. We designed a system that was resilient against this and mostly worked, most of the time, well enough for most purposes even though it was built on an imperfect foundation. - Highly precise clocks are a recent invention. For regular people 200 years ago, one person’s clock would often be 5-10 minutes off from someone else’s. People managed to get things done anyway. I’ll grant you that Roman aqueducts, seasons and the sun are much more reliable than computers (as are all the laws of nature).
    - deleted 1y ago
      [deleted]
    - skydhash 1y ago
      Isn’t technology the invention of more reliable and precise tools?
ath3nd 1y ago
> Dismissal is a common reaction when witnessing AI’s rate of progress. People struggle to reconcile their world model with what AI can now do, and how. By probabilistic pattern matching. Next. > Every model update doesn’t necessarily mean a complete rewrite every time, but it does force you to fundamentally rethink your assumptions each time, making a rewrite a perfectly plausible hypothesis. Translation: our crap is unreliable and will introduce regressions that you have to manage. It can regress so much that anything you built on this has to be redone. Next. > I strongly believe that when it comes to AI, something is happening. This time it does feel different. Said the guy working for the company that is selling you this stuff. Next.
jmogly 1y ago
Right now you have this awesome new dynamic capability that doesn’t mesh with how we are used to building software; well defined, constrained, correct. Software products are close ended. Imagine if they weren’t. Imagine if you were playing an open world game like Skyrim or runescape and new areas were created as you explored, new weapons, entirely new game mechanics spontaneously arose as you played. Or imagine an intent based business analytics app, that had a view on your company’s database and when a user wanted a report or a visual it generated it on the fly. We’re only limited by our imagination here.
- dingnuts 1y ago
  and by reality. the tech is nowhere near ready for this
- thrown-0825 1y ago
  imagine whatever you like, its your fantasy in the real world we expect consistent behavior from engineered systems, and building everything on top of a gpu powered bullshit engine isn't going to give you that its a nice narrative though, maybe you should pick up writing sci-fi
  - jmogly 1y ago
    Yeah today we do … what if engineered systems were more like humans, where we trade consistency for more interesting capabilities. Not to say there won’t still be traditional engineering for stable behavior systems, but these tools do have the ability to add a human touch to our software that wasn’t possible before. I’ll stick to software.
    - rafterydj 1y ago
      That's all well and good to ask as a what if, but in terms of practical applications, the vast majority of the time you want to trade the other way around whenever possible - you want your system to work reliably. That's the cornerstone of any project that wants to be useful for others.
mehulashah 1y ago
I tend to agree with most of what has been said. The only difference is that some workflows need to be deterministic otherwise the penalty for failure is high. In that case, AI is helpful in searching through a space, but something orthogonal needs to verify its output.
camillomiller 1y ago
It seems to me that probabilistic approaches are more akin to magical AI thinking right now, so defending that as the new paradigm sounds quite egregious and reeks of (maybe involuntary?) inevitabilism. Even if the assumption is correct, forcing a probabilistic system on a strongly deterministic society won't end well. Maybe for society, but mostly for the companies drumming up their probabilistic systems and their investors. Also, anyone who wants to make money probabilistically is better off going to the casino. Baccarat is a good one. European Roulette also has a better house margin than chatGPT's error margin.
- ath3nd 1y ago
  > It seems to me that probabilistic approaches are more akin to magical AI thinking right now, so defending that as the new paradigm sounds quite egregious and reeks of (maybe involuntary?) inevitabilism Thank you for saying that! I read it as: "our product is fickle and unreliable and you have to get used to it and love it because we tell you that is the future". But it's not the future, it's just one of many possible futures, and not one I and a large part of society wants to be a part of. These "leaders" are talking and talking but they are just salemen, trying to frame what they are seling you as good or inevitable. It's not. Look for example, at the Ex CEO of Github and his clownish statements: - 2nd of August: Developers, either embrace AI or leave the industry https://www.businessinsider.com/github-ceo-developers-embrace-ai-or-get-out-2025-8 https://www.businessinsider.com/github-ceo-developers-embrac... - 11th of August: Resigns. https://www.techradar.com/pro/github-ceo-resigns-is-this-the-latest-sign-of-its-microsoft-absorption https://www.techradar.com/pro/github-ceo-resigns-is-this-the... Tell me this is not pitiful, tell me this is the person I gotta believe in and who knows what's the future of tech? Tell me I gotta believe Sama when he tells me for the 10th time that AGI is nearly there when his latest features were "study mode", "announcing OpenAI office suite" and ChatGPT5 (aka ChatGPT4.01). Or Musk and his full self driving cars which he promised since 2019? The guy who bought Twitter, tanked its value in half so he can win the election for Trump and then got himself kicked out of the government? The guy making Nazi salutes? Are those the guys telling what's the future and why are we even listening to them?
  - camillomiller 1y ago
thrown-0825 1y ago
this entire endeavor is a fools errand, and any who has used coding agents for anything more complex than a web tut knows it. it doesn't matter how much jargon and mathematical notation you layer on top of your black box next token generator, it will still be unreliable and inconsistent because fundamentally the output is an approximation of an answer and has no basis in reality This is not a limitation you can build around, its a basic limitation of the underlying models. Bonus points if you are relying on an LLM for orchestration or agentic state, its not going to work, just move on to a problem you can actually solve.
- eru 1y ago
  You could more or less use the same reasoning to argue for why humans can't write software. And you'd be half-right: humans are extremely unreliable, and it takes a lot of safeguards and automated testing and PR reviews etc to get reliable software out of humans. (Just to be clear, I agree that current models aren't exactly reliable. But I'm fairly sure with enough resources thrown at the problem, we could get reasonably reliable systems out of them.)
  - skydhash 1y ago
    There’s a lot of projects with only one and two people behind it and they produce widely used software. Still have to see big tech producing something actually useful with one of those agents they’re touting about.
d4rkn0d3z 1y ago
In physics, the change from classical to quantum theory was a change from determinism to probabilistic determinism. There is not one physicist on earth that would ever exhort you to use quantum theory where classical theory will do. Furthermore, when you study physics you must learn the classical theory first or you will be hopelessly lost, just like the author of this article. The central analogy of the article is entirely bogus. This article does not rise to the level of being wrong.
- ares623 1y ago
  > This article does not rise to the level of being wrong. Amazing
- hearsathought 1y ago
  > In physics, the change from classical to quantum theory was a change from determinism to probabilistic determinism. Don't you mean from determinism to nondeterminism? > There is not one physicist on earth that would ever exhort you to use quantum theory where classical theory will do. That's being practical.
  - d4rkn0d3z 1y ago
    > "Don't you mean from determinism to nondeterminism?" No, I mean exactly what I said. Given a system's state, one evolves the state using wave equation du jour, nondeterminism does not occur.
meindnoch 1y ago
What a bunch of pretentious nonsense. It is always a red flag when an author tries to shoehorn mathematical notation into an article that has nothing mathematical about it whatsoever. Gives off "igon value problem"-vibes.
ath3nd 1y ago
Dealing with pretentious pseudoscientific blogs in the LLM era. /s
weego 1y ago
'AI businesses just aren't like anything before them'. The entire article is a business pseudo-philosophy word salad equivalent of tiktok 'I'm not like other (men/women)' posts.
drunx 1y ago
This article can be taken a little less seriously. It is just an opinion/experience of a person. I think he has a lot of good points there. Yes he treads on the thin ice with the very big statements and generalisations, but those we don't have to "sign under with blood". I do like the simple formula concept. It does make sense. It's not an ultimate representation of everything, but it's a nice idea of how to frame the differences between the logics we are dealing with. I choose to not commit the whole message of the article to my core beliefs, but I'll borrow thoughts, ideas for the debates and work ahead.
olddustytrail 1y ago
I vibe like there is a real point trying to escape this article but it sounds like a long form LinkedIn post. Ask ChatGPT to fix it. ps. On a side note, I love that vibe has come back as a word. Feels like the 60s.
selinkocalar 1y ago
This hits on something we think about constantly at Delve. The key is building systems that are transparent about their confidence levels and gracefully handle edge cases. The companies that will win in AI aren't the ones with perfect algorithms - they're the ones who design for human understanding and real-world messiness.

192 points by sdan 1y ago

From the article: “We have a class of products with deterministic cost and stochastic outputs: a built-in unresolved tension. Users insert the coin with certainty, but will be uncertain of whether they'll get back what they expect. This fundamental mismatch between deterministic mental models and probabilistic reality produces frustration — a gap the industry hasn't yet learned to bridge.” And all the news today around AI being a bubble - We’re still learning what we can do with these models and how to evaluate them but industry and capitalism forces our hand into building sellable products rapidly

deleted 1y ago
[deleted]
CGMthrowaway 1y ago
It's like putting money into a (potentially) rigged slot machine
- ACCount37 1y ago
  It's like paying a human to do something. Anyone who thinks humans are reliable must have never met one.
voidhorse 1y ago
Precisely. There is nothing inherently wrong with LLMs and "agent" systems. There are certain classes of problems that they might be great for solving. The problem is that the tech industry has devolved into a late-capitalist clown market supported on pure wild speculation and absurd "everything machine" marketing. This not only leads to active damage (see people falling into delusional spirals thanks to chat bots) but also inhibits us from figuring out what the actual good uses are and investing into the right applications. Radical technology leads to behavior change. Smart phones led to behavior change, but you didn't have to beg people to buy them. LLMs are leading to behavior change but only because they are being forcibly shoved into everyone's faces and people are co-gaslighting each other into the collective hysteria that to not participate is to miss out on something big, but they can never articulate what that big thing actually is.

failiaf 1y ago

(unrelated) what's the font used for the cursive in the article? the heading is ibm plex serif and the content dm mono, but the cursive font is simply labeled as dm mono which isn't accurate

nbbaier 1y ago
Seems to be Dank Mono Regular Italic: https://philpl.gumroad.com/l/dank-mono https://philpl.gumroad.com/l/dank-mono
- failiaf 1y ago
  oh! i mistook 'dm' to be 'dm mono', but this appears to be correct
leutersp 1y ago
Chrome Dev console shows that the italics font is indeed named "dm" just like the rest of the content. It is not really a cursive, only a few letters are stylized ("f", "s" and "l"). It is possible (and often desirable) to use different WOFF fonts for italics, and they can look quite different from the standard font.

hodgehog11 1y ago

This seems like an expression of The Bitter Lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson... https://news.ycombinator.com/item?id=19393432 https://news.ycombinator.com/item?id=19393432

greymalik 1y ago
How so?
- chaos_emergent 1y ago
  Author advocates for building general purpose systems that can accomplish goals within some causal boundary given relevant constraints, versus highly deterministic logical flows that are created from priors like intuition or user research. Parallel with the bitter lesson being that general purpose algorithms that use search and learning leveraged by increasing computational capacity tend to beat out specialized methods that exploit intuition about how cognitive processes work.
adidoit 1y ago
No it isn't...the author is talking about products and building with a completely different mindset to deterministic software. The bitter lesson is about model level performance improvements and the futility of scaffolding in the face of search and scaling.
- hodgehog11 1y ago
  It isn't clear to me why these are so different? The alternative mindset to deterministic software is to use probabilistic models. The common mentality is that deterministic software takes developer knowledge into account. This becomes less effective in the big data era, that's the bitter lesson, and that's why the shift is taking place. I'm not saying that the article is the bitter lesson restated, but I'm saying that this is a realisation of that lesson.
  - 1y ago

pdhborges 1y ago

I will believe this theory if someone shows me that the ratio of scientists to engineers of leading teams of the leading companies deploying AI products is bigger than 1.

layer8 1y ago
I don’t think the dichotomy between scientists and engineers that’s being established here is making much sense in the first place. Applied science is applied science.

mentalgear 1y ago

> After decades of technical innovation, the world has (rightfully) developed some anti-bodies to tech hype. Mainstream audiences have become naturally skeptical of big claims of “the world is changing”. Well, it took about 3 years of non-stop AI hype from the industry and press (and constant ignoring of actual experts) until finally the perception seems to have shifted in recognising it as another bubble. So I wouldn't say any lessons were learned. Get ready for the next bubble when the crypto grifters that moved to "AI" will soon move on the to the NEXT-BIG-THING!

lacy_tinpot 1y ago
Is it really "hype" if like 100s of millions of people are using llms on a daily basis?
- pmg101 1y ago
  The dot-com bubble burst but I'm betting you visited at least one of those "websites" they were hyping today.
- PaulRobinson 1y ago
  It’s not the usage that’s the problem. It’s the valuations.
- nutjob2 1y ago
  Absolutely, it's just that some are just kidding themself as to what it can do now and in the future.
ivape 1y ago
”… finally the perception seems to have shifted in recognising it as another bubble” Who recognized this exactly? The MIT article? Give me a break. NVDA was $90 this year, was that the world recognizing AI was a bubble? No one is privy to anything when it comes to this. Everyone is just going to get blindsided again and again every time they sleep on this stuff.
ACCount37 1y ago
There's been non-stop talk of "AI bubble" for 3 years now. Frontier AI systems keep improving in the meanwhile. Clearly, a lot of people very desperately want AI tech to fail. And if there is such a strong demand for "tell me that AI tech will fail", then there will be hacks willing to supply. I trust most of those "experts" as far as I can throw them.

therobots927 1y ago

This is pure sophistry and the use of formal mathematical notation just adds insult to injury here: “Think about it: we’ve built a special kind of function F' that for all we know can now accept anything — compose poetry, translate messages, even debug code! — and we expect it to always reply with something reasonable.” This forms the axiom from which the rest of this article builds its case. At each step further fuzzy reasoning is used. Take this for example: “Can we solve hallucination? Well, we could train perfect systems to always try to reply correctly, but some questions simply don't have "correct" answers. What even is the "correct" when the question is "should I leave him?".” Yes of course relationship questions don’t have a “correct” answer. But physics questions do. Code vulnerability questions do. Math questions do. I mean seriously? The most disturbing part of my tech career has been witnessing the ability that many highly intelligent and accomplished people have to apparently fool themselves with faulty yet complex reasoning. The fact that this article is written in defense of chatbots that ALSO have complex and flawed reasoning just drives home my point. We’re throwing away determinism just like that? I’m not saying future computing won’t be probabilistic but to say that LLMs are probabilistic, so they are the future of computing can only be said by someone with an incredibly strong prior on LLMs. I’d recommend Baudrillards work on hyperreality. This AI conversation could not be a better example of the loss of meaning. I hope this dark age doesn’t last as long as the last one. I mean just read this conclusion: “It's ontologically different. We're moving away from deterministic mechanicism, a world of perfect information and perfect knowledge, and walking into one made of emergent unknown behaviors, where instead of planning and engineering we observe and hypothesize.” I don’t actually think the above paragraph makes any sense, does anyone disagree with me? “Instead of planning we observe and hypothesize”? That’s called the scientific method. Which is a PRECURSOR to planning and engineering. That’s how we built the technology we have today. I’ll stop now because I need to keep my blood pressure low.

falcor84 1y ago
> Yes of course relationship questions don’t have a “correct” answer. But physics questions do. Code vulnerability questions do. Math questions do. I mean seriously? But as per Gödel's incompleteness theorem and the Halting Problem, math questions (and consequently physics and CS questions) don't always have an answer.

ankit219 1y ago

Building with non deterministic systems isnt new. It does not take a scientist. Though people who have experience with these systems are fewer in number today. You saw the same thing with TCP/IP development where we ended up developing systems that assumed the randomness and made sure that isnt passed on to the next layer. For every game, given the latency involved in previous networks, there is no way on the network games were deterministic.

golergka 1y ago
Isn't any kind of human in the loop make system non-deterministic?

ipdashc 1y ago

While this article is a little overenthusiastic for my taste, I think I agree with the general idea of it - and it's always kind of been my pet peeve when it comes to ML. It's a little depressing to think that's probably where the industry is heading. Does anyone feel the same way? A lot of the stuff the author says resonates deeply, but like, the whole deterministism thing is why I liked programming and computers in the first place. They are complicated but simple; they run on straightforward, man-made rules. As the article says: > Any good engineer will know how the Internet works: we designed it! We know how packets of data move around, we know how bytes behave, even in uncertain environments like faulty connections. I've always loved this aspect of it. We humans built the entire system, from protocols down to transistors (and the electronics/physics is so abstracted away it doesn't matter). If one wants to understand or tweak some aspect of it, with enough documentation or reverse engineering, there is nothing stopping you. Everything makes sense. The author is spot on; every time I've worked with ML it feels more like you're supposed to be a scientist than an engineer, running trials and collecting statistics and tweaking the black box until it works. And I hate that. Props to those who can handle real fields like biology or chemistry, right, but I never wanted to be involved with that kind of stuff. But it seems like that's the direction we're inevitably going.

ACCount37 1y ago
ML doesn't work like programming because it's not programming. It just happens to run on the same computational substrate. Modern ML is at this hellish intersection of underexplored math, twisted neurobiology and applied demon summoning. An engineer works with known laws of nature - but the laws of machine learning are still being written. You have to be at least a little bit of a scientist to navigate this landscape. Unfortunately, the nature of intelligence doesn't seem to yield itself to simple, straightforward, human-understandable systems. But machine intelligence is desirable. So we're building AIs anyway.
- voidhorse 1y ago
  You should read some of the papers written in the 1940s and learn about the history of cybernetics. Your glowing perception of the "demon summoning" nature of ML might change a bit. People want others to think this tech is mysterious. It's not. We've known the theory of these systems since the mid 1900s, we just didn't fully work out the resource arrangements to make them tractable until recently. Yes, there are some unknowns and the end product is a black box insofar as you cannot simply inspect source code, but this description of the situation is pure fantasy.

deleted 1y ago

[deleted]

novok 1y ago

What exactly was rewritten in 3 weeks at replit? Literally everything? The agent part?

bithive123 1y ago

It became evident to me while playing with Stable Diffusion that it's basically a slot machine. A skinner box with a variable reinforcement schedule. Harmless enough if you are just making images for fun. But probably not an ideal workflow for real work.

A4ET8a8uTh0_v2 1y ago
<< But probably not an ideal workflow for real work. Hmm. Ideal is rarely an option, so I have to assume you are being careful about phrasing. Still, despite it being a black box, one can still tip the odds on one's favor, so the real question is what is considered 'real work'? I personally would define that as whatever they you are being paid to do. If that premise is accepted, then the tool is not the issue, despite its obvious handicaps.
diggan 1y ago
> It became evident to me while playing with Stable Diffusion that it's basically a slot machine. It can be, and usually is by default. If you set the seeds to deterministic numbers, and everything else remains the same, you'll get deterministic output. A slot machine implies you keep putting in the same thing and get random good/bad outcomes, that's not really true for Stable Diffusion.
- bithive123 1y ago
  Strictly speaking, yes, but there is so much variability introduced by prompting that even keeping the seed value static doesn't change the "slot machine" feeling, IMHO. While prompting is something one can get better at, you're still just rolling the dice and waiting to see whether the output is delightful or dismaying.
  - diggan 1y ago
    > IMHO. While prompting is something one can get better at, you're still just rolling the dice and waiting to see whether the output is delightful or dismaying. You yourself acknowledge someone can better than another on getting good results from Stable Diffusion, how is that in any way similar to slot machine or rolling the dice? The point of those analogies is precisely that it doesn't matter what skill/knowledge you have, you'll get a random outcome. The same is very much not true for Stable Diffusion usage, something you seem to know yourself too.

techlatest_net 1y ago

[dead]

patrickscoleman 1y ago

Great read. We've been seeing some wild emergent behavior at Rime (tts voice ai) too, e.g. training the model to <laugh> and it being able to <sigh>.

thorum 1y ago

I like this framing, but I don’t think it’s entirely new to LLMs. Humans have been building flexible, multi-purpose tools and using them for things the original inventor or manufacturer didn’t think of since before the invention of the wheel. It’s in our DNA. Our brains have been shaped by a world where that is normal. The rigidness and near-perfect reliability of computer software is the unusual thing in human history, an outlier we’ve gotten used to.

therobots927 1y ago
“The rigidness and near-perfect reliability of computer software is the unusual thing in human history, an outlier we’ve gotten used to.” Ordered approximately by recency: Banking? Clocks? Roman aqueducts? Mayan calendars? The sun rising every day? Predictable rainy and dry season? How is software the outlier here?
- thorum 1y ago
  My point was more “humans are used to tools that don’t always work and can be used in creative ways” than “no human invention has ever been rigid and reliable”. People on HN regularly claim that LLMs are useless if they aren’t 100% accurate all the time. I don’t think this is true. We work around that kind of thing every day. With your examples: - Before computers, fraud and human error was common in the banking system. We designed a system that was resilient against this and mostly worked, most of the time, well enough for most purposes even though it was built on an imperfect foundation. - Highly precise clocks are a recent invention. For regular people 200 years ago, one person’s clock would often be 5-10 minutes off from someone else’s. People managed to get things done anyway. I’ll grant you that Roman aqueducts, seasons and the sun are much more reliable than computers (as are all the laws of nature).
  - deleted 1y ago
    [deleted]
  - skydhash 1y ago
    Isn’t technology the invention of more reliable and precise tools?

ath3nd 1y ago

> Dismissal is a common reaction when witnessing AI’s rate of progress. People struggle to reconcile their world model with what AI can now do, and how. By probabilistic pattern matching. Next. > Every model update doesn’t necessarily mean a complete rewrite every time, but it does force you to fundamentally rethink your assumptions each time, making a rewrite a perfectly plausible hypothesis. Translation: our crap is unreliable and will introduce regressions that you have to manage. It can regress so much that anything you built on this has to be redone. Next. > I strongly believe that when it comes to AI, something is happening. This time it does feel different. Said the guy working for the company that is selling you this stuff. Next.

jmogly 1y ago

Right now you have this awesome new dynamic capability that doesn’t mesh with how we are used to building software; well defined, constrained, correct. Software products are close ended. Imagine if they weren’t. Imagine if you were playing an open world game like Skyrim or runescape and new areas were created as you explored, new weapons, entirely new game mechanics spontaneously arose as you played. Or imagine an intent based business analytics app, that had a view on your company’s database and when a user wanted a report or a visual it generated it on the fly. We’re only limited by our imagination here.

dingnuts 1y ago
and by reality. the tech is nowhere near ready for this
thrown-0825 1y ago
imagine whatever you like, its your fantasy in the real world we expect consistent behavior from engineered systems, and building everything on top of a gpu powered bullshit engine isn't going to give you that its a nice narrative though, maybe you should pick up writing sci-fi
- jmogly 1y ago
  Yeah today we do … what if engineered systems were more like humans, where we trade consistency for more interesting capabilities. Not to say there won’t still be traditional engineering for stable behavior systems, but these tools do have the ability to add a human touch to our software that wasn’t possible before. I’ll stick to software.
  - rafterydj 1y ago
    That's all well and good to ask as a what if, but in terms of practical applications, the vast majority of the time you want to trade the other way around whenever possible - you want your system to work reliably. That's the cornerstone of any project that wants to be useful for others.

mehulashah 1y ago

I tend to agree with most of what has been said. The only difference is that some workflows need to be deterministic otherwise the penalty for failure is high. In that case, AI is helpful in searching through a space, but something orthogonal needs to verify its output.

camillomiller 1y ago

It seems to me that probabilistic approaches are more akin to magical AI thinking right now, so defending that as the new paradigm sounds quite egregious and reeks of (maybe involuntary?) inevitabilism. Even if the assumption is correct, forcing a probabilistic system on a strongly deterministic society won't end well. Maybe for society, but mostly for the companies drumming up their probabilistic systems and their investors. Also, anyone who wants to make money probabilistically is better off going to the casino. Baccarat is a good one. European Roulette also has a better house margin than chatGPT's error margin.

ath3nd 1y ago
> It seems to me that probabilistic approaches are more akin to magical AI thinking right now, so defending that as the new paradigm sounds quite egregious and reeks of (maybe involuntary?) inevitabilism Thank you for saying that! I read it as: "our product is fickle and unreliable and you have to get used to it and love it because we tell you that is the future". But it's not the future, it's just one of many possible futures, and not one I and a large part of society wants to be a part of. These "leaders" are talking and talking but they are just salemen, trying to frame what they are seling you as good or inevitable. It's not. Look for example, at the Ex CEO of Github and his clownish statements: - 2nd of August: Developers, either embrace AI or leave the industry https://www.businessinsider.com/github-ceo-developers-embrace-ai-or-get-out-2025-8 https://www.businessinsider.com/github-ceo-developers-embrac... - 11th of August: Resigns. https://www.techradar.com/pro/github-ceo-resigns-is-this-the-latest-sign-of-its-microsoft-absorption https://www.techradar.com/pro/github-ceo-resigns-is-this-the... Tell me this is not pitiful, tell me this is the person I gotta believe in and who knows what's the future of tech? Tell me I gotta believe Sama when he tells me for the 10th time that AGI is nearly there when his latest features were "study mode", "announcing OpenAI office suite" and ChatGPT5 (aka ChatGPT4.01). Or Musk and his full self driving cars which he promised since 2019? The guy who bought Twitter, tanked its value in half so he can win the election for Trump and then got himself kicked out of the government? The guy making Nazi salutes? Are those the guys telling what's the future and why are we even listening to them?
- camillomiller 1y ago

thrown-0825 1y ago

this entire endeavor is a fools errand, and any who has used coding agents for anything more complex than a web tut knows it. it doesn't matter how much jargon and mathematical notation you layer on top of your black box next token generator, it will still be unreliable and inconsistent because fundamentally the output is an approximation of an answer and has no basis in reality This is not a limitation you can build around, its a basic limitation of the underlying models. Bonus points if you are relying on an LLM for orchestration or agentic state, its not going to work, just move on to a problem you can actually solve.

eru 1y ago
You could more or less use the same reasoning to argue for why humans can't write software. And you'd be half-right: humans are extremely unreliable, and it takes a lot of safeguards and automated testing and PR reviews etc to get reliable software out of humans. (Just to be clear, I agree that current models aren't exactly reliable. But I'm fairly sure with enough resources thrown at the problem, we could get reasonably reliable systems out of them.)
- skydhash 1y ago
  There’s a lot of projects with only one and two people behind it and they produce widely used software. Still have to see big tech producing something actually useful with one of those agents they’re touting about.

d4rkn0d3z 1y ago

In physics, the change from classical to quantum theory was a change from determinism to probabilistic determinism. There is not one physicist on earth that would ever exhort you to use quantum theory where classical theory will do. Furthermore, when you study physics you must learn the classical theory first or you will be hopelessly lost, just like the author of this article. The central analogy of the article is entirely bogus. This article does not rise to the level of being wrong.

ares623 1y ago
> This article does not rise to the level of being wrong. Amazing
hearsathought 1y ago
> In physics, the change from classical to quantum theory was a change from determinism to probabilistic determinism. Don't you mean from determinism to nondeterminism? > There is not one physicist on earth that would ever exhort you to use quantum theory where classical theory will do. That's being practical.
- d4rkn0d3z 1y ago
  > "Don't you mean from determinism to nondeterminism?" No, I mean exactly what I said. Given a system's state, one evolves the state using wave equation du jour, nondeterminism does not occur.

meindnoch 1y ago

What a bunch of pretentious nonsense. It is always a red flag when an author tries to shoehorn mathematical notation into an article that has nothing mathematical about it whatsoever. Gives off "igon value problem"-vibes.

ath3nd 1y ago

Dealing with pretentious pseudoscientific blogs in the LLM era. /s

weego 1y ago

'AI businesses just aren't like anything before them'. The entire article is a business pseudo-philosophy word salad equivalent of tiktok 'I'm not like other (men/women)' posts.

drunx 1y ago

This article can be taken a little less seriously. It is just an opinion/experience of a person. I think he has a lot of good points there. Yes he treads on the thin ice with the very big statements and generalisations, but those we don't have to "sign under with blood". I do like the simple formula concept. It does make sense. It's not an ultimate representation of everything, but it's a nice idea of how to frame the differences between the logics we are dealing with. I choose to not commit the whole message of the article to my core beliefs, but I'll borrow thoughts, ideas for the debates and work ahead.

olddustytrail 1y ago

I vibe like there is a real point trying to escape this article but it sounds like a long form LinkedIn post. Ask ChatGPT to fix it. ps. On a side note, I love that vibe has come back as a word. Feels like the 60s.

selinkocalar 1y ago

This hits on something we think about constantly at Delve. The key is building systems that are transparent about their confidence levels and gracefully handle edge cases. The companies that will win in AI aren't the ones with perfect algorithms - they're the ones who design for human understanding and real-world messiness.