6 ms·
The question posed sounds like "why should we have deterministic behavior if we can have non-deterministic behavior instead?" Am I wrong to think that the answ
by sunaurus 11mo ago
The question posed sounds like "why should we have deterministic behavior if we can have non-deterministic behavior instead?"
Am I wrong to think that the answer is obvious? I mean, who wants web apps to behave differently every time you interact with them?
- admax88qqq 11mo agoWeb apps kind of already do that with most companies shipping constant UX redesigns, A/B tests, new features, etc. For a typical user today’s software isn’t particularly deterministic. Auto updates mean your software is constantly changing under you.
- jeltz 11mo agoAnd most end users hate it.
- Jaygles 11mo agoI don't think that is what the original commenter was getting at. In your case, the company is actively choosing to make changes. Whether its for a good reason, or leads to a good outcome, is beside the point. LLMs being inherently non-deterministic means using this technology as the foundation of your UI will mean your UI is also non-deterministic. The changes that stem from that are NOT from any active participation of the authors/providers. This opens a can of worms where there will always be a potential for the LLM to spit out extremely undesirable changes without anyone knowing. Maybe your bank app one day doesn't let you access your money. This is a danger inherent and fundamental to LLMs.
- admax88qqq 11mo agoRight I get tha. The point I’m making is that from a users perspective it’s functionally very similar. A non deterministic llm or a non deterministic company full of designers and engineers.
- lazide 11mo agoRegardless of what changes the bank makes, it’s not going to let you access someone else’s money. This llm very well might.
- array_key_first 11mo agoWell, software has been known to have vulnerabilities... Consider this: the bank teller is non-deterministic, too. They could give you 500 dollars of someone else's money. But they don't, generally.
- an_guy 11mo agoBank tellers are deterministic though. They have a set protocol for each cases and escalate unknown cases to a more deterministic point of contact. It will be difficult to incorporate relative access or restrictions to features with respect to users current/known state or actions. Might as well write the entire web app at that point.
- array_key_first 11mo agoI think the bank teller's systems and processes are deterministic, but the teller itself is not. They could even rob the bank, if they wanted to. They could shoot the customers. They don't, generally, but they can. I think, if we can efficiently capture a way to "make" LLMs conform to a set of processes, you can cut out the app and just let the LLM do it. I don't think this makes any sense for maybe the next decade, but perhaps at some point it will. And, in such time, software engineering will no longer exist.
- yencabulator 11mo agoThe actual app is the set of processes.
- paulhebert 11mo agoThe rate of change is so different it seems absurd to compare the two in that way. The LLM example gives you a completely different UI on _every_ page load. That’s very different from companies moving around buttons occasionally and rarely doing full redesigns
- samrolken 11mo agoNo, I wouldn’t say that my hypothesis is that non-deterministic behavior is good. It’s an undesirable side effect and illustrates the gap we have between now and the coming post-code world.
- killingtime74 11mo agoAI wouldn't be intelligent though if it was deterministic. It would just be information retrieval
- thih9 11mo ago> who wants web apps to behave differently every time you interact with them? Technically everyone, we stopped using static pages a while ago. Imagine pages that can now show you e.g. infinitely customizable UI; or, more likely, extremely personalized ads.
- ehutch79 11mo agoNo. When I go to the dmv website to renew my license, I want it to renew my license every single time
- hansmayer 11mo ago> infinitely customizable UI; or, more likely, extremely personalized ads Yeah, NO.
- anthk 11mo agoAh, sure; that's why everyone got Adblock and UBo in first place. Even more under phones.
- ozim 11mo agoSmall anecdote. We were releasing UI changes every 2 weeks making app better more user friendly etc. Product owners were happy. Until users came for us with pitchforks as they didn’t want stuff to change constantly. We backed out to releasing on monthly cadence.
- myhf 11mo agoDesigning a system with deterministic behavior would require the developer to think. Human-Computer Interaction experts agree that a better policy is to "Don't Make Me Think" [1] [1] https://en.wikipedia.org/wiki/Don%27t_Make_Me_Think https://en.wikipedia.org/wiki/Don%27t_Make_Me_Think
- krapp 11mo agoThat book is talking about user interaction and application design, not development. We absolutely should want developers to think.
- crabmusket 11mo agoAs experiments like TFA become more common, the argument will shift to whether anybody should think about anything at all.
- krapp 11mo agoWhat argument? I see a business model here, not an argument.
- crabmusket 11mo agoI meant "the discourse", "the conversation we are all having", interpreting the experiment in TFA as an entry in that discourse.
- AstroBen 11mo ago..is this an AI comment?
- _se 11mo agoThis is such a massive misunderstanding of the book. Have you even read it? The developer needs to think so that the user doesn't have to...
- 11mo ago
- jstummbillig 11mo agoBecause nobody actually wants a "web app". People want food, love, sex or: solutions. You or your coworker are not a web app. You can do some of the things that web apps can, and many things that a web app can't, but neither is because of the modality. Coded determinism is hard for many problems and I find it entirely plausible that it could turn out to be the wrong approach in software, that is designed to solve some level of complex problems more generally. Average humans are pretty great at solving a certain class of complex problems that we tried to tackle unsuccessfully with many millions lines of deterministic code, or simply have not had a handle on at all, like (like build a great software CEO).
- 113 11mo ago> Because nobody actually wants a "web app". People want food, love, sex or: solutions. Okay but when I start my car I want to drive it, not fuck it.
- hinkley 11mo agoChristine didn’t end well for anyone.
- lazide 11mo agoEven if it purred real nice when it started up? (I’m sorry)
- ozim 11mo agoLooks like we have a Civic owner xD
- jstummbillig 11mo agoMost of us actually drive a car to get somewhere. The car, and the driving, are just a modality. Which is the point.
- 113 11mo agoYeah but in this case your car is non-deterministic so
- ddalex 11mo agoLike, for sure you can ask the AI to save it's "settings" or "context" to a local file in a format of its own choosing, and then bring that back in the next prompt ; couple this with temperature 0 and you should get to a fixed-point deterministic app immediately
- dehsge 11mo agoThere still maybe some variance at temperature 0. The outputted code could still have errors. LLMs are still bounded by the undecidable problems in computational theory like Rices theorem.
- geraneum 11mo ago> couple this with temperature 0 Not quite the case. Temperature 0 is not the same as random seed. Also there are downsides to lowering temperature (always choosing the most probable next token).
- guelo 11mo agoWhy wouldn't the llm codify that "context" into code so it doesn't have to rethink through it over and over? Just like humans would. Imagine if you were manually operating a website and every time a request came in you had come up with sql queries (without remembering how you did it last time) and manually type the responses. You wouldn't last long before you started automating.
- reissbaker 11mo agoI think it's actually conceptually pretty different. LLMs today are usually constrained to: 1. Outputting text (or, sometimes, images). 2. No long term storage except, rarely, closed-source "memory" implementations that just paste stuff into context without much user or LLM control. This is a really neat glimpse of a future where LLMs can have much richer output and storage. I don't think this is interesting because you can recreate existing apps without coding... But I think it's really interesting as a view of a future with much richer, app-like responses from LLMs, and richer interactions — e.g. rather than needing to format everything as a question, the LLM could generate links that you click on to drill into more information on a subject, which end up querying the LLM itself! And similarly it can ad-hoc manage databases for memory+storage, etc etc.
- pepoluan 11mo agoOr, maybe, just not use LLMs? LLM is just one model used in A.I. It's not a panacea. For generating deterministic output, probably a combination of Neural Networks and Genetic Programming will be better. And probably also much more efficient, energy-wise.
- vidarh 11mo agoFor some things you absolutely want deterministic behaviour. For other things, behaviour that adapts to usage and the context provided by the data the user provides sounds like it could potentially be very exciting. I'm glade people are exploring this. The hard part will be figuring out where the line goes, and when and how to "freeze" certain behaviours that the user seems happy with vs. continuing to adapt to data.
- visarga 11mo agoEvery time you need a rarely used functionality it might be better to wait 60s for an LLM with MCP tools to do its work than to update an app. It only makes sense to optimize and maintain app functionalities when they are reused.
- anon291 11mo agoLlms are easily made deterministic by choosing the selection strategy. More than being deterministic they are also fully analayzable and you don't run into issues like the halting problem if you constrain the output appropriately.
- SecretDreams 11mo agoWhy do good thing consistently when we can do great thing that only works sometimes??? :(