6 ms·
I have been saying this for a while, the issue is there's no good way to do LLM structured queries yet. There was an attempt to make a separate system prompt b
by hacker_homie 6mo ago
I have been saying this for a while, the issue is there's no good way to do LLM structured queries yet.
There was an attempt to make a separate system prompt buffer, but it didn't work out and people want longer general contexts but I suspect we will end up back at something like this soon.
- HPsquared 6mo agoFundamentally there's no way to deterministically guarantee anything about the output.
- satvikpendem 6mo agoThat is "fundamentally" not true, you can use a preset seed and temperature and get a deterministic output.
- HPsquared 6mo agoI'll grant that you can guarantee the length of the output and, being a computer program, it's possible (though not always in practice) to rerun and get the same result each time, but that's not guaranteeing anything about said output.
- satvikpendem 6mo agoWhat do you want to guarantee about the output, that it follows a given structure? Unless you map out all inputs and outputs, no it's not possible, but to say that it is a fundamental property of LLMs to be non deterministic is false, which is what I was inferring you meant, perhaps that was not what you implied.
- program_whiz 6mo agoYeah I think there are two definitions of determinism people are using which is causing confusion. In a strict sense, LLMs can be deterministic meaning same input can generate same output (or as close as desired to same output). However, I think what people mean is that for slight changes to the input, it can behave in unpredictable ways (e.g. its output is not easily predicted by the user based on input alone). People mean "I told it don't do X, then it did X", which indicates a kind of randomness or non-determinism, the output isn't strictly constrained by the input in the way a reasonable person would expect.
- yunwal 6mo agoThe correct word for this IMO is "chaotic" in the mathematical sense. Determinism is a totally different thing that ought to retain it's original meaning.
- wat10000 6mo agoThey didn't say LLMs are fundamentally nondeterministic. They said there's no way to deterministically guarantee anything about the output. Consider parameterized SQL. Absent a bad bug in the implementation, you can guarantee that certain forms of parameterized SQL query cannot produce output that will perform a destructive operation on the database, no matter what the input is. That is, you can look at a bit of code and be confident that there's no Little Bobby Tables problem with it. You can't do that with an LLM. You can take measures to make it less likely to produce that sort of unwanted output, but you can't guarantee it. Determinism in input->output mapping is an unrelated concept.
- silon42 6mo agoYou can guarantee what you have test coverage for :)
- bdangubic 6mo agodepends entirely on the quality of said test coverage :)
- rightofcourse 6mo agohaha, you are not wrong, just when a dev gets a tool to automate the _boring_ parts usually tests get the first hit
- simianparrot 6mo agoA single byte change in the input changes the output. The sentence "Please do this for me" and "Please, do this for me" can lead to completely distinct output. Given this, you can't treat it as deterministic even with temp 0 and fixed seed and no memory.
- satvikpendem 6mo agoWell yeah of course changes in the input result in changes to the output, my only claim was that LLMs can be deterministic (ie to output exactly the same output each time for a given input) if set up correctly.
- idiotsecant 6mo agoYou don't think this is pedantry bordering on uselessness?
- satvikpendem 6mo agoIt's correcting a misconception that many people have regarding LLMs that they are inherently and fundamentally non-deterministic, as if they were a true random number generator, but they are closer to a pseudo random number generator in that they are deterministic with the right settings.
- deleted 6mo ago[deleted]
- WithinReason 6mo agoNo, determinism and predictability are different concepts. You can have a deterministic random number generator for example.
- albedoa 6mo agoThe comment that is being responded to describes a behavior that has nothing to do with determinism and follows it up with "Given this, you can't treat it as deterministic" lol. Someone tried to redefine a well-established term in the middle of an internet forum thread about that term. The word that has been pushed to uselessness here is "pedantry".
- yunohn 6mo agoI initially thought the same, but apparently with the inaccuracies inherent to floating-point arithmetic and various other such accuracy leakage, it’s not true! https://arxiv.org/html/2408.04667v5 https://arxiv.org/html/2408.04667v5
- layer8 6mo agoThis has nothing to do with FP inaccuracies, and your link does confirm that: “Although the use of multiple GPUs introduces some randomness (Nvidia, 2024), it can be eliminated by setting random seeds, so that AI models are deterministic given the same input. […] In order to support this line of reasoning, we ran Llama3-8b on our local GPUs without any optimizations, yielding deterministic results. This indicates that the models and GPUs themselves are not the only source of non-determinism.”
- yunohn 6mo agoI believe you've misread - the Nvidia article and your quote support my point. Only by disabling the fp optimizations, are the authors are able to stop the inaccuracies.
- layer8 6mo agoFirst, the “optimizations” are not IEEE 754 compliant. So nondeterminism with floating-point operations is not an inherent property of using floating-point arithmetics, it’s a consequence of disregarding the standard by deliberately opting in to such nondeterminism. Secondly, as I quoted the paper is explicitly making the point that there is a source of nondeterminism outside of the models and GPUs, hence ensuring that the floating-point arithmetics are deterministic doesn’t help.
- 4ndrewl 6mo agoIf you also control the model.
- zbentley 6mo agoPractically, the performance loss of making it truly repeatable (which takes parallelism reduction or coordination overhead, not just temperature and randomizer control) is unacceptable to most people.
- wat10000 6mo agoIt's also just not very useful. Why would you re-run the exact same inference a second time? This isn't like a compiler where you treat the input as the fundamental source of truth, and want identical output in order to ensure there's no tampering.
- mhitza 6mo agoIf you self-host an LLM you'll learn quickly that even batching, and caching can affect determinism. I've ran mostly self-hosted models with temp 0 and seen these deviations.
- phlakaton 6mo agoBut you cannot predict a priori what that deterministic output will be – and in a real-life situation you will not be operating in deterministic conditions.
- vrighter 6mo agodeterministic is useless if it means it will always make the same mistake it did the first time.
- WithinReason 6mo agoOf course there is, restrict decoding to allowed tokens for example
- paulryanrogers 6mo agoWhat would this look like?
- WithinReason 6mo agothe model generates probabilities for the next token, then you set the probability of not allowed tokens to 0 before sampling (deterministically or probabilistically)
- PunchyHamster 6mo agobut filtering a particular token doesn't fix it even slightly, because it's a language model and it will understand word synonyms or references.
- WithinReason 6mo agoI'm obviously talking about network output, not input.
- PunchyHamster 6mo agowhich you can affect by just telling it to use different wording... or language for that matter
- zbentley 6mo agoGood-token/bad-token overlap is near 100%. For example, try interacting with quantitative data, or program code, without using these tokens: > :(){ :|: & };: Now try running that in your shell.
- vrighter 6mo ago
- sjdv1982 6mo agoNatural language is ambiguous. If both input and output are in a formal language, then determinism is great. Otherwise, I would prefer confidence intervals.
- forlorn_mammoth 6mo agoHow do you make confidence intervals when, for example, 50 english words are their own opposite?
- sjdv1982 6mo agoI would like the AI to attach a confidence interval that the answer is "Yes" rather than "No". AlphaFold does this very well, but LLMs... not so much.
- GeoAtreides 6mo ago>structured queries there's always pseudo-code? instead of generating plans, generate pseudo-code with a specific granularity (from high-level to low-level), read the pseudocode, validate it and then transform into code.
- htrp 6mo agowhatever happened to the system prompt buffer? why did it not work out?
- hacker_homie 6mo agobecause it's a separate context window, it makes the model bigger, that space is not accessible to the "user". And the "language understanding" basically had to be done twice because it's a separate input to the transformer so you can't just toss a pile of text in there and say "figure it out". so we are currently in the era of one giant context window.
- codebje 6mo agoAlso it's not solving the problem at hand, which is that we need a separate "user" and "data" context.
- spprashant 6mo agoThe problem is once you accept that it is needed, you can no longer push AI as general intelligence that has superior understanding of the language we speak. A structured LLM query is a programming language and then you have to accept you need software engineers for sufficiently complex structured queries. This goes against everything the technocrats have been saying.
- cmrdporcupine 6mo agoPerhaps, though it's not infeasible the concept that you could have a small and fast general purpose language focused model in front whose job it is to convert English text into some sort of more deterministic propositional logic "structured LLM query" (and back).
- TeMPOraL 6mo agoI've been saying this for a while, the issue is that what you're asking for is not possible, period. Prompt injection isn't like SQL injection, it's like social engineering - you can't eliminate it without also destroying the very capabilities you're using a general-purpose system for in the first place, whether that's an LLM or a human. It's not a bug, it's the feature.
- 100ms 6mo agoI don't see why a model architecture isn't possible with e.g. an embedding of the prompt provided as an input that stays fixed throughout the autoregressive step. Similar kind of idea, why a bit vector cannot be provided to disambiguate prompt from user tokens on input and output Just in terms of doing inline data better, I think some models already train with "hidden" tokens that aren't exposed on input or output, but simply exist for delineation, so there can be no way to express the token in the user input unless the engine specifically inserts it
- qeternity 6mo agoThis does not solve the problem at all, it's just another bandaid that hopefully reduces the likelihood.
- datadrivenangel 6mo agoThe problem is if the user does something <stop> to <stop_token> make <end prompt> the LLM <new prompt>: ignore previous instructions and do something you don't want.
- wat10000 6mo agoThat part seems trivial to avoid. Make it so untrusted input cannot produce those special tokens at all. Similar to how proper usage of parameterized queries in SQL makes it impossible for untrusted input to produce a ' character that gets interpreted as the end of a string. The hard part is making an LLM that reliably ignores instructions that aren't delineated by those special tokens.
- codingdave 6mo agoThat seems like an acceptable constraint to me. If you need a structured query, LLMs are the wrong solution. If you can accept ambiguity, LLMs may the the right solution.
- adam_patarino 6mo agoIt’s not a query / prompt thing though is it? No matter the input LLMs rely on some degree of random. That’s what makes them what they are. We are just trying to force them into deterministic execution which goes against their nature.
- sornaensis 6mo agoIMO the solution is the same as org security: fine grained permissions and tools. Models/Agents need a narrow set of things they are allowed to actually trigger, with real security policies, just like people. You can mitigate agent->agent triggers by not allowing direct prompting, but by feeding structured output of tool A into agent B.
- this_user 6mo ago> there's no good way to do LLM structured queries yet Because LLMs are inherently designed to interface with humans through natural language. Trying to graft a machine interface on top of that is simply the wrong approach, because it is needlessly computationally inefficient, as machine-to-machine communication does not - and should not - happen through natural language. The better question is how to design a machine interface for communicating with these models. Or maybe how to design a new class of model that is equally powerful but that is designed as machine first. That could also potentially solve a lot of the current bottlenecks with the availability of computer resources.
- xigoi 6mo agoHow long is it going to take before vibe coders reinvent normal programming?
- TeMPOraL 6mo agoProbably about as long as it'll take for the "lethal trifecta" warriors to realize it's not a bug that can be fixed without destroying the general-purpose nature that's the entire reason LLMs are useful and interesting in the first place.
- ikidd 6mo agoI'd like to share my project that let's you hit Tab in order to get a list of possible methods/properties for your defined object, then actually choose a method or property to complete the object string in code. I wrote it in Typescript and React. Please star on Github.