3 ms·

There are a great many possibilities between: - a fully deterministic machine (even if the interface and the way OpenAI let people access ChatGPT make it seem

There are a great many possibilities between: - a fully deterministic machine (even if the interface and the way OpenAI let people access ChatGPT make it seem like it's non-deterministic, there are fully deterministic models out there, who not only only respond to inputs but also always respond the exact same thing given the same inputs [query, seed, ...]), and: - god exists There could be chaos at work when human thinks. There may be interferences at play, say because whatever element that traveled trillion of kilometers just traversed our brain. While a fully deterministic machine that always respond to the same input in the same way is just that: a deterministic machine. P.S: I don't know about other LLMs like Falcon 180b but image-generation models like StableDiffusion are fully deterministic. I think a model is using a broken design and shall quickly hit limitations if it cannot be queried in a deterministic way (and its usecases are certainly limited if repeatability is not achievable). If you want different answers, use a different seed or a different query. But the same query+seed should always give the exact same output.

syntaxfree 3y ago
Also not all notions of God are dualistic — where you get to (notionally) talk to the guy using “you” and “I”. India’s Advaita Vedanta, Ibn Arabi’s version of Sufism and even some sects of Hasidism all hold that everything is in God. I haven’t found the Christianity that does this, but if it was discovered under Islam is probably thinkable here. Are these different ideas of God entirely? Yes: in India there are however many gods and Brahman says these are lesser precisely because they don’t include the whole universe.
recursivecaveat 3y ago
For the record LLM are theoretically fully deterministic, but non-deterministic in practice. First, some randomness is deliberately set via 'temperature', and second some randomness comes from things like the order of floating point operations when you divide the model on your GPU(s) without being super careful about it.
- red75prime 3y ago
  LLMs output probability distribution of the next token. And that automatically makes them non-deterministic. You can make their output deterministic by greedy sampling, fixing the seed of a pseudorandom generator, or by computing exponentially growing probability distribution of all possible continuations, but it doesn't change the fact that LLMs produce probability distributions you need to sample somehow to get a definite result.

syntaxfree 3y ago

Also not all notions of God are dualistic — where you get to (notionally) talk to the guy using “you” and “I”. India’s Advaita Vedanta, Ibn Arabi’s version of Sufism and even some sects of Hasidism all hold that everything is in God. I haven’t found the Christianity that does this, but if it was discovered under Islam is probably thinkable here. Are these different ideas of God entirely? Yes: in India there are however many gods and Brahman says these are lesser precisely because they don’t include the whole universe.

recursivecaveat 3y ago

For the record LLM are theoretically fully deterministic, but non-deterministic in practice. First, some randomness is deliberately set via 'temperature', and second some randomness comes from things like the order of floating point operations when you divide the model on your GPU(s) without being super careful about it.

red75prime 3y ago
LLMs output probability distribution of the next token. And that automatically makes them non-deterministic. You can make their output deterministic by greedy sampling, fixing the seed of a pseudorandom generator, or by computing exponentially growing probability distribution of all possible continuations, but it doesn't change the fact that LLMs produce probability distributions you need to sample somehow to get a definite result.