3 ms·

> Will anyone see the value in a chatbot that takes an hour to solve a problem? Yes! The biggest usability issue with almost all the AI products I’ve used has

> Will anyone see the value in a chatbot that takes an hour to solve a problem? Yes! The biggest usability issue with almost all the AI products I’ve used has been the termination problem, especially with fixed price offerings like ChatGPT. The LLM doesn’t actually have any time to “think” (however you might want to interpret that) except for the attention mechanism that visits each token. I want to be able to query an llm, give it access to web search, and give it a time limit so that it keeps going, consuming sources a hundred pages deep, “thinking” and writing a report for me until it’s given a signal to terminate.

jasfi 2y ago
That's an agent, which makes use of an LLM. The LLM's architecture means its trained once-off.
sottol 2y ago
But the LLM doesn't "think" - it just spits out a sequence of tokens (forever, literally). The chat models have been fine-tuned to eventually emit an "end" token (when the training examples would have been answered) that the UI/backend interprets and cuts off the LLM to stop it from generating more output. But the model would gladly go on forever and devolve into more and more obscure output the further the original query moves out of the context window.
- kelseyfrog 2y ago
  Sorry, but thinking is a social category. Debating it is pointless because it assumes the existence of social facts. Once you're aware of the existence of the meta category of social facts as reified ideas, all of them become unjustifiable. The line of argumentation is already dead.
  - BoxedEmpathy 2y ago
    If I understand currently, you're pointing out 'thinking' wasn't meant literally in your comment? Or did I miss?
    - kelseyfrog 2y ago
      I'm saying there isn't a distinction between literally and figuratively thinking or anything of the sort. "Thinking" isn't scientifically grounded, it's socially grounded, and barely so, given that it seems everyone has a differing justification for what constitutes "thinking". The variability in construing "thinking" is a product of how we each fashion our own internal sense of what it means, which if anything is evidence for the anti-realism of "thinking". Continuing, we largely fool ourselves when we play the shell game of hunting for scientific justification of "social facts". Ie: when we assume a social fact, then hunt for scientific evidence to ground it in then use the scientific evidence to justify the existence and validity of the social fact we assumed. This happens a LOT and it's epistemically invalid. A LOT of folks dismissal of thinking machines reminds me of Feuerbach's The Essence of Christianity, specifically the section on anthropomorphism[1], but in an inverted form - the reason LLMs can't "think" is that it would dilute what we hold dear about ourselves: our ability to think. It's an ego protection mechanism, but no one's jumping to admit that. https://www.gutenberg.org/files/47025/47025-h/47025-h.htm#pb17 https://www.gutenberg.org/files/47025/47025-h/47025-h.htm#pb...
      - BoxedEmpathy 2y ago
        Thank you very much for the elaboration! I think I follow better. Thinking doesn't have a rigorous and testable definition, so it isn't scientific. We all have our own colloquial understand. Since it's not testable and the definition varies person to person, it's not useful when reasoning about AI or intelligence. Also I wanted to say I love how you write!
        kelseyfrog 2y ago
        Even if it had a rigorous and testable definition it would still be suspect by virtue of us having searched for evidence of it under the guise of science. To explain it by analogy, we had the concept of women and men for at least thousands of years. Socially construed by physical appearance and behavior. Then in 1905 Nettie Stevens discovered sex chromosomes. Her discovery "found" the existence of men in women in the epistemological universe of science. But then a curious thing happened, people began placing people in social categories based on sex chromosomes. It's an incestuous loop of logic that hoists itself into validity simply because folks forget how it occurred. If we went searching for a scientifically rigorous definition of "thinking" we would forget we made the same fatal leap of logic.
  - sottol 2y ago
    That seems a little reductionist. We're just exchanging ideas and trying to interpret imprecise terms in-context (ideally in good faith). We're not really discussing whether LLMs are "thinking" in the sense of the "social category" in the abstract sense - that's just the one word that this discussion got hung up on. Concretely, we're discussing whether giving LLMs a larger time-budget would yield better results - OP suggested that letting LLMs iteratively refine or enrich an answer might lead to better results. The term "thinking" was used as an analogy or quick crutch to convey an analogy, not 100% equivalence, between a hypothetic LLM output evaluation algo and people's supposed "fast" and "slow" modes of thinking (if they even exist, whatever). Even if people don't have these two modes of thought, as you said they might be post-fact reifications of sociological ideas, the algorithm/idea of iteratively reflecting on and self-refining LLM outputs might still apply to LLMs as LLMs are not actually thinking. Imo current LLM architecture and/or training data facilitate (mostly?) the automatic non-iterative mode but OpenAI is trying to do more of the iterative, reflective mode with O1 afaict. Imo and currently, there's not much to be gained by iteratively feeding an a current-generation LLM it's own output to iteratively analyze and augment it. At least not in general. Even enriching the input data via RAG doesn't always lead to better results. Imo, there's more fundamental work necessary - and maybe OpenAI is doing that.
- bob1029 2y ago
  Anthropomorphization of the technology is my new favorite canary for when something might be going wrong.
hinkley 2y ago
Either you’re queuing the prompts to smooth out traffic variability without growing the cluster size, or Little’s Law kills you here. If queries actually take an hour on average, and my customers are starting one task per second, then I have 3600 tasks in progress at the same time. I think the main advantage to slow queries would be in discouraging users from monopolizing the system by shotgunning many requests in a short period. Which is probably one of those cost cutting measures that costs you customers. If I asked it to write me a block of code and it told me one hour, I’d just write it myself and tell my boss we aren’t really using the service.

jasfi 2y ago

That's an agent, which makes use of an LLM. The LLM's architecture means its trained once-off.

sottol 2y ago

But the LLM doesn't "think" - it just spits out a sequence of tokens (forever, literally). The chat models have been fine-tuned to eventually emit an "end" token (when the training examples would have been answered) that the UI/backend interprets and cuts off the LLM to stop it from generating more output. But the model would gladly go on forever and devolve into more and more obscure output the further the original query moves out of the context window.

kelseyfrog 2y ago
Sorry, but thinking is a social category. Debating it is pointless because it assumes the existence of social facts. Once you're aware of the existence of the meta category of social facts as reified ideas, all of them become unjustifiable. The line of argumentation is already dead.
- BoxedEmpathy 2y ago
  If I understand currently, you're pointing out 'thinking' wasn't meant literally in your comment? Or did I miss?
  - kelseyfrog 2y ago
    I'm saying there isn't a distinction between literally and figuratively thinking or anything of the sort. "Thinking" isn't scientifically grounded, it's socially grounded, and barely so, given that it seems everyone has a differing justification for what constitutes "thinking". The variability in construing "thinking" is a product of how we each fashion our own internal sense of what it means, which if anything is evidence for the anti-realism of "thinking". Continuing, we largely fool ourselves when we play the shell game of hunting for scientific justification of "social facts". Ie: when we assume a social fact, then hunt for scientific evidence to ground it in then use the scientific evidence to justify the existence and validity of the social fact we assumed. This happens a LOT and it's epistemically invalid. A LOT of folks dismissal of thinking machines reminds me of Feuerbach's The Essence of Christianity, specifically the section on anthropomorphism[1], but in an inverted form - the reason LLMs can't "think" is that it would dilute what we hold dear about ourselves: our ability to think. It's an ego protection mechanism, but no one's jumping to admit that. https://www.gutenberg.org/files/47025/47025-h/47025-h.htm#pb17 https://www.gutenberg.org/files/47025/47025-h/47025-h.htm#pb...
    - BoxedEmpathy 2y ago
      Thank you very much for the elaboration! I think I follow better. Thinking doesn't have a rigorous and testable definition, so it isn't scientific. We all have our own colloquial understand. Since it's not testable and the definition varies person to person, it's not useful when reasoning about AI or intelligence. Also I wanted to say I love how you write!
      - kelseyfrog 2y ago
        Even if it had a rigorous and testable definition it would still be suspect by virtue of us having searched for evidence of it under the guise of science. To explain it by analogy, we had the concept of women and men for at least thousands of years. Socially construed by physical appearance and behavior. Then in 1905 Nettie Stevens discovered sex chromosomes. Her discovery "found" the existence of men in women in the epistemological universe of science. But then a curious thing happened, people began placing people in social categories based on sex chromosomes. It's an incestuous loop of logic that hoists itself into validity simply because folks forget how it occurred. If we went searching for a scientifically rigorous definition of "thinking" we would forget we made the same fatal leap of logic.
- sottol 2y ago
  That seems a little reductionist. We're just exchanging ideas and trying to interpret imprecise terms in-context (ideally in good faith). We're not really discussing whether LLMs are "thinking" in the sense of the "social category" in the abstract sense - that's just the one word that this discussion got hung up on. Concretely, we're discussing whether giving LLMs a larger time-budget would yield better results - OP suggested that letting LLMs iteratively refine or enrich an answer might lead to better results. The term "thinking" was used as an analogy or quick crutch to convey an analogy, not 100% equivalence, between a hypothetic LLM output evaluation algo and people's supposed "fast" and "slow" modes of thinking (if they even exist, whatever). Even if people don't have these two modes of thought, as you said they might be post-fact reifications of sociological ideas, the algorithm/idea of iteratively reflecting on and self-refining LLM outputs might still apply to LLMs as LLMs are not actually thinking. Imo current LLM architecture and/or training data facilitate (mostly?) the automatic non-iterative mode but OpenAI is trying to do more of the iterative, reflective mode with O1 afaict. Imo and currently, there's not much to be gained by iteratively feeding an a current-generation LLM it's own output to iteratively analyze and augment it. At least not in general. Even enriching the input data via RAG doesn't always lead to better results. Imo, there's more fundamental work necessary - and maybe OpenAI is doing that.
bob1029 2y ago
Anthropomorphization of the technology is my new favorite canary for when something might be going wrong.

hinkley 2y ago

Either you’re queuing the prompts to smooth out traffic variability without growing the cluster size, or Little’s Law kills you here. If queries actually take an hour on average, and my customers are starting one task per second, then I have 3600 tasks in progress at the same time. I think the main advantage to slow queries would be in discouraging users from monopolizing the system by shotgunning many requests in a short period. Which is probably one of those cost cutting measures that costs you customers. If I asked it to write me a block of code and it told me one hour, I’d just write it myself and tell my boss we aren’t really using the service.