4 ms·

Reading this, I realize that the "prompt" is what's lacking, in terms of the output produced. A writer has a prompt too -- which is created through the entirety

Reading this, I realize that the "prompt" is what's lacking, in terms of the output produced. A writer has a prompt too -- which is created through the entirety of their experience leading up to the moment they sit down to write, often heavily weighted towards the hours leading up to that moment. AI will catch up to actual human writers when it can spend a day's visual, auditory, and linguistic stimuli crafting the prompt that leads to moment of creation.

garba_dlm 3y ago
it's not a prompt, it's a dialog, it's many prompts.
- FrustratedMonky 3y ago
  Yes When I have a conversation with GPT4, it keeps the history, and the entire history is a prompt for the next prompt. I would say, not totally unlike a human memory. just GPT4 is limited to 8000 characters or something. Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.
  - garba_dlm 3y ago
    I think that human memory "keeps" the next prompt by baking it into the parameters of the "neural net" (the brain) immediately this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations. I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.
    - jprete 3y ago
      This is rather pseudoscientific.
      - FrustratedMonky 3y ago
        99% of conversations on Internet are pseudoscientific. Think here we are just 'conceptualizing', 'spit-balling', 'blue-sky-discussion', 'wondering out loud', 'hypothesizing'. etc...
  - inciampati 3y ago
    But we do manage to remember stuff that happened "quadrillions" of tokens (of sensory input) back. Sure, we are wrong a lot, but we have a context that goes way back. A LLM has a fruit fly's version of what we work with, although in some versions it has perfect recall in that context.
    - seanthemon 3y ago
      Or it can be said that we boil down our experience to a few millions of tokens, only a set of experiences, emotions and often wrong/made up memories. Your comment assumes a human can recall all (potentially incorrect) memories and use those to make some judgement, but fact of the matter is we don't.
      - FrustratedMonky 3y ago
        Yeah, Humans do ton of aggregation and filtering. Thus you can't remember individual days of commuting, because it is so similar, the brain just lets it go. I'd think something like this would be added to the models eventually. Of course, over-simplifying.
m3kw9 3y ago
More like a constant chain of prompts, except the prompts are exponentially more data and each output is combined with more sensory inputs for another inference and they happen in real time.
inciampati 3y ago
Beautifully said. I can add only that the right pattern of prompts may drive the creation of legitimately useful and even beautiful work. Humans need to guide the machines, maybe precisely because they lack the evolutionary drive that gets us up in the morning and keeps us cleaning our babies' behinds.

garba_dlm 3y ago

it's not a prompt, it's a dialog, it's many prompts.

FrustratedMonky 3y ago
Yes When I have a conversation with GPT4, it keeps the history, and the entire history is a prompt for the next prompt. I would say, not totally unlike a human memory. just GPT4 is limited to 8000 characters or something. Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.
- garba_dlm 3y ago
  I think that human memory "keeps" the next prompt by baking it into the parameters of the "neural net" (the brain) immediately this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations. I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.
  - jprete 3y ago
    This is rather pseudoscientific.
    - FrustratedMonky 3y ago
      99% of conversations on Internet are pseudoscientific. Think here we are just 'conceptualizing', 'spit-balling', 'blue-sky-discussion', 'wondering out loud', 'hypothesizing'. etc...
- inciampati 3y ago
  But we do manage to remember stuff that happened "quadrillions" of tokens (of sensory input) back. Sure, we are wrong a lot, but we have a context that goes way back. A LLM has a fruit fly's version of what we work with, although in some versions it has perfect recall in that context.
  - seanthemon 3y ago
    Or it can be said that we boil down our experience to a few millions of tokens, only a set of experiences, emotions and often wrong/made up memories. Your comment assumes a human can recall all (potentially incorrect) memories and use those to make some judgement, but fact of the matter is we don't.
    - FrustratedMonky 3y ago
      Yeah, Humans do ton of aggregation and filtering. Thus you can't remember individual days of commuting, because it is so similar, the brain just lets it go. I'd think something like this would be added to the models eventually. Of course, over-simplifying.

m3kw9 3y ago

More like a constant chain of prompts, except the prompts are exponentially more data and each output is combined with more sensory inputs for another inference and they happen in real time.

inciampati 3y ago

Beautifully said. I can add only that the right pattern of prompts may drive the creation of legitimately useful and even beautiful work. Humans need to guide the machines, maybe precisely because they lack the evolutionary drive that gets us up in the morning and keeps us cleaning our babies' behinds.