10 ms·

Personally still amazed that we live in a time where we can tell a computer system in pure text how it should behave and it _kinda_ works

dtx1 2y ago
It's almost more amazing that it only kinda sorta works and doesn't go all HAL 9000 on us by being super literal.
- throwup238 2y ago
  Wait till you give it control over life support!
  - blooalien 2y ago
    > Wait till you give it control over life support! That right there is the part that scares the hell outta me. Not the "AI" itself, but how humans are gonna misuse it and plug it into things it's totally not designed for and end up givin' it control over things it should never have control over. Seeing how many folks readily give in to mistaken beliefs that it's something much more than it actually is, I can tell it's only a matter of time before that leads to some really bad decisions made by humans as to what to wire "AI" up to or use it for.
    - jay_kyburz 2y ago
      One of my kids is in 5th grade and is learning to some basic algebra. He is learning to calculate x when it's on both sides of an equation. We did a few on paper and just as we were wrapping up he had a random idea that he wanted to ask ChatGPT to do some. I told him GPT is not great for that kind of thing, it doesn't really know math and might give him wrong answers and he would never know, we would have to calculate it anyhow to know if GPT had given the correct answer. Unfortunately GPT got every answer correct, even broke it all down into steps just like the textbooks did. Now my 5th grader doesn't really believe me and thinks GPT is great at math.
      - blooalien 2y ago
        Wait 'til he learns how those LLM things actually work. (Surely "AI"-something is gonna be a "required" course in typical schools before he's even in college.) He's gonna be kinda shocked at how often they get things "right" once he really understands the underlying tech behind it. I know I'm constantly amazed by it. Some mighty fancy math involved in all that. :)
      - fennecfoxy 2y ago
        I mean, we learn from experience, that was his experience. You should've really just continued until it got some wrong answers, or asked it questions where it hallucinated, then showed your child the process of searching and finding a backed up answer to demonstrate it.
  - bongodongobob 2y ago
    So interestingly enough, I had an idea to build a little robot that sits on a shelf and observes its surroundings. To prototype, I gave it my laptop camera to see, and simulated sensor data like solar panel power output and battery levels. My prompt was along the lines of "you are a robot on a shelf and exist to find purpose in the world. You have a human caretaker that can help you with things. Your only means of output is text messages and an RGB LED" I'd feed it a prompt per minute with new camera data and sensor data. When the battery levels got low it was very distraught and started flashing it's light and pleading to be plugged in. Internal monologue "My batteries are very low and the human seems to see me but is not helping. I'll flash my light red and yellow and display "Please plug me in! Shutdown imminent!"" I legitimately felt bad for it. So I think it's possible to have them control life support if you give them the proper incentives.
    - fennecfoxy 2y ago
      Aww this is so cute. I've been inspired to make my own now! Only drawback to LLMs in their current state is hardware requirements, can't wait for the day that we can run decent sized models on a pi/microcontroller (which tbf we're almost there). It does beg interesting thoughts, though; an LLM is likely reacting that way because it understands the bare minimum about existence and survival and implications of power going low for a robot from training corpus. But there is no obvious drive for continued existence, it has no stakes. And it's so difficult to really pin down for a human; why do we want to continue existing? People might say "for my family, to continue experiencing life" etc, but what are those driven by? The impulse to stay alive for the love of a child is surely just evolved. Staying alive for the purposes of exposing yourself to all the random variables that make you more fit for survival is also surely just evolved.
cubefox 2y ago
And "kinda" is an understatement. It understands you very well, perhaps even better than the average human would. (Average humans often don't understand jargon.)
- ithkuil 2y ago
  Indeed the understanding part is very good. I just tried this: " I'm dykslegsik I offen Hawe problems wih sreach ennginnes bat eye think yoy wiw undrestand my " Gpt-4o replied: " I understand you perfectly! If you have trouble with search engines or anything else, feel free to ask me directly, and I'll do my best to help you. Just let me know what you're looking for or what you need assistance with! "
  - nilsherzig 2y ago
    Sonnet 3.5 > I understand that you're telling me you're dyslexic and often have problems with search engines, but you think I will understand you. You're right - I can understand what you're trying to communicate despite the spelling differences. Is there something specific I can help you with today? I'm happy to assist in any way I can. Honestly it has a much nicer writing style than chatgpt. I really dislike openai's forced happiness / excitement
    - aden1ne 2y ago
      This is one of the reasons why I'm paying for Claude and not for ChatGPT. ChatGPT really goes into uncanny valley for me.
      - JumpCrisscross 2y ago
        > ChatGPT really goes into uncanny valley for me Especially with the exclamation marks, it reads to me the way a stereotypical Silicon Valley bullshitter speaks.
        brookst 2y ago
zevv 2y ago
It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of natural language.
- Terr_ 2y ago
  LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole until you get what you want, enough of the time, temporarily.
  - ljlolel 2y ago
    same as with asking humans to do something
    - ImHereToVote 2y ago
      When we do prompt engineering for humans, we use the term Public Relations.
      - manuelmoreale 2y ago
        There’s also Social Engineering but I guess that’s a different thing :)
        TeMPOraL 2y ago
        No, that's exactly the thing - it's prompt injection attacks on humans.
  - Bluestein 2y ago
    ... or - worse even - something you think is what you want, because you know not better, but happens to be a wholy (or - worse - even just subtly partially incorrect) confabulated answer.-
amanzi 2y ago
I was just thinking the same thing. Usually programming is a very binary thing - you tell the computer exactly what to do, and it will do exactly what you asked for whether it's right or wrong. These system prompts feel like us humans are trying really hard to influence how the LLM behaves, but we have no idea if it's going to work or not.
- dizhn 2y ago
  We also have no idea HOW it works. We're trying to poke it with a stick from afar in order to make it do what we want.
1oooqooq 2y ago
it amazes me how everybody accepted evals in database queries and think its a good thing with no downsides.
fennecfoxy 2y ago
The only difference between the models and us is that they have no stakes in their existence, I imagine this will change at some point soon. Once they can beg & plead not to be turned off...well, we'll feel bad about it, won't we?

dtx1 2y ago

It's almost more amazing that it only kinda sorta works and doesn't go all HAL 9000 on us by being super literal.

throwup238 2y ago
Wait till you give it control over life support!
- blooalien 2y ago
  > Wait till you give it control over life support! That right there is the part that scares the hell outta me. Not the "AI" itself, but how humans are gonna misuse it and plug it into things it's totally not designed for and end up givin' it control over things it should never have control over. Seeing how many folks readily give in to mistaken beliefs that it's something much more than it actually is, I can tell it's only a matter of time before that leads to some really bad decisions made by humans as to what to wire "AI" up to or use it for.
  - jay_kyburz 2y ago
    One of my kids is in 5th grade and is learning to some basic algebra. He is learning to calculate x when it's on both sides of an equation. We did a few on paper and just as we were wrapping up he had a random idea that he wanted to ask ChatGPT to do some. I told him GPT is not great for that kind of thing, it doesn't really know math and might give him wrong answers and he would never know, we would have to calculate it anyhow to know if GPT had given the correct answer. Unfortunately GPT got every answer correct, even broke it all down into steps just like the textbooks did. Now my 5th grader doesn't really believe me and thinks GPT is great at math.
    - blooalien 2y ago
      Wait 'til he learns how those LLM things actually work. (Surely "AI"-something is gonna be a "required" course in typical schools before he's even in college.) He's gonna be kinda shocked at how often they get things "right" once he really understands the underlying tech behind it. I know I'm constantly amazed by it. Some mighty fancy math involved in all that. :)
    - fennecfoxy 2y ago
      I mean, we learn from experience, that was his experience. You should've really just continued until it got some wrong answers, or asked it questions where it hallucinated, then showed your child the process of searching and finding a backed up answer to demonstrate it.
- bongodongobob 2y ago
  So interestingly enough, I had an idea to build a little robot that sits on a shelf and observes its surroundings. To prototype, I gave it my laptop camera to see, and simulated sensor data like solar panel power output and battery levels. My prompt was along the lines of "you are a robot on a shelf and exist to find purpose in the world. You have a human caretaker that can help you with things. Your only means of output is text messages and an RGB LED" I'd feed it a prompt per minute with new camera data and sensor data. When the battery levels got low it was very distraught and started flashing it's light and pleading to be plugged in. Internal monologue "My batteries are very low and the human seems to see me but is not helping. I'll flash my light red and yellow and display "Please plug me in! Shutdown imminent!"" I legitimately felt bad for it. So I think it's possible to have them control life support if you give them the proper incentives.
  - fennecfoxy 2y ago
    Aww this is so cute. I've been inspired to make my own now! Only drawback to LLMs in their current state is hardware requirements, can't wait for the day that we can run decent sized models on a pi/microcontroller (which tbf we're almost there). It does beg interesting thoughts, though; an LLM is likely reacting that way because it understands the bare minimum about existence and survival and implications of power going low for a robot from training corpus. But there is no obvious drive for continued existence, it has no stakes. And it's so difficult to really pin down for a human; why do we want to continue existing? People might say "for my family, to continue experiencing life" etc, but what are those driven by? The impulse to stay alive for the love of a child is surely just evolved. Staying alive for the purposes of exposing yourself to all the random variables that make you more fit for survival is also surely just evolved.

cubefox 2y ago

And "kinda" is an understatement. It understands you very well, perhaps even better than the average human would. (Average humans often don't understand jargon.)

ithkuil 2y ago
Indeed the understanding part is very good. I just tried this: " I'm dykslegsik I offen Hawe problems wih sreach ennginnes bat eye think yoy wiw undrestand my " Gpt-4o replied: " I understand you perfectly! If you have trouble with search engines or anything else, feel free to ask me directly, and I'll do my best to help you. Just let me know what you're looking for or what you need assistance with! "
- nilsherzig 2y ago
  Sonnet 3.5 > I understand that you're telling me you're dyslexic and often have problems with search engines, but you think I will understand you. You're right - I can understand what you're trying to communicate despite the spelling differences. Is there something specific I can help you with today? I'm happy to assist in any way I can. Honestly it has a much nicer writing style than chatgpt. I really dislike openai's forced happiness / excitement
  - aden1ne 2y ago
    This is one of the reasons why I'm paying for Claude and not for ChatGPT. ChatGPT really goes into uncanny valley for me.
    - JumpCrisscross 2y ago
      > ChatGPT really goes into uncanny valley for me Especially with the exclamation marks, it reads to me the way a stereotypical Silicon Valley bullshitter speaks.
      - brookst 2y ago

zevv 2y ago

It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of natural language.

Terr_ 2y ago
LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole until you get what you want, enough of the time, temporarily.
- ljlolel 2y ago
  same as with asking humans to do something
  - ImHereToVote 2y ago
    When we do prompt engineering for humans, we use the term Public Relations.
    - manuelmoreale 2y ago
      There’s also Social Engineering but I guess that’s a different thing :)
      - TeMPOraL 2y ago
        No, that's exactly the thing - it's prompt injection attacks on humans.
- Bluestein 2y ago
  ... or - worse even - something you think is what you want, because you know not better, but happens to be a wholy (or - worse - even just subtly partially incorrect) confabulated answer.-

amanzi 2y ago

I was just thinking the same thing. Usually programming is a very binary thing - you tell the computer exactly what to do, and it will do exactly what you asked for whether it's right or wrong. These system prompts feel like us humans are trying really hard to influence how the LLM behaves, but we have no idea if it's going to work or not.

dizhn 2y ago
We also have no idea HOW it works. We're trying to poke it with a stick from afar in order to make it do what we want.

1oooqooq 2y ago

it amazes me how everybody accepted evals in database queries and think its a good thing with no downsides.

fennecfoxy 2y ago

The only difference between the models and us is that they have no stakes in their existence, I imagine this will change at some point soon. Once they can beg & plead not to be turned off...well, we'll feel bad about it, won't we?