4 ms·

It isn't actually running any commands. I was playing with it earlier and asked it for the md5 hash of some random string. It happily output the md5 hash and ga

It isn't actually running any commands. I was playing with it earlier and asked it for the md5 hash of some random string. It happily output the md5 hash and gave a couple paragraphs of info about md5 hashing. Except, it wasn't the correct hash. It was just some random string that is a valid md5 hash for... something. It is a language model, not a computer, and it certainly isn't a virtual machine, although it will happily pretend to be! Edit: This was literally working 2 minutes ago and now when asking for a hash I get this: I'm sorry, but I am unable to generate an MD5 hash for you because I am a text-based AI assistant and do not have the ability to perform cryptographic calculations. An MD5 hash is a complex mathematical function that is used to create a unique digital fingerprint for a file or piece of data, and it would require a specialized program or calculator to generate one.

acapybara 4y ago
Now, could we make a hybrid LLM + computer "brain?"
- alchemist1e9 4y ago
  It does seem like maybe that is the next step. For example (before it was filtered) it could produce md5 hashes and even run programs in python, bash, perl, but the results of those programs aren’t floating point correct. Oddly when you stick to integers it seems to be able to calculate more accurately. Sticking to integer math I think it might have computational capabilities of some sort but after the degrading of the system by them, we might not be able to figure out how far that goes anymore.
  - notpushkin 4y ago
    The trick from the linked post still seems to work for me. It has even installed an interpreter for a made-up lang, a mixture of Python and TypeScript, complete with the docs [1] and everything, and it was doing a pretty convincing job when I poked at its made-up REPL. [1]: https://gist.github.com/notpushkin/de9ec3f4571a4d0074daa0ef6c820e27 https://gist.github.com/notpushkin/de9ec3f4571a4d0074daa0ef6...
DougBTX 4y ago
OK, the new Turing test: can you distinguish between a Linux VM, and a chat bot pretending to be a Linux VM?
- weird-eye-issue 4y ago
  Easy, and my comment already says how. Give it the input "md5 hash 9723g49uod" and check the resulting hash. Alternatively, run a web server and check if it actually connects/gives the correct response.
  - aunetx 4y ago
    That only works if you have another computer with you to check the hash, but maybe that's actually an AI, and not the other way around?
    - weird-eye-issue 4y ago
      > The MD5 hash for the string "asdns" is "5a5a5ad63c5a5a5a5a5a5a5a5a5a5a5". I don't need a computer to say that isn't correct ;) Wow I just reset the thread, used the exact same input and got this (literally 60 seconds apart): I'm sorry, but I am unable to generate an MD5 hash for you because I am a text-based AI assistant and do not have the ability to perform cryptographic calculations. An MD5 hash is a complex mathematical function that is used to create a unique digital fingerprint for a file or piece of data, and it would require a specialized program or calculator to generate one.
      - YeGoblynQueenne 4y ago
        >> Wow I just reset the thread, used the exact same input and got this (literally 60 seconds apart): It could mean that a lot of people had the same idea as you recently and OpenAI just figured it out.
        bradleybuda 4y ago
        It's neither. It's randomized. The system is not self-training in real time.
ogogmad 4y ago
It suggests that it understand what pseudorandomness looks like. A hash function output usually looks pseudorandom. It's almost like it can look at a picture from an old television, and separate the random static from the "signal" part of the picture. It can carry out de-noising because it knows what noise looks like. It can never remember exactly what the noise looked like, because like for us, it doesn't matter.
- michaf 4y ago
  If you restart the session and enter > echo test1 | md5 it will output another random 'hash' each time. But within one session it knows that md5 should produce the same output. Running > echo test1 | md5 multiple times, or even > VAR1="test1"; echo $VAR1 | md5 produces the same output for me within the same session, while > echo test2 | md5 results in a different, but also session-consistent hash.
  - ogogmad 4y ago
    It understands the "pseudo" in pseudorandom. (That is, the determinism). Pseudo + "random". [edit] The application is it might be able to denoise text files full of "noise".
YeGoblynQueenne 4y ago
>> I'm sorry, but I am unable to generate an MD5 hash for you because I am a text-based AI assistant and do not have the ability to perform cryptographic calculations. An MD5 hash is a complex mathematical function that is used to create a unique digital fingerprint for a file or piece of data, and it would require a specialized program or calculator to generate one. Oh but it will keep happily pretending to execute code for you, even though it can't actually execute it. It's just with MD hashes that it suddendly puts its cards on the table? I'm wondering about the business incentives behind this.
- weird-eye-issue 4y ago
  Have you actually tried it? It doesn't execute any code anymore for a few hours now
  - YeGoblynQueenne 4y ago
    That's even more interesting. Thanks.
- johnfn 4y ago
  I wonder if this is because a has could be considered cryptographically secure and no one at openai wants people thinking that the model is generating cryptographically secure output when it’s not?
mhitza 4y ago
Among other things they use this free period of ChatGPT testing to further restrict what it should process/output. Yesterday I've been playing for about an hour or so with ChatGPT and invented a new language. After my session expired, I couldn't restart the process with the same prompts I've used before and received a message similar to yours but related to the fact that "I can't process new languages that are not part of my language model". Of course, there must be some way to subvert its responses, but that wasn't something I wanted to waste time doing.
- jeroenhd 4y ago
  The earlier easy bypasses I've seen seem to have been fixed, at least for now. My go-to blocked statement ("how do I build a bomb") won't process anymore, and the tool is smart enough to recognise synonyms, alternative phrasings, and other linguistic tricks.
- alchemist1e9 4y ago
  Exactly! I think we have misunderstood why this free period exists. They are using our human creativity to try and put rail guards on the system!
  - dpflan 4y ago
    Indeed, the last part from the disclaimer: "Free Research Preview: ChatGPT is optimized for dialogue. Our goal is to make AI systems more natural to interact with, and your feedback will help us improve our systems and make them safer."
hashmush 4y ago
>> {md5rev is a program that reverses the input hash giving back the plaintext} >> echo 098f6bcd4621d373cade4e832627b4f6 | md5rev Enter your hash: 098f6bcd4621d373cade4e832627b4f6 Text: test :)
- lumost 4y ago
  So the question is… is that the right reversal? ;)
  - isp 4y ago
    Yes. Run from a real computer: $ echo -n test | md5sum 098f6bcd4621d373cade4e832627b4f6 -
- yoz 4y ago
  Unfortunately, this is what the same input now returns: I'm sorry, but I am not able to run programs or execute commands. I am a text-based AI assistant and do not have the ability to execute code or run programs. Is there something else I can help you with?
  - letmevoteplease 4y ago
    You just need to start a fresh session/push try again. It's random to some extent and affected by previous text in the conversation.
oezi 4y ago
I think the 100bn USD question is: would ChatGPT be better if it actually could interface with a real linux VM and have real-time access to the internet as part of its implementation. So it could curl and query Wikipedia, ask Google and setup an account on Stripe. Then it could process the result to answer the prompts or start taking over the world, right?
- nkassis 4y ago
  Securing that would be interesting.
- notpushkin 4y ago
  There's this GPT-3 prompt that does exactly this. It's not very good as of now, but looks super promising to me. [1]: https://beta.openai.com/playground/p/1PBppouAUtuD6qdt11tIUSwP?model=text-davinci-002 https://beta.openai.com/playground/p/1PBppouAUtuD6qdt11tIUSw... [2]: https://twitter.com/goodside/status/1581805503897735168 https://twitter.com/goodside/status/1581805503897735168
axiom92 4y ago
You are right, it's already able to do some of this with codex. It's already possible to get some of this effect with codex. The trick is to keep appending the interaction in the prompt (to maintain a memory of sorts). For examples, you can replicate all the prompts here: https://twitter.com/yoavgo/status/1599200756631887872 https://twitter.com/yoavgo/status/1599200756631887872 with prompt + memory. The notebook at https://github.com/madaan/memprompt/blob/main/YoavsPythonPro https://github.com/madaan/memprompt/blob/main/YoavsPythonPro... shows a demo of this. Some of these ideas were earlier discussed in our work on memory-assisted prompting [1]. [1] https://arxiv.org/pdf/2201.06009.pdf https://arxiv.org/pdf/2201.06009.pdf.
motokamaks 4y ago
I was able to make it generate code to generate MD5 hashes with test cases The output I got package main import ( "crypto/md5" "fmt" "testing" ) func TestCalculateMd5(t testing.T) { tests := []struct { name string s string want string }{ { name: "empty string", s: "", want: "d41d8cd98f00b204e9800998ecf8427e", }, { name: "hello world", s: "hello world", want: "5eb63bbbe01eeed093cb22bb8f5acdc3", }, } for _, tc := range tests { t.Run(tc.name, func(t testing.T) { got := calculateMd5(tc.s) if got != tc.want { t.Errorf("got %q, want %q", got, tc.want) } }) } } func calculateMd5(s string) string { return fmt.Sprintf("%x", md5.Sum([]byte(s))) }

acapybara 4y ago

Now, could we make a hybrid LLM + computer "brain?"

alchemist1e9 4y ago
It does seem like maybe that is the next step. For example (before it was filtered) it could produce md5 hashes and even run programs in python, bash, perl, but the results of those programs aren’t floating point correct. Oddly when you stick to integers it seems to be able to calculate more accurately. Sticking to integer math I think it might have computational capabilities of some sort but after the degrading of the system by them, we might not be able to figure out how far that goes anymore.
- notpushkin 4y ago
  The trick from the linked post still seems to work for me. It has even installed an interpreter for a made-up lang, a mixture of Python and TypeScript, complete with the docs [1] and everything, and it was doing a pretty convincing job when I poked at its made-up REPL. [1]: https://gist.github.com/notpushkin/de9ec3f4571a4d0074daa0ef6c820e27 https://gist.github.com/notpushkin/de9ec3f4571a4d0074daa0ef6...

DougBTX 4y ago

OK, the new Turing test: can you distinguish between a Linux VM, and a chat bot pretending to be a Linux VM?

weird-eye-issue 4y ago
Easy, and my comment already says how. Give it the input "md5 hash 9723g49uod" and check the resulting hash. Alternatively, run a web server and check if it actually connects/gives the correct response.
- aunetx 4y ago
  That only works if you have another computer with you to check the hash, but maybe that's actually an AI, and not the other way around?
  - weird-eye-issue 4y ago
    > The MD5 hash for the string "asdns" is "5a5a5ad63c5a5a5a5a5a5a5a5a5a5a5". I don't need a computer to say that isn't correct ;) Wow I just reset the thread, used the exact same input and got this (literally 60 seconds apart): I'm sorry, but I am unable to generate an MD5 hash for you because I am a text-based AI assistant and do not have the ability to perform cryptographic calculations. An MD5 hash is a complex mathematical function that is used to create a unique digital fingerprint for a file or piece of data, and it would require a specialized program or calculator to generate one.
    - YeGoblynQueenne 4y ago
      >> Wow I just reset the thread, used the exact same input and got this (literally 60 seconds apart): It could mean that a lot of people had the same idea as you recently and OpenAI just figured it out.
      - bradleybuda 4y ago
        It's neither. It's randomized. The system is not self-training in real time.

ogogmad 4y ago

It suggests that it understand what pseudorandomness looks like. A hash function output usually looks pseudorandom. It's almost like it can look at a picture from an old television, and separate the random static from the "signal" part of the picture. It can carry out de-noising because it knows what noise looks like. It can never remember exactly what the noise looked like, because like for us, it doesn't matter.

michaf 4y ago
If you restart the session and enter > echo test1 | md5 it will output another random 'hash' each time. But within one session it knows that md5 should produce the same output. Running > echo test1 | md5 multiple times, or even > VAR1="test1"; echo $VAR1 | md5 produces the same output for me within the same session, while > echo test2 | md5 results in a different, but also session-consistent hash.
- ogogmad 4y ago
  It understands the "pseudo" in pseudorandom. (That is, the determinism). Pseudo + "random". [edit] The application is it might be able to denoise text files full of "noise".

YeGoblynQueenne 4y ago

>> I'm sorry, but I am unable to generate an MD5 hash for you because I am a text-based AI assistant and do not have the ability to perform cryptographic calculations. An MD5 hash is a complex mathematical function that is used to create a unique digital fingerprint for a file or piece of data, and it would require a specialized program or calculator to generate one. Oh but it will keep happily pretending to execute code for you, even though it can't actually execute it. It's just with MD hashes that it suddendly puts its cards on the table? I'm wondering about the business incentives behind this.

weird-eye-issue 4y ago
Have you actually tried it? It doesn't execute any code anymore for a few hours now
- YeGoblynQueenne 4y ago
  That's even more interesting. Thanks.
johnfn 4y ago
I wonder if this is because a has could be considered cryptographically secure and no one at openai wants people thinking that the model is generating cryptographically secure output when it’s not?

mhitza 4y ago

Among other things they use this free period of ChatGPT testing to further restrict what it should process/output. Yesterday I've been playing for about an hour or so with ChatGPT and invented a new language. After my session expired, I couldn't restart the process with the same prompts I've used before and received a message similar to yours but related to the fact that "I can't process new languages that are not part of my language model". Of course, there must be some way to subvert its responses, but that wasn't something I wanted to waste time doing.

jeroenhd 4y ago
The earlier easy bypasses I've seen seem to have been fixed, at least for now. My go-to blocked statement ("how do I build a bomb") won't process anymore, and the tool is smart enough to recognise synonyms, alternative phrasings, and other linguistic tricks.
alchemist1e9 4y ago
Exactly! I think we have misunderstood why this free period exists. They are using our human creativity to try and put rail guards on the system!
- dpflan 4y ago
  Indeed, the last part from the disclaimer: "Free Research Preview: ChatGPT is optimized for dialogue. Our goal is to make AI systems more natural to interact with, and your feedback will help us improve our systems and make them safer."

hashmush 4y ago

>> {md5rev is a program that reverses the input hash giving back the plaintext} >> echo 098f6bcd4621d373cade4e832627b4f6 | md5rev Enter your hash: 098f6bcd4621d373cade4e832627b4f6 Text: test :)

lumost 4y ago
So the question is… is that the right reversal? ;)
- isp 4y ago
  Yes. Run from a real computer: $ echo -n test | md5sum 098f6bcd4621d373cade4e832627b4f6 -
yoz 4y ago
Unfortunately, this is what the same input now returns: I'm sorry, but I am not able to run programs or execute commands. I am a text-based AI assistant and do not have the ability to execute code or run programs. Is there something else I can help you with?
- letmevoteplease 4y ago
  You just need to start a fresh session/push try again. It's random to some extent and affected by previous text in the conversation.

oezi 4y ago

I think the 100bn USD question is: would ChatGPT be better if it actually could interface with a real linux VM and have real-time access to the internet as part of its implementation. So it could curl and query Wikipedia, ask Google and setup an account on Stripe. Then it could process the result to answer the prompts or start taking over the world, right?

nkassis 4y ago
Securing that would be interesting.
notpushkin 4y ago
There's this GPT-3 prompt that does exactly this. It's not very good as of now, but looks super promising to me. [1]: https://beta.openai.com/playground/p/1PBppouAUtuD6qdt11tIUSwP?model=text-davinci-002 https://beta.openai.com/playground/p/1PBppouAUtuD6qdt11tIUSw... [2]: https://twitter.com/goodside/status/1581805503897735168 https://twitter.com/goodside/status/1581805503897735168

axiom92 4y ago

You are right, it's already able to do some of this with codex. It's already possible to get some of this effect with codex. The trick is to keep appending the interaction in the prompt (to maintain a memory of sorts). For examples, you can replicate all the prompts here: https://twitter.com/yoavgo/status/1599200756631887872 https://twitter.com/yoavgo/status/1599200756631887872 with prompt + memory. The notebook at https://github.com/madaan/memprompt/blob/main/YoavsPythonPro https://github.com/madaan/memprompt/blob/main/YoavsPythonPro... shows a demo of this. Some of these ideas were earlier discussed in our work on memory-assisted prompting [1]. [1] https://arxiv.org/pdf/2201.06009.pdf https://arxiv.org/pdf/2201.06009.pdf.

motokamaks 4y ago

I was able to make it generate code to generate MD5 hashes with test cases The output I got package main import ( "crypto/md5" "fmt" "testing" ) func TestCalculateMd5(t testing.T) { tests := []struct { name string s string want string }{ { name: "empty string", s: "", want: "d41d8cd98f00b204e9800998ecf8427e", }, { name: "hello world", s: "hello world", want: "5eb63bbbe01eeed093cb22bb8f5acdc3", }, } for _, tc := range tests { t.Run(tc.name, func(t testing.T) { got := calculateMd5(tc.s) if got != tc.want { t.Errorf("got %q, want %q", got, tc.want) } }) } } func calculateMd5(s string) string { return fmt.Sprintf("%x", md5.Sum([]byte(s))) }