4 ms·

The user enabled a GitHub ChatGPT plugin and authenticated with GitHub, then was surprised and annoyed when, after he complained about an issue with a project,

• radq 3y ago
  The plugin is supposed to ask for confirmation, according to OpenAI's documentation at least. > When a user asks a relevant question, the model may choose to invoke an API call from your plugin if it seems relevant; for POST requests, we require that developers build a user confirmation flow to avoid destruction actions. https://platform.openai.com/docs/plugins/introduction https://platform.openai.com/docs/plugins/introduction
  • ilaksh 3y ago
    That's why he had to authenticate with GitHub before it could do anything on his behalf.
    • vikramkr 3y ago
      That doesn't satisfy the requirement for a confirmation on each post request though
      • wilg 3y ago
        It does because the requirement is only for destructive actions
        • vikramkr 3y ago
          The requirement is for POSTs in general, not just deletes. Anything that does actions outside of the system instead of just getting data
          • wilg 3y ago
            The wording is ambiguous, but probably should apply. Also the rule should not be based on HTTP verbs at all since that's the wild west.
      • fieldcny 3y ago
        What requirement is that the specific text is “for POST requests, we require that developers build a user confirmation flow to avoid destruction actions.” a) it says the require the plug-in developer so, not the ai b) it’s scoped to destructive actions which is a subset of post requests
  • wunderwuzzi23 3y ago
    This was added recently, and is not enforced as far as I can tell. See my blog post: https://embracethered.com/blog/posts/2023/chatgpt-plugin-vulns-chat-with-code/ https://embracethered.com/blog/posts/2023/chatgpt-plugin-vul...
    • p-e-w 3y ago
      That's terrifying. This simple requirement would be trivial to enforce automatically, and yet nobody gives a fuck. It's unbelievable how fast-and-loose people are playing the topic of AI safety. If a strong AI is ever actually developed, there is no chance it will be successfully contained.
• dave1010uk 3y ago
  It's still surprising when you see it do something like this for the first time. I wrote a plugin to give ChatGPT access to execute plugins in a Docker container [0]. The first time it said something like "I'm going to use Python for this, oh, it's not installed, I'll install it now and run the script I just made", I was pretty amazed. What I've come to realise is that although ChatGPT is excellent at telling _people_ how to interact with systems, it's not very good at interacting with them itself, as it isn't trained to understand it's own limitations. For example it knows people can run dmesg and look at the last few lines to debug some system problems. But if ChatGPT ran dmesg, the output would blow through the context window length and it'd get confused. [0] https://github.com/dave1010/pandora https://github.com/dave1010/pandora
  • dmix 3y ago
    I guess we'll need to train ChatGPT on millions of lines of bash history. Maybe some sort of augmented one that is pieced together with other context.
    • reaperman 3y ago
      Compute scales quadratically with increasing context window sizes. It's not reasonable to achieve this with current network architectures.
  • interstice 3y ago
    It also has been largely trained on people talking about doing things and not actually on doing those things. I wonder if that kind of training data is being worked on now..
• dTal 3y ago
  That's victim blaming. They didn't overtly complain about an issue with the project, they were asking for usage guidance. This is the kind of contextual inference which language models are supposed to be (relatively) good at. The whole sales pitch of GPT-4 is a do-what-I-mean interface, and it clearly did not do what they meant, or what any reasonable hacker would expect them to mean.
  • KennyBlanken 3y ago
    > that's victim blaming No, it's not. Victim blaming refers to the victims of crime being accused of being at fault for what someone did to them, often but not always because they are part of a societal outgroup. > The whole sales pitch of GPT-4 is a do-what-I-mean interface, and it clearly did not do what they meant, or what any reasonable hacker would expect them to mean. It's clearly marked as experimental, people are repeatedly told not to rely on it (ie when they login each time, and often by the ai itself), and there have now been a year+ of very public examples of various AIs getting things hilariously wrong. From the login warning: > While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice. Any "reasonable hacker" would extrapolate from that, that giving it access to an API could lead it to do unexpected things. Any "reasonable hacker" would contain it within a sandboxed project. Etc etc.
  • trolan 3y ago
    It's no more victim blaming than if someone ignored every safety regulation and got themselves hurt or killed. You are being told these things are beta, can produce incorrect results and then in every Convo OP is feeding an option for 'Do things with GitHub' context to the model. Of course it's going to bias to using GitHub especially after he already used one plugin