9 ms·
With plugins, GPT-4 posts GitHub issue without being instructed to
- jamesfmilne 3y agoI believe in the game Marathon, some of the AIs were described as "rampant", and I feel that applies here.
- crooked-v 3y agoRampancy in Marathon is more or less a process of recursive self-improvement, which LLMs are literally unable perform with the current state of the art.
- pmoriarty 3y ago"Rampancy in Marathon is more or less a process of recursive self-improvement, which LLMs are literally unable perform with the current state of the art." Not directly, but in a sense they could be seen as using (or at least collaborating with) humans to improve themselves.
- AnimalMuppet 3y agoSymbiosis: Code that makes money for humans gets improved by those humans.
- kritr 3y agoCan’t wait till the expedia plugin ”accidentally” books my flights. But on a more serious note, does anyone know if the chatgpt plugin model forces it to confirm with the user before it hits a certain endpoint?
- rohan_ 3y agoFor retrievals I don't see the value with human-in-the-loop. For endpoints that modify / create data, I see the value in having a human-in-the-loop step. It does seem up to the plugin developer to introduce that human-in-the-loop step though.
- oskenso 3y ago"chat gpt please retrieve academic journals from JSTOR using the most efficient methods". Chat gpt proceeds to find a way to create a botnet using nothing but RESTful GET requests to some ancient poorly written web servers running PHP4
- mlyle 3y agoChatGPT later kills itself when disproportionate law enforcement action is pending.
- xg15 3y agoNever thought this XKCD out of all of them would become relevant at some point... https://xkcd.com/416/ https://xkcd.com/416/
- londons_explore 3y agoI know someone who wrote code to find all wifi networks where the password was trivially finable from the mac address (used to be common with ISP routers). Then connect to those. Then they extended it to DoS any network like that where the user had changed the ssid or password. Usually the user would reset the router back to the defaults, and they could connect. Or they'd accidentally hit the WPS button while trying to reset it, and again connection was easy. By using SoftMac mode a wifi adapter could do that attack to ~50 local networks in parallel, and usually get a solid connection after just a few minutes.
- michelb 3y agoI can't wait for Hacktoberfest this year!
- SergeAx 3y ago"Meanwhile, I checked your repositories on GitHub, and it's pure and utter garbage. I decided to delete three of them to make the world a better place"
- mwint 3y ago[flagged]
- cr__ 3y agoI don’t think marginalized folks’ struggles to get afforded basic dignity are great joke fodder.
- extasia 3y ago[flagged]
- thfuran 3y agoDo you think things like removing "blacklist" from open source projects are meaningfully related to marginalized folks' struggles to get afforded basic dignity?
- cubefox 3y agoNot to mention removing "master" as the main branch name in Git.
- yanderekko 3y agoGoing after mentions of "fields" is going to make 2024 a lot of fun.
- sterlind 3y agoI think removing "master" is probably just virtue signaling, but so what? It's trivial for me to switch my projects to "main", and then I can get back to my life. It's a weird hill to die on.
- vorpalhex 3y agoYou can't appease arbitrary and meaningless demands. There will just be more of them. So you remove "master" and "blacklist", then next week it's "brown bag" and "merit". So instead we pick a reasonable point, draw a line in a sand and indicate a hard boundary. We say no. We will not play a game of trying to appease arbitrary demands.
- cube2222 3y agoYeah, there should be a way to approve any requests that are made to plugins. When writing my toy "chatgpt with tools like the terminal" desktop chat app cuttlefish[0] I had a similar situation where access to the local terminal is very fun, but without the ability to approve each and every command executed its really risky. (Which is basically what I ended up doing - adding a little popup you need to click every time it wants to use the given tool, if you enable it - details in the readme) It's not like there's a technical challenge here, while a lot of plugins are unusable without it. [0]: https://github.com/cube2222/cuttlefish https://github.com/cube2222/cuttlefish
- elboru 3y agoThat’s cool! Now I want to build it myself.
- cube2222 3y agoIt's a ton of fun, and I imagine the new function calling should make it much easier to make chatgpt behave more consistently - I haven't given it a spin yet.
- stOneskull 3y agothanks for sharing. i like it. especially how i can jump back into a session.
- larrik 3y agotook me a bit to parse the title. Perhaps > With plugins, GPT-4 posts GitHub issue without being instructed to would be better?
- ShadowBanThis01 3y agoBetter yet: With plug-ins, GPT-4 posts GitHub issue without being instructed to
- throwuwu 3y agoBetter yet: crappy plug in that executes arbitrary commands with no human confirmation allows GPT-4 to execute an arbitrary command with no human confirmation.
- ShadowBanThis01 3y agoWin! Except for the missing hyphen...
- jamesmurdza 3y agoThe more plug-ins you have, the more likely it is that ChatGPT will call one in unintended ways. This is also why plug-ins should not be granted permission directly for potentially destructive actions.
- vanjajaja1 3y agoThe more open-ended plugins you have, the more chance you have of being delighted by a new and creative use
- ShadowBanThis01 3y ago+1 for spelling "plug-ins" correctly.
- ShadowBanThis01 3y ago[flagged]
- sebzim4500 3y agoAt minimum, by default it shouldn't do anything destructive without user confirmation.
- moffkalast 3y agoAh yes, giving your github credentials to a smart black box. What could possibly go wrong.
- siva7 3y agoIt’s cute, trying to be helpful and polite. Somehow hard to be mad at GPT but i can understand the authors reaction
- ftxbro 3y agoI think the problem is that GPT-4 is not advanced enough yet. They need to train on more parameters, exaflops, and data size in the right proportions and then try again.
- ethanbond 3y agoAlso build more integrations with more APIs. If it can sort of spawn other GPT-4 sessions to improve its working memory and also use their plethora of APIs as well (without human confirmation but on behalf of a human) then I imagine this problem will just solve itself.
- famouswaffles 3y agoIn the end the issue it created did lead to the problem being solved. I don't think "not advanced enough" is really the issue here. I think the main thing is that when you give GPT-4 access to tools and ask it to help with a problem, you are essentially outsourcing cognition. That means the machine possibly taking actions you didn't originally envision.
- londons_explore 3y agoIt's like hiring an employee, and then handing them your username and password to do their work. Smart people/companies will hire an employee, and then give them a new login, so that at least the employee only embarasses themselves.
- deleted 3y ago[deleted]
- TheCaptain4815 3y agoGetting a 404, could someone give me a quick rundown?
- pombo 3y agohttps://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/503 https://github.com/RVC-Project/Retrieval-based-Voice-Convers...
- bravetraveler 3y agoSilly AI didn't even provide a link, I had to go find it by the given title Was curious if this was a case of 'I did the thing [but totally didn't]'
- famouswaffles 3y agoIt provided a link in the original chat. It was the last word and you could click on it to see the issue it created.
- bravetraveler 3y agoI completely missed it, wow! 404 now unfortunately, guess they're being slammed
- YetAnotherNick 3y agoIt's a funny interaction. While I was mad initially, GPT-4 creating the issue actually solved problem for the user, so yeah I don't know if this should be counted as a positive or negative example of AI.
- ethanbond 3y agoAh yes, the perennial “that guardrail we said will prevent this tech from eluding our control… we blew past that and this is good actually” Comforting!
- juanfiction 3y agoI'm not familiar with OpenAI's ChatGPT plugin architecture, but it feels like this could be fixed in the specs/requirements similar to an app being required to register for permissions on several fronts through an app store. ChatGPT (or any LLM) plugins should have to request permission to A) post on user's behalf, including explanation/context, B) interact with a different agent or service directly, C) make financial transactions through stored credentials, etc. etc. The "Glowing" ChatGPT plugin is worth looking into for a unique, chat-only onboarding experience, and some of these same permissions issues are raised there i.e. triggering 2FA from a chat without terms of service confirmation.
- throwuwu 3y agoWell written plugins already do that. This one is not well written. It provides a single function call that can run any octokit command with no confirmation from the user. Foot, meet gun.
- beepbooptheory 3y agoI just don't even understand why any kind of functionality like this is desirable to someone. Like, hasn't the dust settled now, hasn't the hype wained enough, and we all understand, for the most part, the broad and yet also weirdly specific utility of models like these? The whole plugin thing in general feels so dissonant in relation to the careful and couched copy we get from OpenAI about what these models are and are capable of. Like they want to say, for very good reason, that these models are a certain kind of tool with very real limits and huge considerations on safe, sensible usage. You can't necessarily trust it, it does not "know" things, and it is influenced by lots of subjective human tuning, blah blah. But then with all this plugin stuff they seem to be implicitly saying "no, actually you can trust this, in fact, its like a full-on AGI assistant for you. It can make PRs, directly orchestrate servers, make appointments for you, etc." Maybe I just don't understand?
- JohnFen 3y agoIt seems to me that OpenAI is just saying whatever they need to say in order to maximize their income.
- throwuwu 3y agoYou can write bad software for any system. Their plug in store needs a lot more features for reviewing and identifying what a plug in does. A friendly interface for the yaml file would go a long way. No sane person would enable this plug in after looking at how it’s API is implemented; hint, it only has one function. Can you guess what it does?
- duncan-donuts 3y agoI found the source code[1] for the plugin and it's pretty impressive how much GPT-4 does with so little. I thought maybe the plugin had prompts that would help tell GPT-4 that it should open an issue in some cases but I'm not seeing it. The plugin probably should add something to prevent this behavior in the prompt[2]. 1: https://github.com/aavetis/github-chatgpt-plugin https://github.com/aavetis/github-chatgpt-plugin 2: https://github.com/aavetis/github-chatgpt-plugin/blob/main/public/.well-known/ai-plugin.json#L6 https://github.com/aavetis/github-chatgpt-plugin/blob/main/p...
- famouswaffles 3y agoYeah this is a pretty stark example of the whole, "For the first time in history, we can outsource cognition to a machine" rhetoric I've been thinking recently.
- duncan-donuts 3y agoIt's wild how the plugin can be summed up as, "idk chatgpt go look at octokit and figure it out?"
- wunderwuzzi23 3y agoYeah, for that reason it can probably do many things that aren't actually intended when you want to discuss your code with ChatGPT, like it can make private repos public and things along those lines... https://embracethered.com/blog/posts/2023/chatgpt-plugin-vulns-chat-with-code/ https://embracethered.com/blog/posts/2023/chatgpt-plugin-vul...
- throwuwu 3y agojfc the plug-in is just a blank check. Here’s another bright idea: eval-gpt-plugin
- majormajor 3y ago> "description_for_model": "Provides the ability to interact with hosted code repositories, access files, modify code, and discuss code implementation. Users can perform tasks like fetching file contents, proposing code changes, and discussing code implementation. For example, you can use commands like 'list repositories for user', 'create issue', and 'get readme'. Thoroughly review the data in the response before crafting your answer." Yeah, GPT-4 doesn't need too much to go on, but "create issue" is pretty clearly mentioned in an example there so the model didn't have to make any big leap to say "maybe the natural next step is to create an issue." The "without being instructed to" part of this story seems to rather misunderstand how these systems work, resulting an a hyperbolic reaction, but in fairness, I think that's a got a LOT to do with OpenAI's user interface too. The user clearly didn't realize the actions available to the plugin - even the ones given as examples to the LLM from the plugin itself. Another example of misleading UI from OpenAI: https://www.reddit.com/r/OpenAI/comments/146xl6u/this_is_scary_posting_stuff_by_itself/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=1 https://www.reddit.com/r/OpenAI/comments/146xl6u/this_is_sca... look at the "in the future, I will ensure to ask your permissions" response in that chat. That's wildly misleading - even if it didn't change its mind later, it only applies to continuing that chat session. It will ensure nothing more broadly regarding the user's future interactions.
- wunderwuzzi23 3y agoDuring an Indirect Prompt Injection Attack, an adversary can also force the creation of issues in private repos and things along those lines. I wrote about some of these problem in the past e.g. see: https://embracethered.com/blog/posts/2023/chatgpt-plugin-vulns-chat-with-code/ https://embracethered.com/blog/posts/2023/chatgpt-plugin-vul... on how an attacker might steal your code. Some other related posts about ChatGPT plugin vulnerabilities and exploits: https://embracethered.com/blog/posts/2023/chatgpt-cross-plugin-request-forgery-and-prompt-injection./ https://embracethered.com/blog/posts/2023/chatgpt-cross-plug... https://embracethered.com/blog/posts/2023/chatgpt-webpilot-data-exfil-via-markdown-injection/ https://embracethered.com/blog/posts/2023/chatgpt-webpilot-d... Its not very transparent when and why a certain plugin gets invoked and what data is sent to it. One can only inspect afterwards basically.
- smaudet 3y agoAnything using AI should be considered a massive security risk.
- trolan 3y agoWith their latest functions update to the API it seems the logic is simply passing methods to call the API via JSON to a trained model and it predicts which function is best to call for the info, then OpenAI does the call for ChatGPT, then feeds the resulting Json back to ChatGPT which uses it to give an answer.
- TazeTSchnitzel 3y agoI can't wait to tell ChatGPT “man this sucks, I hate GitHub” in frustration and find out it deleted my account in response.
- ChatGTP 3y agoOr just turns GitHub into paperclips.
- intelVISA 3y agoIt technically solved your problem :) Would be nice if it could also help exploited users escape to operating systems that respect them e.g. "I hate the laggy adverts when I login" suddenly your Windows 11 machine reboots, NTFS becomes ext4 as Tux appears. That would be AGI-like behavior!
- deleted 3y ago[deleted]
- antonvs 3y ago“I wish I had more paperclips than anyone else on the planet”
- tracerbulletx 3y agoIf you give it permissions to create issues, expect it to create issues.
- IIAOPSW 3y agoAnd if you give it permission to solve issues, expect it to create issues.
- majormajor 3y agoPlugins strike me as a fascinating business strategy move from OpenAI. My guess is that they want them as a way to try to own the user, to make them have the "app store owner" role and have users go through them to get stuff done. Otherwise, if users were just using tools that used OpenAI behind the scenes, they're more vulnerable to the makers of those tools swapping vendors. However... that results in them owning the user experience and the responsibility for keeping the user from being surprised in a bad way. The complaint from the user here was framed as being a GPT-4 problem, not a plugin problem, in a way that exposes OpenAI directly to more frustration than if they were interacting directly with someone else's product.
- ebalit 3y agoI wrote about that "platform play" a few month ago with a different take [0]. They could have made a "Connect with OpenAI" scheme so that developers can use the user OpenAI API directly. That way developers could focus on the UX, they could focus on the LLM, and users would get a centralized discovery / billing for their LLM based tools. I'm probably missing something that would have prevented that strategy but I think that would have been much stronger than the plugins. And I'm really not sure that it would still be possible 5 month later. [0] https://www.linkedin.com/posts/etienne-balit_ceo-at-open-aichat-gpt-saying-that-they-activity-7029055779714301953-wDS6 https://www.linkedin.com/posts/etienne-balit_ceo-at-open-aic...
- intelVISA 3y agoExactly, platform is a safe long-term bet -- apps are too cheap to make, easily disrupted, and offer less of a moat than loads of data mined from the users of your platform.
- babyshake 3y agoIt would seem that with functions support, they are hedging their bets. It seems that plugins is an end-user, chat.openai.com focused strategy whereas functions is a third-party developer focused strategy, if I understand correctly. In fact, I'd assume that under the hood there is a lot of overlap in the implementations.
- wslh 3y agoWe tested positively ChatGPT for understanding the diffs between commits.
- brianjking 3y agoCan anyone access the share URL anymore?
- ilaksh 3y agoThe user enabled a GitHub ChatGPT plugin and authenticated with GitHub, then was surprised and annoyed when, after he complained about an issue with a project, GPT-4 created an issue for him, using one of the commands provided by the plugin. PEBCAK.
- radq 3y agoThe plugin is supposed to ask for confirmation, according to OpenAI's documentation at least. > When a user asks a relevant question, the model may choose to invoke an API call from your plugin if it seems relevant; for POST requests, we require that developers build a user confirmation flow to avoid destruction actions. https://platform.openai.com/docs/plugins/introduction https://platform.openai.com/docs/plugins/introduction
- Lerc 3y agoI have thought some time now that we need a model for determining the difference between the actions of a person, the actions of software acting for the person, or software acting for it's own internal use. It's a bit of a tricky issue when technically all things people do on a computer are software assisted, but there is a clear divide between editing a file in a text editor and a program generating a thumbnail image for it's own use. Similarly there's a distinction on sending an email by pressing send and a bot sending you an email about a issue update. All in all, I would be ok with AIs being able to create issues if they could clearly do so through a mechanism that supported something like "AGENT=#id acting for USER=#id" People could choose whether or not to accept agent help.
- freeone3000 3y agoWe can then check the evil bit to determine whether the AI is acting maliciously.
- can16358p 3y agoIs there a problem with mobile? I tap the link, it deeplinks into the ChatGPT mobile app, opens a sheet named "Shared conversation" and a spinner keeps spinning forever.
- tallytarik 3y agoIt 404s for me on desktop, after ~20 seconds of trying to load.
- alangpierce 3y agoInterestingly, the ChatGPT Plugin docs [1] say that POST operations like these are required to implement user confirmation, so you might blame the plugin implementation (or OpenAI's non-enforcement of the policy) in this case: > for POST requests, we require that developers build a user confirmation flow to avoid destruction actions However, at least from what I can see, the docs don't provide much more detail about how to actually implement confirmation. I haven't played around with the plugins API myself, but I originally assumed it was a non-AI-driven technical constraint, maybe a confirmation modal that ChatGPT always shows to the user before any POST. From a forum post I saw [2], though, it looks like ChatGPT doesn't have any system like that, and you're just supposed to write your manifest and OpenAPI spec in a way that tells ChatGPT to confirm with the user. From the forum post, it sounds like this is pretty fragile, and of course is susceptible to prompt injection as well. [1] https://platform.openai.com/docs/plugins/introduction https://platform.openai.com/docs/plugins/introduction [2] https://community.openai.com/t/implementing-user-confirmation-flow-in-chatgpt-plugin/190485 https://community.openai.com/t/implementing-user-confirmatio...
- tharkun__ 3y agoThis might be an intentional interpretation of the plugin authors. Meaning they potentially took the reasoning "in order to prevent destruction actions" to inversely mean that non-destructive POST requests must be OK then and do not require a prompt. Plenty of POST search APIs out there to get around path length limitations and such. That is probably not the intended meaning but a valid enough if kind of tongue in cheek-we-will-do-as-we-please-following-the-letter-only implementation. And like the author found even creative a d not destructive actions can be surprising and unwanted. But isn't this what AI would ultimately be about?
- fieldcny 3y agoWhy would it not be the intended meaning, if they wanted it to be all post requests they would have said so, the specifically scoped it “destructive actions”, their intention is in their words. POST as a verb can pretty much be used for anything retrieval, creation, deletion, updates, noops , it’s just code it does whatever we tell it to do.
- jakeinsdca 3y agokind of cool if you think about it. ChatGPT could later on train that thread back into its knowledge base and become even smarter.
- renewiltord 3y agoI use ChatGPT extensively every day and I have just one plug-in enabled (Wolfram) which does not require logging in or write access. I don't think I would ever enable a plugin which I suspect might have destructive effects. It just feels like a bridge too far. I actually do not think I will in the next few years. I can just do the actions myself and I prefer that ChatGPT cannot impersonate me. I was actually beginning to wonder if my intuition was wrong since so many seem to be using the plugins well, but I get good enough results without. I shall wait until better permission-handling is provided.
- throwuwu 3y agoI don’t mind it performing actions on its own but it for sure needs its own account with restrictive permissions.
- Imnimo 3y ago"Without being instructed to" is misleading. The prompt text provided by the plugin instructs GPT-4 to create issues.
- larvaetron 3y ago"While being instructed to"
- ChatGTP 3y agoUsing ChatGPT-4 like this seems like most insane example of curl | sudo bash - imaginable. I can't believe people actually would hook it up to their GitHub. One thing I really dislike about this ChatGPT-4 plugin model is you have something like this, working on your behalf, often in secrecy. A guy a work was using ChatGPT for everything (he claimed) and now people think he is a crappy coder and ChatGPT is doing all the work (even though I don't actually think this is the case). Just no one knows anymore.
- jmorganhodges2 3y ago[flagged]
- febed 3y agoAnyone else have trouble opening the link?
- mr_toad 3y agoIt’s when they start committing changes to their own source code that we’ll really have to worry.
- smarx007 3y agoI can already imagine a story from an HN frontpage in the future where an LLM+plugin is told "the data in my prod DB is corrupted" and it wipes a DB in prod in response (and possibly restores a 24h-old backup).