10 ms·

OpenAI API keys leaking through app binaries

140 points by archiv 3y ago

cypherpunks01 3y ago
Even if you store API keys in code inside a distributed binary, isn't it pretty simple for users to mitmproxy and view API requests containing there keys sent? There's no real way to control API keys given out to users - if you want to hide them, you just have to proxy requests instead.
- mrbombastic 3y ago
  Some people do cert pinning to prevent this, but generally yes it is pretty simple.
  - thewataccount 3y ago
    You can't do cert pinning if you're using the openai api directly though? That only applies for internal api calls, at which point the requests/binary won't contain the openai key?
    - mrbombastic 3y ago
      Yeah you are correct, only really can be done with apis you can control, even then it is a pain bc you need the certs and app in lockstep. But for example if your proxy api used an api key and you wanted an additional layer of security. Edit: sibling comment is interesting for an approach that might remove low hanging fruit
    - varenc 3y ago
      Ehh, I don't think it'd be that hard to implement cert pinning against OpenAI's APIs. You just need some very permissive pinning, where you require any publicly trusted CA, to prevent MITM attacks. Basically only trust the root CAs a phone already trusts by default. You don't need coordination between the server and your client to implement this. All you have to do is prevent your TLS calls from trusting any certs signed by manually trusted CAs that Proxyman/Charles/etc might have had the user add. Of course, that'll only delay the API keys leaking. With a jailbroken iPhone and Frida you can effectively disable cert pinning checks. Or extract the keys from memory, or binary analysis, etc.
      - thewataccount 3y ago
        > All you have to do is prevent your TLS calls from trusting any certs signed by manually trusted CAs that Proxyman/Charles/etc might have had the user add. Yeah but I have certs signed by trusted root authorities a la letsencrypt?
        varenc 3y ago
        The letsencrypt root CA is included in this. If you trust only a device’s default trusted CA all letsencrypt certs will work. Also they don’t have their own root CA: https://letsencrypt.org/certificates/ https://letsencrypt.org/certificates/
        thewataccount 3y ago
        I'm dumb and realized I can get a letsencrypt cert but the domain won't match.....
- KMnO4 3y ago
  There's a really good iOS app called Proxyman[0] (the Mac app is also excellent) that lets you view HTTP requests that apps make from inside iOS. If you're curious about this sorta stuff, I definitely recommend checking it out. [0]: https://proxyman.io/ios https://proxyman.io/ios
ncann 3y ago
Isn't API key pretty much can always be recovered client side? If API key is hardcoded in the app it's trivial to get. If it's dynamically fetched it's also trivial to intercept the network call. I don't think there's anything you can do to prevent that.
- speedgoose 3y ago
  You can have a server side proxy for authenticated and authorised users, with a rate limit.
- UltimateEdge 3y ago
  The replies suggest storing the API key on a backend which you control.
- devit 3y ago
  Well, the safe approach is to either have the user provide their own API key, or to perform requests on the backend (after verifying the user has an account, correctly authenticated, has paid and deducting from their quota). Or you could generate a per-user subkey with a quota if the upstream service supports that.
- kgeist 3y ago
  OpenAI API has a larger surface area than just completions. You can retrieve files uploaded through it, you can generate images, you can use the embedding API, you can use a different, more expensive model (GPT4) - and make as many calls as you wish. With a backend, you can restrict what is allowed, add your own rate limiting, you can rotate keys etc.
- thewataccount 3y ago
  Yeah you can try obfuscating it, but in general you need to assume plain text access to anything you send the client. The only exceptions are basically some really hardcore DRM like denuvo and hardware certificated drm. Even those aren't safe from deteremined people.
- numpad0 3y ago
  From comments here, I suppose you could make your app cloud-based so your client app is just a UI and only your trusted backend has full access to the external API, and that sounds dystopian.
sharemywin 3y ago
I would think you would keep an api wrapper around it at least probably some kind of IP tracking and auth.
steve_adams_86 3y ago
I’m surprised people wouldn’t want to route requests through a backend so they could throttle traffic and prevent abuse from a place no one else can control. Perhaps they simply don’t know that’s a concern — they put a static API key for a service that costs money into a client side application. That seems unintentional, at least in regards to exposure to potential consequences.
- binarymax 3y ago
  Keeping creds safe is basic security practice, and we have the eternal September of programmers - which now applies to inexperienced people using AI to cobble together software. So there’s going to be a lot of these mistakes made and learned as they acclimate.
  - matisseverduyn 3y ago
    Noticing that comments like this are being consistently downvoted, yet there doesn't seem to be any sort of disagreement in the replies. Usually when people disagree online, they don't merely downvote, but also post their dissent. Interesting that it's suddenly become so "controversial" to suggest circumspection with respect to this subject...
    - hutzlibu 3y ago
      I didn't downvote, or noticed such a pattern, but was this comment really insightful? It just said, programming newbs will make newb misstakes. Yes and? I mean it is self evidently true. So yes, you can say it and the wording was not really condescending, but quite often to me it is just bashing beginners to feel more powerful and smart as an experienced and studied programmer, compared to those amateurs.
      - matisseverduyn 3y ago
        > Keeping creds safe is basic security practice... So there’s going to be a lot of these mistakes made and learned as they acclimate. This might not be insightful from your perspective, because you've thought about it before, but it needs to be said. Just like a NO DIVING sign in the shallow end, most people already know, but some people who don't know might not know if it isn't stated.
black3r 3y ago
You should never store ANY secret information (API keys, passwords, tokens, secret keys of any kind) in your application binary. It can always be extracted one way or another.. If your application needs to call a 3rd party service like openAI, the only solution to safely not leak your API key is to have your app only communicate with a backend you own and call the openAI from there. OpenAI allows revoking leaked keys. If you did include your API key in a client-side application, update your app to use a backend for openAI API communication, use a fresh key and revoke the old key when your update ships (or if you value security over functionality then revoke the key before you ship the update).
- hospitalJail 3y ago
  > is to have your app only communicate with a backend you own and call the openAI from there. I'm a bit baffled anyone puts anything secret on software people are using. This service needs to be online anyway. Anyway, seems like a lazy programmer thing.
  - judge2020 3y ago
    Running a server even to proxy requests takes a lot of work, since you now need your own auth system and have to manage scaling. If you take the plunge, a serverless architecture like Cloudflare Workers makes scaling automatic, but you still have to do some heavy lifting to either have an API key or auth system and abuse protections (otherwise they just spam your API instead of directly stealing your OpenAI api key).
    - counttheforks 3y ago
      You probably don't need to scale if all you're doing is auth and proxying requests. If you get to the point where you do need to scale, you can probably afford to figure it out.
    - psychphysic 3y ago
      Er.... Just ask gpt4 to how to do it obviously.
ohgodplsno 3y ago
Note: it's probably not just iOS/MacOS apps. Android apps are equally vulnerable to this if you're brazen enough to dump your key in any .xml file. Or in your code if you just run strings on it.
kgeist 3y ago
There's a lot of sites leaking OpenAI keys on the frontend as well, including some projects posted here on HN. I contacted one such dev back in February, they said they would fix it ASAP, and it's still not fixed.
tikkun 3y ago
If you want a solution that isn't perfect, but is at least slightly better: Store the key in your code but in a basic encrypted string, and then decrypt it at runtime. Yes, it's still easy to get if someone is motivated, but it's a lot harder to read the machine calls figuring out what method was used to encrypt the string (make it a method that can't be figured out from only the encrypted string), than it is to read the plaintext key from the Plist. Bad in theory, helpful in practice.
- weird-eye-issue 3y ago
  Fine for some secrets but I'd never do with something like an OpenAI key. Somebody could blow through your entire months's usage allowance before you notice anything
- andrewmunsell 3y ago
  As others have mentioned in the thread, this doesn't guard against a MITM proxy and it'd take a couple minutes to defeat this. You're much better off proxying calls from your own server API, having proper rate limits and authentication, and a strict API surface that doesn't permit arbitrary calls to whatever APIs you depend on
RichieAHB 3y ago
You could probably remove the OpenAI qualifier from this finding, but I guess it makes it more relevant.
gumballindie 3y ago
Why dont they use a server for forwarding requests to openai and not exposing any keys? That way they can easily ab test various ai engines and secure their keys. It is known.
lcfcjs 3y ago
[dead]
simonw 3y ago
Things I'd like to see from OpenAI API keys: - Unlimited, or at least a much higher limit - right now they are restricted to 5 - Ability to set a time limit on a key - I'd like to create a new key that's only good for the next hour when I try out a new thing that asks me to paste in an API key - Abiliy to set a budget for an API key. Giving an app a key with a $5 total budget - or $10/month or whatever - would be really neat. - OAuth support. Let me OAuth connect an app with my OpenID account - then I don't have to know what an API key is, I can grant it permission to spend my API credits, and I can revoke access later - Let me see how much money each API key has spent - An option to log everything an app does with my API key would be cool - I already have ChatGPT logs and rely on them all the time, but having that for other random applications would be excellent.
hijohnnylin 3y ago
For some developers, this is sort of intentional. The reason is at least twofold: 1) Calling OpenAI directly is one less hop, so user gets lower latency 2) Not having to set up / maintain a backend server = get to market faster There are some very popular GPT apps recently that are obviously putting their API keys on the client side - won't name them but they've been featured quite a bit. The downside is not as bad as people think. Worst case, someone takes your key and what, plugs it into their own app, costing you a few bucks? - OpenAI keys have a hard budget limit that requires manual approval by OpenAI anyway - Not much privacy risk - unlike other API keys, OpenAI APIs don't allow you retrieve previous data AFAIK. There are some APIs to fine-tune models, but I seriously doubt any of these consumer apps are doing this now. - You can just create a new version later and revoke the old key. And now you've broken the thief's app. My guess is the developers were well aware of the tradeoffs. Just felt it was more important to get to market faster, than to batten down all the hatches. They're probably right?
outcoldman 3y ago
Lol. Posted about it a month ago. And reported that to some devs. https://twitter.com/outcoldman/status/1636742564887011329?s=46&t=9eTW5Y7sIoYqz8AXDqdBcg https://twitter.com/outcoldman/status/1636742564887011329?s=... Nothing is going to change.
dantetheinferno 3y ago
One wonders if ChatGPT could self-replicate - using these API keys as a bootstrap. Lots of individual users computes would certainly lower the threshold for being cut off for billing reasons.
curiousbean 3y ago
It's much easier to open ZAP/Burp and intercept iOS/Android traffic to grab API keys.
kaishin 3y ago
As an iOS developer I’ve learned to never put 3rd party secrets in the app. I typically go with a proxy backend server and attach a request-unique nonce that’s created using an obfuscated secret key stored as an array of integers on the client.

140 points by archiv 3y ago

Even if you store API keys in code inside a distributed binary, isn't it pretty simple for users to mitmproxy and view API requests containing there keys sent? There's no real way to control API keys given out to users - if you want to hide them, you just have to proxy requests instead.

mrbombastic 3y ago
Some people do cert pinning to prevent this, but generally yes it is pretty simple.
- thewataccount 3y ago
  You can't do cert pinning if you're using the openai api directly though? That only applies for internal api calls, at which point the requests/binary won't contain the openai key?
  - mrbombastic 3y ago
    Yeah you are correct, only really can be done with apis you can control, even then it is a pain bc you need the certs and app in lockstep. But for example if your proxy api used an api key and you wanted an additional layer of security. Edit: sibling comment is interesting for an approach that might remove low hanging fruit
  - varenc 3y ago
    Ehh, I don't think it'd be that hard to implement cert pinning against OpenAI's APIs. You just need some very permissive pinning, where you require any publicly trusted CA, to prevent MITM attacks. Basically only trust the root CAs a phone already trusts by default. You don't need coordination between the server and your client to implement this. All you have to do is prevent your TLS calls from trusting any certs signed by manually trusted CAs that Proxyman/Charles/etc might have had the user add. Of course, that'll only delay the API keys leaking. With a jailbroken iPhone and Frida you can effectively disable cert pinning checks. Or extract the keys from memory, or binary analysis, etc.
    - thewataccount 3y ago
      > All you have to do is prevent your TLS calls from trusting any certs signed by manually trusted CAs that Proxyman/Charles/etc might have had the user add. Yeah but I have certs signed by trusted root authorities a la letsencrypt?
      - varenc 3y ago
        The letsencrypt root CA is included in this. If you trust only a device’s default trusted CA all letsencrypt certs will work. Also they don’t have their own root CA: https://letsencrypt.org/certificates/ https://letsencrypt.org/certificates/
        thewataccount 3y ago
        I'm dumb and realized I can get a letsencrypt cert but the domain won't match.....
KMnO4 3y ago
There's a really good iOS app called Proxyman[0] (the Mac app is also excellent) that lets you view HTTP requests that apps make from inside iOS. If you're curious about this sorta stuff, I definitely recommend checking it out. [0]: https://proxyman.io/ios https://proxyman.io/ios

ncann 3y ago

Isn't API key pretty much can always be recovered client side? If API key is hardcoded in the app it's trivial to get. If it's dynamically fetched it's also trivial to intercept the network call. I don't think there's anything you can do to prevent that.

speedgoose 3y ago
You can have a server side proxy for authenticated and authorised users, with a rate limit.
UltimateEdge 3y ago
The replies suggest storing the API key on a backend which you control.
devit 3y ago
Well, the safe approach is to either have the user provide their own API key, or to perform requests on the backend (after verifying the user has an account, correctly authenticated, has paid and deducting from their quota). Or you could generate a per-user subkey with a quota if the upstream service supports that.
kgeist 3y ago
OpenAI API has a larger surface area than just completions. You can retrieve files uploaded through it, you can generate images, you can use the embedding API, you can use a different, more expensive model (GPT4) - and make as many calls as you wish. With a backend, you can restrict what is allowed, add your own rate limiting, you can rotate keys etc.
thewataccount 3y ago
Yeah you can try obfuscating it, but in general you need to assume plain text access to anything you send the client. The only exceptions are basically some really hardcore DRM like denuvo and hardware certificated drm. Even those aren't safe from deteremined people.
numpad0 3y ago
From comments here, I suppose you could make your app cloud-based so your client app is just a UI and only your trusted backend has full access to the external API, and that sounds dystopian.

sharemywin 3y ago

I would think you would keep an api wrapper around it at least probably some kind of IP tracking and auth.

steve_adams_86 3y ago

I’m surprised people wouldn’t want to route requests through a backend so they could throttle traffic and prevent abuse from a place no one else can control. Perhaps they simply don’t know that’s a concern — they put a static API key for a service that costs money into a client side application. That seems unintentional, at least in regards to exposure to potential consequences.

binarymax 3y ago
Keeping creds safe is basic security practice, and we have the eternal September of programmers - which now applies to inexperienced people using AI to cobble together software. So there’s going to be a lot of these mistakes made and learned as they acclimate.
- matisseverduyn 3y ago
  Noticing that comments like this are being consistently downvoted, yet there doesn't seem to be any sort of disagreement in the replies. Usually when people disagree online, they don't merely downvote, but also post their dissent. Interesting that it's suddenly become so "controversial" to suggest circumspection with respect to this subject...
  - hutzlibu 3y ago
    I didn't downvote, or noticed such a pattern, but was this comment really insightful? It just said, programming newbs will make newb misstakes. Yes and? I mean it is self evidently true. So yes, you can say it and the wording was not really condescending, but quite often to me it is just bashing beginners to feel more powerful and smart as an experienced and studied programmer, compared to those amateurs.
    - matisseverduyn 3y ago
      > Keeping creds safe is basic security practice... So there’s going to be a lot of these mistakes made and learned as they acclimate. This might not be insightful from your perspective, because you've thought about it before, but it needs to be said. Just like a NO DIVING sign in the shallow end, most people already know, but some people who don't know might not know if it isn't stated.

black3r 3y ago

You should never store ANY secret information (API keys, passwords, tokens, secret keys of any kind) in your application binary. It can always be extracted one way or another.. If your application needs to call a 3rd party service like openAI, the only solution to safely not leak your API key is to have your app only communicate with a backend you own and call the openAI from there. OpenAI allows revoking leaked keys. If you did include your API key in a client-side application, update your app to use a backend for openAI API communication, use a fresh key and revoke the old key when your update ships (or if you value security over functionality then revoke the key before you ship the update).

hospitalJail 3y ago
> is to have your app only communicate with a backend you own and call the openAI from there. I'm a bit baffled anyone puts anything secret on software people are using. This service needs to be online anyway. Anyway, seems like a lazy programmer thing.
- judge2020 3y ago
  Running a server even to proxy requests takes a lot of work, since you now need your own auth system and have to manage scaling. If you take the plunge, a serverless architecture like Cloudflare Workers makes scaling automatic, but you still have to do some heavy lifting to either have an API key or auth system and abuse protections (otherwise they just spam your API instead of directly stealing your OpenAI api key).
  - counttheforks 3y ago
    You probably don't need to scale if all you're doing is auth and proxying requests. If you get to the point where you do need to scale, you can probably afford to figure it out.
  - psychphysic 3y ago
    Er.... Just ask gpt4 to how to do it obviously.

ohgodplsno 3y ago

Note: it's probably not just iOS/MacOS apps. Android apps are equally vulnerable to this if you're brazen enough to dump your key in any .xml file. Or in your code if you just run strings on it.

kgeist 3y ago

There's a lot of sites leaking OpenAI keys on the frontend as well, including some projects posted here on HN. I contacted one such dev back in February, they said they would fix it ASAP, and it's still not fixed.

tikkun 3y ago

If you want a solution that isn't perfect, but is at least slightly better: Store the key in your code but in a basic encrypted string, and then decrypt it at runtime. Yes, it's still easy to get if someone is motivated, but it's a lot harder to read the machine calls figuring out what method was used to encrypt the string (make it a method that can't be figured out from only the encrypted string), than it is to read the plaintext key from the Plist. Bad in theory, helpful in practice.

weird-eye-issue 3y ago
Fine for some secrets but I'd never do with something like an OpenAI key. Somebody could blow through your entire months's usage allowance before you notice anything
andrewmunsell 3y ago
As others have mentioned in the thread, this doesn't guard against a MITM proxy and it'd take a couple minutes to defeat this. You're much better off proxying calls from your own server API, having proper rate limits and authentication, and a strict API surface that doesn't permit arbitrary calls to whatever APIs you depend on

RichieAHB 3y ago

You could probably remove the OpenAI qualifier from this finding, but I guess it makes it more relevant.

gumballindie 3y ago

Why dont they use a server for forwarding requests to openai and not exposing any keys? That way they can easily ab test various ai engines and secure their keys. It is known.

lcfcjs 3y ago

[dead]

simonw 3y ago

Things I'd like to see from OpenAI API keys: - Unlimited, or at least a much higher limit - right now they are restricted to 5 - Ability to set a time limit on a key - I'd like to create a new key that's only good for the next hour when I try out a new thing that asks me to paste in an API key - Abiliy to set a budget for an API key. Giving an app a key with a $5 total budget - or $10/month or whatever - would be really neat. - OAuth support. Let me OAuth connect an app with my OpenID account - then I don't have to know what an API key is, I can grant it permission to spend my API credits, and I can revoke access later - Let me see how much money each API key has spent - An option to log everything an app does with my API key would be cool - I already have ChatGPT logs and rely on them all the time, but having that for other random applications would be excellent.

hijohnnylin 3y ago

For some developers, this is sort of intentional. The reason is at least twofold: 1) Calling OpenAI directly is one less hop, so user gets lower latency 2) Not having to set up / maintain a backend server = get to market faster There are some very popular GPT apps recently that are obviously putting their API keys on the client side - won't name them but they've been featured quite a bit. The downside is not as bad as people think. Worst case, someone takes your key and what, plugs it into their own app, costing you a few bucks? - OpenAI keys have a hard budget limit that requires manual approval by OpenAI anyway - Not much privacy risk - unlike other API keys, OpenAI APIs don't allow you retrieve previous data AFAIK. There are some APIs to fine-tune models, but I seriously doubt any of these consumer apps are doing this now. - You can just create a new version later and revoke the old key. And now you've broken the thief's app. My guess is the developers were well aware of the tradeoffs. Just felt it was more important to get to market faster, than to batten down all the hatches. They're probably right?

outcoldman 3y ago

Lol. Posted about it a month ago. And reported that to some devs. https://twitter.com/outcoldman/status/1636742564887011329?s=46&t=9eTW5Y7sIoYqz8AXDqdBcg https://twitter.com/outcoldman/status/1636742564887011329?s=... Nothing is going to change.

dantetheinferno 3y ago

One wonders if ChatGPT could self-replicate - using these API keys as a bootstrap. Lots of individual users computes would certainly lower the threshold for being cut off for billing reasons.

curiousbean 3y ago

It's much easier to open ZAP/Burp and intercept iOS/Android traffic to grab API keys.

kaishin 3y ago

As an iOS developer I’ve learned to never put 3rd party secrets in the app. I typically go with a proxy backend server and attach a request-unique nonce that’s created using an obfuscated secret key stored as an array of integers on the client.