4 ms·
> My two favourite hypothetical questions regarding this used to be: > If I'm running Codex and one of my API keys accidentally gets consumed in the context, w
by tosh 1mo ago
> My two favourite hypothetical questions regarding this used to be:
> If I'm running Codex and one of my API keys accidentally gets consumed in the context, what are the chances that someone else might ask for an API key in the future and get mine back? (I asked someone at OpenAI once and they called this the "regurgitation" problem and assured me that they take great pains to prevent that... but wouldn't describe how.)
> If I brainstorm with ChatGPT about potential new directions for my company, what's the chance that information might be exposed to a competitor in six months' time who asks "what might company X plan to do next"?
> My new preferred hypothetical for this is:
> If I use ChatGPT to help me partially solve a Millennium Prize problem, what are the chances that my work will influence training such that a later model helps someone else solve it first?
- weinzierl 1mo agoMaybe just a rumor of a high value target having their API keys accidentally consumed in the context...
- feverzsj 1mo agoLLM can't be trained that easily. More like actual human are checking your logs and stealing valuable things from you.
- grey-area 1mo agoOr searching anonymised logs for mentions of this problem and using that as part of the context or training. This would work just as well and have plausible deniability.
- rakejake 1mo agoExactly! This is the real Occam's Razor explanation.
- rzzzt 1mo agoThey wouldn't appear in weights but could be added to the context. My conversations regularly go "regarding your Java problem"... which was a separate item in the history from earlier. As long as I only see these (and nobody else sees mine), it can be helpful.
- FeepingCreature 1mo agoLLMs can learn from one sample.