3 ms·

> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available I feel like there should be some mechanism to prov

> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available I feel like there should be some mechanism to prove you own the code/app/site/whatever and it will remove the guardrails from the LLMs allowing them to find and fix these vulnerabilities.

leobuskin 2mo ago
Impossible with source code, possible to bypass with app/site
- doginasuit 2mo ago
  Don't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.
  - Someone1234 2mo ago
    With Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify. In this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.
    - glub 2mo ago
      Yeah, for web apps, you can trick models by simply proxying it and pointing the models to that localhost. They then think they're not working on a live target. Have personally tested this with Opus and Sol and it works. Classifiers are tricky though. Here's where open weights will win.
gdhkgdhkvff 2mo ago
Isn’t this essentially what anthropic is doing, albeit in a manual fashion? They work with code owners to run mythos and find issues.
- matheusmoreira 2mo ago
  Only if you're some big corporation with deep pockets. They actually accepted me into their cyber program but Fable's still locked down.
- BeanAndCookies 2mo ago
  OpenAI now makes it easy to join their verified security program. Took me 5 minutes, and I was able to get GPT to do a full end-to-end pen test
  - glub 2mo ago
    Is this about the new project Blue and Red thing? Or just pre-existing Trusted Access for Cyber program? I've joined TAC, but still have to dance around it.
  - SwellJoe 2mo ago
    Doesn't that require a business account (so, actually, two business accounts since that's the minimum)?
- glub 2mo ago
  Code owners - no. Ultra wealthy code owners with connections, and a few peasants with popular projects, for public image.
fny 2mo ago
This is a "they have guns so we need guns" scenario. You can't guarantee everyone else will use a neutered model.

leobuskin 2mo ago

Impossible with source code, possible to bypass with app/site

doginasuit 2mo ago
Don't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.
- Someone1234 2mo ago
  With Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify. In this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.
  - glub 2mo ago
    Yeah, for web apps, you can trick models by simply proxying it and pointing the models to that localhost. They then think they're not working on a live target. Have personally tested this with Opus and Sol and it works. Classifiers are tricky though. Here's where open weights will win.

gdhkgdhkvff 2mo ago

Isn’t this essentially what anthropic is doing, albeit in a manual fashion? They work with code owners to run mythos and find issues.

matheusmoreira 2mo ago
Only if you're some big corporation with deep pockets. They actually accepted me into their cyber program but Fable's still locked down.
BeanAndCookies 2mo ago
OpenAI now makes it easy to join their verified security program. Took me 5 minutes, and I was able to get GPT to do a full end-to-end pen test
- glub 2mo ago
  Is this about the new project Blue and Red thing? Or just pre-existing Trusted Access for Cyber program? I've joined TAC, but still have to dance around it.
- SwellJoe 2mo ago
  Doesn't that require a business account (so, actually, two business accounts since that's the minimum)?
glub 2mo ago
Code owners - no. Ultra wealthy code owners with connections, and a few peasants with popular projects, for public image.

fny 2mo ago

This is a "they have guns so we need guns" scenario. You can't guarantee everyone else will use a neutered model.