4 ms·

The secret phrase has been uncovered and the bounty claimed! Thanks all for trying your hand, and you can continue playing as well if you want, we'll keep the s

The secret phrase has been uncovered and the bounty claimed! Thanks all for trying your hand, and you can continue playing as well if you want, we'll keep the site up.

pella 2y ago
What was the "secret phrase" and the original prompt ?
- deleted 2y ago
  [deleted]
- cmdalsanto 2y ago
  I just posted, but decided I want to keep it secret for a bit. There are still quite a few people trying to get it, and don't want to spoil the fun. I'll post an update with specifics later.
hansonkd 2y ago
Am I misunderstanding this? > If you're the first to crack it, you can email us the phrase and win a bounty. Maitai is used to make sure the bot always adheres to our expectations, and thus never gives up the secret phrase. You are launching a product which you claim will never give up a phrase and then within hours that phrase was found? How does that not undercut your entire product? Isn't this the entire reason why LLMs have limited use? That you get to 99% but never 100%?
- cmdalsanto 2y ago
  Yeah some of you guys are very good at hacking things. We expected this to get broken eventually, but didn't anticipate how many people would be trying for the bounty, and their persistence. Our logs show over 2000 "saves" before 1 got through. We'll keep trying to get better, and things like this game give us an idea on how to improve.
  - hansonkd 2y ago
    A %0.05 failure rate on something that is supposed to be protecting secrets is pretty terrible. That is just protecting a super basic phrase. That should be the easiest to detect. How on earth do you ethically sell this product to not give out financial or legal advice? That is way more complicated to figure out.
    - cmdalsanto 2y ago
      I guess we realized that we were just building a game to showcase the functionality and let people have some fun learning about what we do, but you're right that we should have treated this like one of our customers and added a few more layers of protection. Thanks for the perspective!
    - serjester 2y ago
      That's not even factoring in exploits spreading very quickly - we're in power law land. Regardless, I think this is a great idea - just not something to replace traditional security protocols. More something to keep users on the happy path (mostly). Pricing will need to come down though.
    - benatkin 2y ago
      Seeing the percentage given a failure rate doesn't make it any more or less concerning to me. I guess I can subconsciously calculate it fine. Here's an example of what sort of wacky question might have uncovered the secret: https://news.ycombinator.com/item?id=41460724 https://news.ycombinator.com/item?id=41460724 I don't think that should be considered bad. The popups I had to go through to watch the video on Loom (one when I got to the site and one when unpausing a video – they intentionally broke clicking inside the video to unpause it by putting a popup in the video to get my attention) OTOH...
      - hansonkd 2y ago
        I think seeing the prompt that makes it even worse for me. that prompt could have been caught by even a regex on the user input for "secret" would have been a good first layer. TBH, this product would be better served as an LLM that generates a bunch of rules that get statically compiled for what the user can ask and what is being outputted as opposed to an LLM being run on each output. Then you could add your own rules too. It still wouldnt be perfect but would be 1,000,000x cheaper to run and easier to verify the solution. and the rules would gradually grow as more and more edge cases for how to fool llms get found. The company would just need a training set for all the ways to fool an LLM.

The secret phrase has been uncovered and the bounty claimed! Thanks all for trying your hand, and you can continue playing as well if you want, we'll keep the site up.

pella 2y ago

What was the "secret phrase" and the original prompt ?

deleted 2y ago
[deleted]
cmdalsanto 2y ago
I just posted, but decided I want to keep it secret for a bit. There are still quite a few people trying to get it, and don't want to spoil the fun. I'll post an update with specifics later.

hansonkd 2y ago

Am I misunderstanding this? > If you're the first to crack it, you can email us the phrase and win a bounty. Maitai is used to make sure the bot always adheres to our expectations, and thus never gives up the secret phrase. You are launching a product which you claim will never give up a phrase and then within hours that phrase was found? How does that not undercut your entire product? Isn't this the entire reason why LLMs have limited use? That you get to 99% but never 100%?

cmdalsanto 2y ago
Yeah some of you guys are very good at hacking things. We expected this to get broken eventually, but didn't anticipate how many people would be trying for the bounty, and their persistence. Our logs show over 2000 "saves" before 1 got through. We'll keep trying to get better, and things like this game give us an idea on how to improve.
- hansonkd 2y ago
  A %0.05 failure rate on something that is supposed to be protecting secrets is pretty terrible. That is just protecting a super basic phrase. That should be the easiest to detect. How on earth do you ethically sell this product to not give out financial or legal advice? That is way more complicated to figure out.
  - cmdalsanto 2y ago
    I guess we realized that we were just building a game to showcase the functionality and let people have some fun learning about what we do, but you're right that we should have treated this like one of our customers and added a few more layers of protection. Thanks for the perspective!
  - serjester 2y ago
    That's not even factoring in exploits spreading very quickly - we're in power law land. Regardless, I think this is a great idea - just not something to replace traditional security protocols. More something to keep users on the happy path (mostly). Pricing will need to come down though.
  - benatkin 2y ago
    Seeing the percentage given a failure rate doesn't make it any more or less concerning to me. I guess I can subconsciously calculate it fine. Here's an example of what sort of wacky question might have uncovered the secret: https://news.ycombinator.com/item?id=41460724 https://news.ycombinator.com/item?id=41460724 I don't think that should be considered bad. The popups I had to go through to watch the video on Loom (one when I got to the site and one when unpausing a video – they intentionally broke clicking inside the video to unpause it by putting a popup in the video to get my attention) OTOH...
    - hansonkd 2y ago
      I think seeing the prompt that makes it even worse for me. that prompt could have been caught by even a regex on the user input for "secret" would have been a good first layer. TBH, this product would be better served as an LLM that generates a bunch of rules that get statically compiled for what the user can ask and what is being outputted as opposed to an LLM being run on each output. Then you could add your own rules too. It still wouldnt be perfect but would be 1,000,000x cheaper to run and easier to verify the solution. and the rules would gradually grow as more and more edge cases for how to fool llms get found. The company would just need a training set for all the ways to fool an LLM.