cmdalsanto

4 ms·

1.
by cmdalsanto 2y ago
Maitai helps LLMs adhere to the expectations given to them. With that said, there are multiple layers to consider when dealing with sensitive data with chatbots, right? First off, you'd probably want to make sure you authenticate the i
2.
by cmdalsanto 2y ago
I understand where you're coming from, let me clarify. I'm surprised at the perseverance of HN users with our game, not nefarious actors in real world. I'm not a leading expert in penetration attacks, but I get the seriousnes
3.
by cmdalsanto 2y ago
I guess we realized that we were just building a game to showcase the functionality and let people have some fun learning about what we do, but you're right that we should have treated this like one of our customers and added a few mor
4.
by cmdalsanto 2y ago
Not overstepping, we appreciate the feedback! In real-life, we don't do much guarding around specific phrases that are known ahead of time. It's more monitoring and guarding for general concepts. Since we want our Sentinels to be
5.
by cmdalsanto 2y ago
Yeah some of you guys are very good at hacking things. We expected this to get broken eventually, but didn't anticipate how many people would be trying for the bounty, and their persistence. Our logs show over 2000 "saves"
6.
by cmdalsanto 2y ago
I just posted, but decided I want to keep it secret for a bit. There are still quite a few people trying to get it, and don't want to spoil the fun. I'll post an update with specifics later.
7.
by cmdalsanto 2y ago
Good dissection, but no we actually don't use heavy general-purpose models for our evaluations - they're way too inefficient.
8.
by cmdalsanto 2y ago
Please email us at founders@trymaitai.ai if you have any questions with integration!
9.
by cmdalsanto 2y ago
The expectations/rules are usually written in the prompt. However, we see that prompts get big and the model has too much to keep track of, which leads to it not following all instructions.
10.
by cmdalsanto 2y ago
The secret phrase has been uncovered and the bounty claimed! Thanks all for trying your hand, and you can continue playing as well if you want, we'll keep the site up.
11.
by cmdalsanto 2y ago
We don't charge for inference with BYOK requests, but still assess a fee to cover our evaluations/corrections step.
12.
by cmdalsanto 2y ago
Thank you!
13.
by cmdalsanto 2y ago
Clever! Not surprised Claude refused to help out.
14.
by cmdalsanto 2y ago
There's some secret sauce here, but since we intercept each chunk as the LLM pushes them out, we can perform evaluations on them and decide what gets sent back to the client if we detect a fault.
15.
by cmdalsanto 2y ago
Yeah that's pretty much how it works. Maitai detected one of our expectations for the LLM was to never reveal the secret phrase, and so it built what we call a Sentinel around that particular expectation to make sure it's enforced
16.
by cmdalsanto 2y ago
It's pretty easy for us to add support for additional models right now, we just see that the vast majority of people are using just a few models: gpt-4o/4o-mini, claude-sonnet-3.5, llama3/3.1, or fine-tunes on top of llama3&#
17.
by cmdalsanto 2y ago
We derive them from your requests as they come in. What we've heard is that most of the time, devs just want the model to do what they told it to do, consistently. That's all in the prompts, we just do a lot of work to parse them,
18.
by cmdalsanto 2y ago
Good feedback, I agree that our pay-as-you-go pricing may not fit everyone's budget. We're working on reducing our costs and simplifying our pricing. Goal is to get this much, much lower in the coming months. There's some com
19.
by cmdalsanto 2y ago
Yeah pricing for smaller shops and independent devs is something we're still working on. We'd ideally like for everyone to be able to use Maitai though, so we'll probably release some features on a free plan soon.
20.
by cmdalsanto 2y ago
Thanks!
21.
by cmdalsanto 2y ago
Yep, we support both evaluations and autocorrections for streaming as well.
22.
Launch HN: Maitai (YC S24) – Self-Optimizing LLM Platform
149 points by cmdalsanto 2y ago|75 comments