5 ms·

I've been looking for this exact thing. A couple questions: Are your agents good at testing other agents? e.g. I want your agent to ask our agent a few questio

I've been looking for this exact thing. A couple questions: Are your agents good at testing other agents? e.g. I want your agent to ask our agent a few questions and complete a few UI interactions with the results. How do you handle testing onboarding flows? e.g. I want your agent to create a new account in our app (https://www.definite.app/ https://www.definite.app/) and go thru the onboarding flow (e.g. add Stripe and Hubspot as integrations).

ttamslam 11mo ago
> Are your agents good at testing other agents? e.g. I want your agent to ask our agent a few questions and complete a few UI interactions with the results. I'd say this is one of our strong suits I think, specifically the UIs tend to be easy to navigate for browser agents, and the LLM as a judge offers pretty good feedback on chat quality and it can inform later actions. (I'd be remiss not to mention though that a good LLM eval framework like Braintrust is probably the best first line though) > How do you handle testing onboarding flows? We can step through most onboarding flows if you start from logged out state & give the context it'll need (i.e. a stripe test card, etc.) That said though, setting up integrations that require multi-page hops is still a pain point in our system and leaves a lot to be desired. Would love to talk more about your specific case and see if we can help! founders@propolis.tech
- tommy18 11mo ago
  Then how do you compare with braintrust? Aren’t they doing same thing for Agents?
kodefreeze 11mo ago
I just did this test with our web QA agent - kodefreeze.com, it was able to test creating an account until it reached the screen that requires email confirmation. Support for being able to receive email/custom actions is on our roadmap, but would love to see if getting this far would be valuable to you. The test was with the email=test@kodefreeze.com.

ttamslam 11mo ago

> Are your agents good at testing other agents? e.g. I want your agent to ask our agent a few questions and complete a few UI interactions with the results. I'd say this is one of our strong suits I think, specifically the UIs tend to be easy to navigate for browser agents, and the LLM as a judge offers pretty good feedback on chat quality and it can inform later actions. (I'd be remiss not to mention though that a good LLM eval framework like Braintrust is probably the best first line though) > How do you handle testing onboarding flows? We can step through most onboarding flows if you start from logged out state & give the context it'll need (i.e. a stripe test card, etc.) That said though, setting up integrations that require multi-page hops is still a pain point in our system and leaves a lot to be desired. Would love to talk more about your specific case and see if we can help! founders@propolis.tech

tommy18 11mo ago
Then how do you compare with braintrust? Aren’t they doing same thing for Agents?

kodefreeze 11mo ago

I just did this test with our web QA agent - kodefreeze.com, it was able to test creating an account until it reached the screen that requires email confirmation. Support for being able to receive email/custom actions is on our roadmap, but would love to see if getting this far would be valuable to you. The test was with the email=test@kodefreeze.com.