2 ms·
> Astra for Law passed the evaluation’s overall correctness check on 54.0% of questions. How is this a product that you are selling?
by CobrastanJorji 2mo ago
> Astra for Law passed the evaluation’s overall correctness check on 54.0% of questions.
How is this a product that you are selling?
- prodigycorp 2mo agoHelp me understand your consternation.
- fragmede 2mo ago54.0% isn't particularly high.
- prodigycorp 2mo agoWhat I'm trying to say which doesnt seem obvious is that all models are x% correct at benchmarks until they get saturated, and then new benchmarks get made.
- jambalaya8 2mo agowaiting for the actual Prodigy Corp. to show up and offer to buy your username out? <my own consternation, stated>
- calyhre 2mo agoIt’s barely any better than a coin flip
- CobrastanJorji 2mo agoWhat is a possible use case for a legal reasoning tool that is wrong 46% of the time?
- noisy_boy 2mo agoOn the promise of potential cost savings by means of firing and/or not hiring as much.