3 ms·
> nearly as accurate as a human book keeper Anything to avoid using the metric system. Though seriously, what is this metric? Why would I care if an LLM is ac
by malfist 3mo ago
> nearly as accurate as a human book keeper
Anything to avoid using the metric system.
Though seriously, what is this metric? Why would I care if an LLM is accurate as a human bookkeeper? Humans aren't exactly known for perfect recall.
- wat10000 3mo agoYou'd care if you had a human bookkeeper and you were considering replacing them with this company's AI bookkeeper.
- altruios 3mo agoWhy would I want both a less accurate book keeper and to incur all of the liability of doing the books myself?!
- murderfs 3mo agoPresumably your current book keeper is not your slave, and you have to pay them...
- altruios 3mo ago...again: liability is the key issue. Cost savings a not exactly an isolated issue here.
- murderfs 3mo agoThen why only have one human bookkeeper? Surely two would be better, since you can compare their results. But then, perhaps you should hire three, so you can figure out which one is right.
- CamperBob2 3mo agoGot some bad news for you about that "liability" thing: it's always been on you. Read the fine print on your tax forms sometime.
- onraglanroad 3mo agoIt's not less accurate. As commented above, the control knew they were being tested against the machine, so made sure to be super careful. In everyday life the human is less careful, and the machine costs 1% of the human.
- wat10000 3mo agoBecause it's way cheaper. Not saying it's a good tradeoff, but that appears to be their pitch.
- adamkurkiewicz 3mo agoHey, author of the blog post here. I was one of the human book-keepers for this benchmark (the preparer; my co-founder verified the VAT submission once ready), and given that at the time of doing this I knew I was eventually going to use this data for evaluating the models, I was super careful. So I guess this is a "good book-keeper". In the previous company our book-keepers made lots of mistakes; some serious enough that we had to restate our company's accounts.
- infecto 3mo agoIt’s a service that provides an AI bookkeeper so it’s a pretty relevant metric.