4 ms·
Another paper in a long series that confuses "our tests against currently available LLMs tuned for specific tasks found that they didn't perform well on our tas
by hiddencost 3y ago
Another paper in a long series that confuses "our tests against currently available LLMs tuned for specific tasks found that they didn't perform well on our task" with "LLMs are architecturally unsuitable for our task".
- famouswaffles 3y agoIt's a weird title anyway. I was expecting worse results but GPT-4V is close to or matching Human median performance on most of the tests besides the multimodal "Intuitive Psychology" tests.
- dsr_ 3y agoThere is no reason to believe (evidence) that any meaning ascribed to an LLM's utterances comes from the LLM rather than being pareidolia. If you've found some, please let everyone know.
- kkzz99 3y agoYou would first have to define what you mean with "meaning" and "pareidolia" in this context.
- Tadpole9181 3y agoThere is no reason to believe (evidence) that any meaning ascribed to anyone but me's utterances comes from the person rather than being pareidolia. If you've found some, please let everyone know.
- mistermann 3y agoIt's worse: there is technically no way to know if there is in fact no evidence, it is a colloquial phrase that people cannot think twice about.
- LesZedCB 3y agoall text ever written has only ever had meaning imbued by the reader (including this text).
- marcosdumay 3y agoOur tests against current cars found that they didn't perform well on transatlantic flights... But who knows what the future holds? Maybe we should test them again next year. LLM names an specific product, aimed at solving an specific problem.
- LesZedCB 3y agoour tests of combustion engine driven crankshafts connected to a spinning mechanism can perform well in both transatlantic flights and cross country road trips.
- AnimalMuppet 3y agoThat's not the hard part about building a working airplane, though...
- TeMPOraL 3y agoYes, but as the saying goes, anything can fly if you strap a powerful enough engine to it... so we've demonstrated basic capability, and the rest is now optimizing its performance a couple orders of magnitude.
- AnimalMuppet 3y agoNot at all. "Anything can fly", but not everything can fly without killing you. The fundamental problem is three-axis control with stability. A source of power does exactly nothing to solve that.
- TeMPOraL 3y agoNo, that just means you need a bigger engine. (And, perhaps, an inertial dampener, to survive the gees.) Think of it this way: the more thrust you get, the farther you'll go before control problems start to manifest. Keep pushing, and eventually the rest of the craft becomes a rounding error in the math :).