> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive. That’s because the idea of it being a super

> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive. That’s because the idea of it being a super human intelligence (an undefined metric) is being sold. So you have to lie and say “it’s amazing, it’s going to change everything”. If I tell you “it’s okay and is often wrong” you wouldn’t buy my product would you? This is just to say I can’t blame that on easy/hard task agency, specifically. === addendumb === “it’s okay and is often wrong” Sounds like working with my junior coworker who I don’t enjoy pairing with. If I said “it’s impressive how the results are to the level of a junior engineer” you sell me on your product.