I can tell you who would love to hear your answer to that question. Me for starters. If it works, I can quit. Next up are Karpathy and the CTO of OpenAI. Arou

I can tell you who would love to hear your answer to that question. Me for starters. If it works, I can quit. Next up are Karpathy and the CTO of OpenAI. Around July and September both talked about production challenges. AI ops was the largest subcategory of fall YC startups. Every single ml and LLM ops individual who gets far enough deals with evals. I don’t know, but maybe - just maybe- the issue is that people don’t understand themselves enough, to avoid assuming too much of those emergent properties. As I recall there was also a paper that pointed out the issues with how LLMs are measured, and that the emergence of properties was not a step change once the tests were updated. Edit- found it: https://hai.stanford.edu/news/ais-ostensible-emergent-abilities-are-mirage https://hai.stanford.edu/news/ais-ostensible-emergent-abilit...