5 ms·
The article makes it sound like Pelicans riding a bike are solved, and we can move on to something else. Nothing to do with excitement or wonder of the tech, it
by didibus 2mo ago
The article makes it sound like Pelicans riding a bike are solved, and we can move on to something else. Nothing to do with excitement or wonder of the tech, it just claims one thing and I feel I've not seen that be the case so the claim seems incorrect to me.
- nomel 2mo ago> We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". First sentence of the post is clear that it's not solved.
- didibus 2mo agoMight be my reading comprehension, but that reads to me like it's implying we're leaving the territory because it's solved and now need something harder.
- nomel 2mo ago"Starting to leave" is not the same as "leave" or "left". When I start to leave my house in the morning, I have not yet gone through the door, but I'm on my way. I walk towards the door to leave the house. I'm leaving the house as I pass through the door. I look back and can see that I've left the house.
- didibus 2mo agoBut why are you starting to leave your house? Likely because you're ready to leave no? To me that's where it was implying that models are ready to leave that benchmark behind, and starting the process is coming up with a new benchmark. Anyways, if it's just semantic misunderstanding, I have no issue. That means the fact is, models still can't draw a pelican riding a bike? With maybe the exception of Fable 5?
- nomel 2mo agoIn this case, the preparation for "starting to leave" this benchmark would be getting a new benchmarks ready, because this one appears it will soon be obsolete.