4 ms·
Interesting that they built their own benchmark and that it primarily features images from vehicles. Tesla overlap? > ... we are introducing a new benchmark, R
by sebmellen 3y ago
Interesting that they built their own benchmark and that it primarily features images from vehicles. Tesla overlap?
> ... we are introducing a new benchmark, RealWorldQA. This benchmark is designed to evaluate basic real-world spatial understanding capabilities of multimodal models. While many of the examples in the current benchmark are relatively easy for humans, they often pose a challenge for frontier models.
> The initial release of the RealWorldQA consists of over 700 images, with a question and easily verifiable answer for each image. The dataset consists of anonymized images taken from vehicles, in addition to other real-world images ... RealWorldQA is released under CC BY-ND 4.0.
Will be interesting to see the feedback once someone has a change to look into the dataset (https://data.x.ai/realworldqa.zip https://data.x.ai/realworldqa.zip).
Side note — I'm very impressed with their "Explaining a meme" example.
- pr337h4m 3y agoLooks like Optimus is an intended use case.
- drak0n1c 3y agoOptimus for indoor data collection and model deployment, Tesla for outdoor.
- blackeyeblitzar 3y agoOpenAI and others seem to be focused on a brute force approach towards achieving AGI. Others, such as Yann LeCun, have advocated for intelligence that can build understanding of the world similar to how humans learn. This seems like an approach in that direction?
- p1esk 3y agoThis seems like an approach in that direction? That makes it seem so? Aren’t they trying to catch up with OpenAI using exactly the same models and training methods that OpenAI uses?
- hackerlight 3y agoThis is what Elon wanted for OpenAI before he left the board according to the emails. He wanted them to become for-profit under Tesla, probably to assist with self-driving.