2 ms·
"The task consists of going from English-language specifications to Wolfram Language code. The test cases are exercises from Stephen Wolfram's An Elementary Int
by botro 2y ago
"The task consists of going from English-language specifications to Wolfram Language code. The test cases are exercises from Stephen Wolfram's An Elementary Introduction to the Wolfram Language."
I think this benchmark would really only tell me whether Wolframs book was in the training data.
- smusamashah 2y agoYeah, may be should skip that benchmark.
- bufferoverflow 2y agoIt's available online in HTML form, for free: https://www.wolfram.com/language/elementary-introduction/3rd-ed/01-starting-out-elementary-arithmetic.html https://www.wolfram.com/language/elementary-introduction/3rd...