3 ms·
Need to run this experiment on a problem that is t already on its training set.
by lupire 2y ago
Need to run this experiment on a problem that is t already on its training set.
- passwordoops 2y agoShhhh... Don't ruin it
- aulin 2y agoReminds me when I used to ace Ancient Greek translation tests (it's a thing in Italy) by looking up whole translated sentences listed as usage examples in the dictionary
- navane 2y agoIt's the equivalent of cramming for a test, or memorizing leete code -- not very useful but very humane. Imagine if that's the direction this goes, finally we make humane ai, but it is as opportunitistic and deceitful as we are, and not really that smart.
- sabrina_ramonov 2y agosometimes for physics/math exams, we'd get to create our own 1 pager cheat sheet to use. I'd just cram tons of actual problems/solutions on there, then scan for similarity.
- krsrhe 2y ago[dead]
- bearjaws 2y agoIs there any good literature on this topic? I feel like math is naturally one of the easiest sets of synthetic data we can produce, especially since you can represent the same questions multiple ways in word problems. You could just increment the numbers infinitely and generate billions of examples of every formula. If we can't train them to be excellent at math, what hope do we ever have at programming or any other skill?
- cchance 2y agoNot in the training set? The dataset is ALL OF THE INTERNET, i'd love you to find something it hasn't seen before.
- deleted 2y ago[deleted]
- ec109685 2y agoYes, I feel like asking an LLM this type of problem is like giving someone access to Google during an open book math test. It’d be interesting to obfuscate it so that the concepts are all the same but the subject matter would be hard to google for. Another failure mode for LLM’s is if you take something with a well-known answer and add a twist to it.