4 ms·
In their readme they mention `output: str, the answer to the instruction as generated by text-davinci-003`. And in the alpaca-data file there's this: {
by bestest 4y ago
In their readme they mention `output: str, the answer to the instruction as generated by text-davinci-003`.
And in the alpaca-data file there's this:
{
"instruction": "Perform the following arithmetic operation.",
"input": "(5+7)\*3.",
"output": "60."
},
Do we need human QA on the training data? It's contaminating the set! I wonder how incorrect data like this will be filtered out in the future. Unless we're ok with that kind of new math.
- JimmyAustin 4y agoYou can actually use AI to validate some of these answers! For questions involving math, it’s easy to get the LLM to write a problem to solve the problem, then use that to double check the result. For example, with the following prompt to ChatGPT: ‘’’ Can a program that solves the following problem accurately be written, and if so, write it in Python: {“Problem”: “Perform the following arithmetic operation." "input": "(5+7)3”} ‘’’ I got the answer: ‘’’ Yes, a program can be written to perform the arithmetic operation given in the input. Here is a Python implementation: input_str = "(5+7)3" # input string result = eval(input_str) # using eval function to evaluate the expression print(result) # output: 36 The eval() function in Python evaluates the expression given as a string and returns the result. In this case, the input string "(5+7)*3" is evaluated to 36. The program then prints the result using the print() function. ’’’