3 ms·
It doesn't have anything to do with tokenization. You can define binary addition using symbols, e.g. a and b, and provide properly tokenized strings to GPT-4. G
by rcme 3y ago
It doesn't have anything to do with tokenization. You can define binary addition using symbols, e.g. a and b, and provide properly tokenized strings to GPT-4. GPT-4 appears to solve the arithmetic puzzles for a few bits, but quickly falls apart on larger examples.
- iliane5 3y agoWhat I was saying is that because you need to go out of your way to make sure it's tokenized properly, I wouldn't be surprised if there are enough non properly tokenized examples in the dataset. If that was the case, it would make it difficult to generalize these concepts.