3 ms·
Albeit a bit condescending, you answer didn’t actually answer the OP’s question. So how could GPT-4 encode and decode base64? Was it explicitly trained on this
by sinuhe69 3y ago
Albeit a bit condescending, you answer didn’t actually answer the OP’s question. So how could GPT-4 encode and decode base64? Was it explicitly trained on this task or there are enough base64 text in the world so GPT could “automatically” pick it up? Or is it even an emergent property?
- famouswaffles 3y agoIf there was enough Base64 in the training set then it would pick it up. It's just more text to predict. The best prediction is the correct prediction. Maybe this answer seems a bit lacking but no one, not even the people training these models know exactly what it is these artificial neurons are learning or doing during inference so anything more is more speculation than not.
- beebmam 3y ago"Base64 conversions are simple arithmetic", I think is the missing piece to connecting their argument to the OP's question
- famouswaffles 3y agoThe problem with that is that there's no guarantee it Zero shots base64 enc/dec the same way people would do it. and there's no way to tell. for all we know, it treats it as just another "translate x to y" task.