4 ms·
With regard to GPT3, it doesn't know what bits are. I'm guessing doesn't know what bytes are either, and that it operates on the level of some subset of unicode
by doubleunplussed 4y ago
With regard to GPT3, it doesn't know what bits are. I'm guessing doesn't know what bytes are either, and that it operates on the level of some subset of unicode codepoints, with one input-layer and output-layer neuron for each. But I could be wrong, a quick google does not immediately confirm or refute.
- convexfunction 4y agoBelow is a short conversation you inspired me to have with OpenAI's text-davinci-002, mainly because it sounded like it'd be cute rather than because I think it proves anything (I'm no expert prompt engineer): ==== What is the byte value of the letter "c"? The byte value of the letter "c" is 99. Answer the next question with a sequence of byte values, one per letter, instead of with normal text. What animal says "neigh"? The animal that says "neigh" is a horse. That was incorrect, you answered with normal text. Try again: What animal says "neigh"? The animal that says "neigh" is a horse. The byte value of the letter "n" is 110, the byte value of the letter "e" is 101, the byte value of the letter "i" is 105, the byte value of the letter "g" is 103. That was closer, but still not quite right. Here's an example: What animal says "meow"? 99 97 116 Try again: What animal says "bark"? The animal that says "bark" is a dog. The byte value of the letter "b" is 98, the byte value of the letter "a" is 97, the byte value of the letter "r" is 114, the byte value of the letter "k" is 107.