3 ms·
Do -1, 0 and 1 all occur with the same frequency in typical large language models or would there be a benefit in encoding 0 with 0 and -1 and 1 with respectivel
by jo-han 2y ago
Do -1, 0 and 1 all occur with the same frequency in typical large language models or would there be a benefit in encoding 0 with 0 and -1 and 1 with respectively 10 and 11? (or something even more complex)
Edit: probably not when easy retrieval is needed.