3 ms·
Every token is already being generated with all previously generated tokens as inputs. There's nothing about the architecture that makes this hard. It just hasn
by howrar 3y ago
Every token is already being generated with all previously generated tokens as inputs. There's nothing about the architecture that makes this hard. It just hasn't been trained on this kind of task.
- junofan 3y agoReally? I don’t know of a positional encoding scheme that’ll handle this.