3 ms·
The chat template adding an assistant prompt by default for example is also shown in the technical report - so they did this during training. The issue is infer
by danielhanchen 2y ago
The chat template adding an assistant prompt by default for example is also shown in the technical report - so they did this during training. The issue is inference workloads should not have this, otherwise inference workloads might inadvertently append extra assistant prompts or forget about it - so hence I removed it.
The rest I'm not sure - for eg the EOS token should be im_end and not endoftext - it could be a small mistake
- greensh 2y agoThanks. I guess this means for Benchmarks they didn't use it. I find it fascinating and admire your dedication to fixing and improving those models.