3 ms·
think about training.
by danielmarkbruce 2y ago
think about training.
- moffkalast 2y agoI suppose it would act as a concrete separator when instruct tuning, but lots of prompt templates don't use it, especially older ones like Alpaca. Maybe it leads to more overall coherence?
- m00x 2y agoNot instruct tuning, you use it in general training. If you have a bunch of small prompts/answers, you can fit them into bigger batches if you use start/stop tokens.