3 ms·
Ah yes, it's this paper: https://arxiv.org/abs/2103.06332 https://arxiv.org/abs/2103.06332. With recent advances in alignment, this problem has gotten much bet
by rushingcreek 4y ago
Ah yes, it's this paper: https://arxiv.org/abs/2103.06332 https://arxiv.org/abs/2103.06332.
With recent advances in alignment, this problem has gotten much better. Furthermore, the LLM used in that paper is only ~400 million parameters, which is orders of magnitude smaller than models being used for this task today.
Scale + alignment has been the solution so far.
- throwaway1851 4y agoThanks for the reply! It’s interesting (counterintuitive?) that the problem of models drawing too much from their stored knowledge is mitigated by using models with much greater stored knowledge.