3 ms·
Paper (which is more useful than the linked HF model) is at https://arxiv.org/abs/2401.02415 https://arxiv.org/abs/2401.02415 There is also an -Instruct varian
by dweekly 3y ago
Paper (which is more useful than the linked HF model) is at https://arxiv.org/abs/2401.02415 https://arxiv.org/abs/2401.02415
There is also an -Instruct variant of this model published on HF.
They are claiming the best balance of performance on language and coding tasks in an open source LLM of this size.
- deleted 3y ago[deleted]
- Jackson__ 3y agoInteresting, this sounds very similar to this other recent paper as well: https://arxiv.org/abs/2401.02412 https://arxiv.org/abs/2401.02412 Especially with keeping the original model weights the same.
- miven 3y agoWhat do they consider to be an "LLM of this size"? While this technique of scaling up an existing pre-trained model via fine-tuning is really impressive, it feels a bit unfair to compare what's essentially now an 8.3B model to mostly standard 7B ones, especially considering how important scale is in predicting LLM performance.