4 ms·
Yeah I get that, after all attension is all you need. But unless you want to spend a bunch of money on the 32k context version I don't think there are other opt
by newswasboring 4y ago
Yeah I get that, after all attension is all you need. But unless you want to spend a bunch of money on the 32k context version I don't think there are other options than embeddings and index.
- cced 4y agoWhat is the state of the art for running models locally in terms of context size?
- newswasboring 4y agoI think local models SOTA is llama which has 2048 context[1]. [1] https://github.com/facebookresearch/llama/issues/16 https://github.com/facebookresearch/llama/issues/16