2 ms·
All the open an freely available models seem to have a short context window. Is there any one that I can play with, with a large one?
by weinzierl 3y ago
All the open an freely available models seem to have a short context window. Is there any one that I can play with, with a large one?
- btobolaski 3y agompt-7b and the falcon models have longer contexts available. They aren’t trained on longer contexts so, the results might not be very good.
- weinzierl 3y agoIsn't this true for any model. I always thought the context window exists on the training side and on the query side but not in the model itself and with query side bigger than training side not useful.