3 ms·
Are there any models aside from OpenAI’s that can handle large prompts with task breakdowns? I haven’t tried the Anthropic stuff, but every flavor of LLama and
by bugglebeetle 4y ago
Are there any models aside from OpenAI’s that can handle large prompts with task breakdowns? I haven’t tried the Anthropic stuff, but every flavor of LLama and other open source models do not seem capable of this.
- htrp 4y agoevery flavor of llama up to 65b?
- bugglebeetle 4y agoThat’s true, I’ve only run up to 30B. My understanding was they’re limited to a context window of 2048 tokens based on their training and stuff like llama.cpp has an even smaller input context. You can quickly run over that if you’re doing things like appending a result set to a complex prompt. But if others have working examples of using LLama models with large prompts, I’d be interested to see them.
- logicchains 4y agoIn llama.cpp you can use a flag on ./main to set a custom context size, that can be up to 2048.
- bugglebeetle 4y agoAh, ok. I’ve been working with the Python bindings most recently and must have missed that.