29 ms·
V nice. Can you hook in your own LLM (eg Bloom, t5 etc?)
by csfyrakis 3y ago
V nice. Can you hook in your own LLM (eg Bloom, t5 etc?)
- reasonabl_human 3y agoAlso interested. Would be really great to run this against a self hosted LLM agent.
- louiskw 3y agoWe’re experimenting with this, and the answer is yes and no. GPT4 is the only model that can just about run the agent execution, mainly due to context length and quality. We use our own model for the embedding based code retrieval, and will be replacing some of the GPT3.5 calls with fine tuned models over the coming months.
- ranguna 3y agoWould be great if the app allowed connecting with local LLMs like text generation webui. As for quality, it's up to the user to choose their LLM, so I don't see this as relevant.