5 ms·
Does anyone know how to run it against Ollama?
by piotrrojek 2y ago
Does anyone know how to run it against Ollama?
- throwaway4aday 2y agoI'd just try setting the env vars and see what happens, if it breaks you'll have to read the code though. https://microsoft.github.io/graphrag/posts/config/overview/ https://microsoft.github.io/graphrag/posts/config/overview/
- ftkftk 2y agoYou will have to do a fair amount of refactoring.
- piotrrojek 2y agoReplying to my own comment - it's trivially possible to replace `api_base` in settings.yaml. The only problem is that Ollama doesn't support OpenAI style embedding endpoint yet. But looking through PRs to Ollama, this is very close to being merged. All steps before actual embedding work fine with e.g. Llama3-8B.
- goofystasy 2y agoYou can use litellm with drop_params for embedding endpoint