3 ms·

I've been playing with Khoj for the past day - it's really neat, well done! A few observations: 1. Telemetry is enabled by default, and may contain the API an

I've been playing with Khoj for the past day - it's really neat, well done! A few observations: 1. Telemetry is enabled by default, and may contain the API and chat queries. I've logged an issue for this along with some suggestions here: https://github.com/khoj-ai/khoj/issues/389 https://github.com/khoj-ai/khoj/issues/389 2. It would be advantageous to have configuration in the UI rather than baking it's YAML into the container image. (added a note on that in the aforementioned issue on Github). 3. It's not clear if you can bring your own models, e.g. can I configure a model from huggingface/gpt4all? if so, will it be automatically downloaded based on the name or should I put the .bin (and yaml?) in a volume somewhere? 4. AMD GPU/APU acceleration (CLBLAS) would be really nice, I've logged an issue for this feature request as well. https://github.com/khoj-ai/khoj/issues/390 https://github.com/khoj-ai/khoj/issues/390

sabaimran 3y ago
Thanks for the feedback! Much appreciated. I responded in the issue, but I'll paste here as well for those also curious: Khoj does not collect any search or chat queries. As mentioned in the docs, you can see our telemetry server[1]. If you see anything amiss, point it out to me and I'll hotfix it right away. You can see all the telemetry metadata right here[2]. [1]: https://github.com/khoj-ai/khoj/tree/master/src/telemetry https://github.com/khoj-ai/khoj/tree/master/src/telemetry [2]: https://github.com/khoj-ai/khoj/blob/master/src/khoj/routers/helpers.py#L29 https://github.com/khoj-ai/khoj/blob/master/src/khoj/routers... Configuration with the `docker-compose` setup is a little bit particular, see the issue^ for details. Thanks for the reference points for GPU integration! Just to clarify, we do use GPU optimization for indexing, but not for local chat with Llama. We're looking into getting that working.
tmzt 3y ago
Would it be possible to support a custom URL for the local model, such as running ./server in ggml would give you? This may be more difficult if you are pre-tokenizing the search context. Very cool project.

sabaimran 3y ago

Thanks for the feedback! Much appreciated. I responded in the issue, but I'll paste here as well for those also curious: Khoj does not collect any search or chat queries. As mentioned in the docs, you can see our telemetry server[1]. If you see anything amiss, point it out to me and I'll hotfix it right away. You can see all the telemetry metadata right here[2]. [1]: https://github.com/khoj-ai/khoj/tree/master/src/telemetry https://github.com/khoj-ai/khoj/tree/master/src/telemetry [2]: https://github.com/khoj-ai/khoj/blob/master/src/khoj/routers/helpers.py#L29 https://github.com/khoj-ai/khoj/blob/master/src/khoj/routers... Configuration with the `docker-compose` setup is a little bit particular, see the issue^ for details. Thanks for the reference points for GPU integration! Just to clarify, we do use GPU optimization for indexing, but not for local chat with Llama. We're looking into getting that working.

tmzt 3y ago

Would it be possible to support a custom URL for the local model, such as running ./server in ggml would give you? This may be more difficult if you are pre-tokenizing the search context. Very cool project.