4 ms·
AI Models as Private Microservice
- barqawiz 3y agoThis project provides high-level functionality on top of the popular LLMs, allowing you to decouple the models and switch between them based on your use case. The available functionality: - Chatbot: define the provider as chatGPT or llama chat. - Evaluation: Run evaluation across multiple models by sending a query and array of target answers. This will call the models in the background, generate the vector, and compare the distances. This supports cohere, openai, replicate, and sage maker models. - Semantic Search: Apply search beyond the keywords using vectors. - Direct models: directly access the model providers like Sage maker llama or hugging face. - Offline model loader: under development. The micro service published to docker hub: ``` docker pull intellinode/intelliserver:latest docker run -p 80:80 -e API_KEY=$API_KEY -e ADMIN_KEY=$ADMIN_KEY intellinode/intelliserver:latest ``` I am working to add more features to the micro-service, Let me know which functions you think going to be useful.