3 ms·

Geniusrise – inference APIs for text, vision, audio, multi-modal AI models

3 points by ixaxaar 3y ago

ixaxaar 3y ago
Geniusrise is a modular, loosely-coupled AI-microservices framework. It can be used to perform various tasks, including hosting inference endpoints, performing bulk inference, fine tune etc with open source models or closed source APIs. - The framework provides structure for modules and operationalizes and orchestrates them. - The modular ecosystem provides a layer of abstraction over the myriad of models, libraries, tools, parameters and optimizations underlying the operationalization of modern AI models. 1. Install geniusrise and libs pip install torch pip install geniusrise pip install geniusrise-vision # vision multi-modal models pip install geniusrise-text # text models, LLMs pip install geniusrise-audio # audio models 2. Create YAML file version: '1' bolts: my_multimodal_api: name: VisualQAAPI state: type: none input: type: batch args: input_folder: ./input output: type: batch args: output_folder: ./output method: listen args: model_name: 'llava-hf/bakLlava-v1-hf' model_class: 'LlavaForConditionalGeneration' processor_class: 'AutoProcessor' device_map: 'cuda:0' use_cuda: True precision: 'bfloat16' quantization: 0 max_memory: None torchscript: False compile: False flash_attention: False better_transformers: False endpoint: '*' port: 3000 cors_domain: 'http://localhost:3000' username: 'user' password: 'password' 3. Launch API service genius rise MY_IMAGE=/path/to/test/image (base64 -w 0 $MY_IMAGE | awk '{print "{\"image_base64\": \""$0"\", \"question\": \"<image>\nUSER: Whats the content of the image?\nASSISTANT:\", \"do_sample\": false, \"max_new_tokens\": 128}"}' > /tmp/image_payload.json) curl -X POST http://localhost:3000/api/v1/answer_question \ -H "Content-Type: application/json" \ -u user:password \ -d @/tmp/image_payload.json | jq More: https://docs.geniusrise.ai/guides/usage/ https://docs.geniusrise.ai/guides/usage/ and https://github.com/geniusrise/examples https://github.com/geniusrise/examples

3 points by ixaxaar 3y ago

Geniusrise is a modular, loosely-coupled AI-microservices framework. It can be used to perform various tasks, including hosting inference endpoints, performing bulk inference, fine tune etc with open source models or closed source APIs. - The framework provides structure for modules and operationalizes and orchestrates them. - The modular ecosystem provides a layer of abstraction over the myriad of models, libraries, tools, parameters and optimizations underlying the operationalization of modern AI models. 1. Install geniusrise and libs pip install torch pip install geniusrise pip install geniusrise-vision # vision multi-modal models pip install geniusrise-text # text models, LLMs pip install geniusrise-audio # audio models 2. Create YAML file version: '1' bolts: my_multimodal_api: name: VisualQAAPI state: type: none input: type: batch args: input_folder: ./input output: type: batch args: output_folder: ./output method: listen args: model_name: 'llava-hf/bakLlava-v1-hf' model_class: 'LlavaForConditionalGeneration' processor_class: 'AutoProcessor' device_map: 'cuda:0' use_cuda: True precision: 'bfloat16' quantization: 0 max_memory: None torchscript: False compile: False flash_attention: False better_transformers: False endpoint: '*' port: 3000 cors_domain: 'http://localhost:3000' username: 'user' password: 'password' 3. Launch API service genius rise MY_IMAGE=/path/to/test/image (base64 -w 0 $MY_IMAGE | awk '{print "{\"image_base64\": \""$0"\", \"question\": \"<image>\nUSER: Whats the content of the image?\nASSISTANT:\", \"do_sample\": false, \"max_new_tokens\": 128}"}' > /tmp/image_payload.json) curl -X POST http://localhost:3000/api/v1/answer_question \ -H "Content-Type: application/json" \ -u user:password \ -d @/tmp/image_payload.json | jq More: https://docs.geniusrise.ai/guides/usage/ https://docs.geniusrise.ai/guides/usage/ and https://github.com/geniusrise/examples https://github.com/geniusrise/examples