3 ms·

Great question! We will be adding a ChatGPT-based user-defined function this week (https://github.com/georgia-tech-db/eva/pull/655/ https://github.com/georgia-t

Great question! We will be adding a ChatGPT-based user-defined function this week (https://github.com/georgia-tech-db/eva/pull/655/ https://github.com/georgia-tech-db/eva/pull/655/). With LLM-based functions, EVA will support more interesting queries like this: SELECT ChatGPT(TextSummarizer(SpeechRecognizer(audio)), "Is this video related to the Russia-Ukraine war?") FROM VIDEO_CLIPS; Here, EVA sends the audio of each video clip to a speech recognition model on Hugging Face. It then sends the recognized text to a text summarizer model. EVA executes both models on local GPUs. Lastly, EVA sends the text summary to ChatGPT as a part of the prompt. The ChatGPT UDF is executed remotely. The critical feature of EVA is that the query optimizer factors the dollar cost of running models for a given AI task (like a question-answering LLM). It picks the appropriate model pipeline with the lowest price that satisfies the user's accuracy requirement.

la64710 3y ago
Great but personally I am interested in locally runnable LLM models instead of sending data to the cloud service like chatGPT.
- jarulraj 3y ago
  Got it! EVA is designed for the local use case. You can define a Python function that wraps around the LLM model and use it anywhere in the query (we refer to such functions as user-defined functions or UDFs). This notebook illustrates a UDF that wraps around a custom PyTorch vision model: https://evadb.readthedocs.io/en/stable/source/tutorials/04-custom-model.html https://evadb.readthedocs.io/en/stable/source/tutorials/04-c... These functions can be written quickly (~50 lines of Python code). Here is the built-in Resnet50 UDF in EVA: https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/feature_extractor.py https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/... This page describes the steps involved in writing a UDF in EVA: https://evadb.readthedocs.io/en/stable/source/reference/udf.html https://evadb.readthedocs.io/en/stable/source/reference/udf.... Please open an issue on the Github repo; we will gladly support your use case :)

la64710 3y ago

Great but personally I am interested in locally runnable LLM models instead of sending data to the cloud service like chatGPT.

jarulraj 3y ago
Got it! EVA is designed for the local use case. You can define a Python function that wraps around the LLM model and use it anywhere in the query (we refer to such functions as user-defined functions or UDFs). This notebook illustrates a UDF that wraps around a custom PyTorch vision model: https://evadb.readthedocs.io/en/stable/source/tutorials/04-custom-model.html https://evadb.readthedocs.io/en/stable/source/tutorials/04-c... These functions can be written quickly (~50 lines of Python code). Here is the built-in Resnet50 UDF in EVA: https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/feature_extractor.py https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/... This page describes the steps involved in writing a UDF in EVA: https://evadb.readthedocs.io/en/stable/source/reference/udf.html https://evadb.readthedocs.io/en/stable/source/reference/udf.... Please open an issue on the Github repo; we will gladly support your use case :)