3 ms·
Great but personally I am interested in locally runnable LLM models instead of sending data to the cloud service like chatGPT.
by la64710 3y ago
Great but personally I am interested in locally runnable LLM models instead of sending data to the cloud service like chatGPT.
- jarulraj 3y agoGot it! EVA is designed for the local use case. You can define a Python function that wraps around the LLM model and use it anywhere in the query (we refer to such functions as user-defined functions or UDFs). This notebook illustrates a UDF that wraps around a custom PyTorch vision model: https://evadb.readthedocs.io/en/stable/source/tutorials/04-custom-model.html https://evadb.readthedocs.io/en/stable/source/tutorials/04-c... These functions can be written quickly (~50 lines of Python code). Here is the built-in Resnet50 UDF in EVA: https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/feature_extractor.py https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/... This page describes the steps involved in writing a UDF in EVA: https://evadb.readthedocs.io/en/stable/source/reference/udf.html https://evadb.readthedocs.io/en/stable/source/reference/udf.... Please open an issue on the Github repo; we will gladly support your use case :)