3 ms·

It's not clear to me that only a vector DB should be used for RAG. Vector DBs give you stochastic responses. For customer chatbots, it seems that structured da

It's not clear to me that only a vector DB should be used for RAG. Vector DBs give you stochastic responses. For customer chatbots, it seems that structured data - from an operational database or a feature store adds more value. If the user asks about an order they made or a product they have a question about, you use the user-id (when logged in) to retrieve all info about what the user bought recently - the LLM will figure out what the prompt is referring to. Reference: https://www.hopsworks.ai/dictionary/retrieval-augmented-llm https://www.hopsworks.ai/dictionary/retrieval-augmented-llm

J_Shelby_J 3y ago
And for technical documentation or code I'm unclear how well semantic search works for CEQ. I would assume the embedding model isn't trained on code and specific words that are industry/company specific.
jarulraj 3y ago
Thanks for sharing that observation on customer chatbots. 1. Will that query look like this: SELECT LLM("{user_question}", order_info) FROM postgres_data.order_table WHERE user_id = “101”; 2. How will a feature store, like Hopsworks, help in this app? Shameless self-plug: We are building EvaDB [1], a query engine for shipping fast AI-powered apps with SQL. Would love to exchange notes on such apps if you're up for it! [1] https://github.com/georgia-tech-db/evadb https://github.com/georgia-tech-db/evadb
- jamesblonde 3y ago
  Why would your projection be this - SELECT LLM("{user_question}", ? You can train a small llm on your private data to map the user question to tables in your db. Then Just select with a limit ( or time bounded). The feature store is just another operational store that could have relevant data for the query.
  - fkyoureadthedoc 3y ago
    > You can train a small llm on your private data to map the user question to tables in your db. Can you? You've personally done this? Deployed it to production at some kind of non trivial scale and it's working well? I'm not aware of any "small llm" that approaches the quality of gpt-3.5.
  - philipodonnell 3y ago
    This is called Text2SQL or NL2SQL, it’s a surprisingly difficult problem even with RAG and GPT4 as soon as the query is non trivial, especially if there are semantic differences between the question and the db schema.

J_Shelby_J 3y ago

And for technical documentation or code I'm unclear how well semantic search works for CEQ. I would assume the embedding model isn't trained on code and specific words that are industry/company specific.

jarulraj 3y ago

Thanks for sharing that observation on customer chatbots. 1. Will that query look like this: SELECT LLM("{user_question}", order_info) FROM postgres_data.order_table WHERE user_id = “101”; 2. How will a feature store, like Hopsworks, help in this app? Shameless self-plug: We are building EvaDB [1], a query engine for shipping fast AI-powered apps with SQL. Would love to exchange notes on such apps if you're up for it! [1] https://github.com/georgia-tech-db/evadb https://github.com/georgia-tech-db/evadb

jamesblonde 3y ago
Why would your projection be this - SELECT LLM("{user_question}", ? You can train a small llm on your private data to map the user question to tables in your db. Then Just select with a limit ( or time bounded). The feature store is just another operational store that could have relevant data for the query.
- fkyoureadthedoc 3y ago
  > You can train a small llm on your private data to map the user question to tables in your db. Can you? You've personally done this? Deployed it to production at some kind of non trivial scale and it's working well? I'm not aware of any "small llm" that approaches the quality of gpt-3.5.
- philipodonnell 3y ago
  This is called Text2SQL or NL2SQL, it’s a surprisingly difficult problem even with RAG and GPT4 as soon as the query is non trivial, especially if there are semantic differences between the question and the db schema.