4 ms·

This works well for simple use cases, but quickly breaks down when faced with real-life scenarios. Database schemas might contain hundreds of tables, and they

by dcastm 3y ago

This works well for simple use cases, but quickly breaks down when faced with real-life scenarios. Database schemas might contain hundreds of tables, and they may not always have intuitive names. Also, the relationships between the tables aren't always clear, and there are always company-specific mystery clauses that you might need to apply (such as excluding certain users) when running a query. Anyone building a chat interface for databases has probably realized this by now.

vorticalbox 3y ago
This is where the function feature comes in, you can define functions that need to be called with some data then you control the database searching. For instance you can have a function with "Id" and "type" where type is an enum of ["post", "comment"] which would return the latest row and another that returns all rows. Then asking "for userID 123 find me the latest post" will call the function and use the results to answer the question Though this isn't always perfect as it can decide that it cannt find that information (ignored functions exist) or if it disagrees with the function response will just completely ignore it. It also has issues with "find me the latest post and comment for ID 456" where it will call the function with "post" then state it doesn't have any comments data. It will find it after father prompting
getmeinrn 3y ago
This is solvable by augmenting the database schema with comments. When you integrate your database with an LLM, you'll notice the LLM will produce flawed queries based on wrinkles in your database schema. This is because the LLM relies on conventional understanding of how the schema is probably tied together. When you see the flawed queries, you augment the schema with a comment that explains why the schema has a wrinkle. The LLM takes that into consideration and the resulting queries are improved. A concrete example[1]: I found that when querying the Sakila movie rental database, the generated query would frequently attempt to join the `rental` table to the `film` table through a nonexistent `film_id` column on the rental table. By adding the linked comment, the LLM stopped doing that. 1. https://github.com/amoffat/HeimdaLLM/blob/dev/notebooks/sakila-schema.sql#L130-L131 https://github.com/amoffat/HeimdaLLM/blob/dev/notebooks/saki...

by dcastm 3y ago

vorticalbox 3y ago

This is where the function feature comes in, you can define functions that need to be called with some data then you control the database searching. For instance you can have a function with "Id" and "type" where type is an enum of ["post", "comment"] which would return the latest row and another that returns all rows. Then asking "for userID 123 find me the latest post" will call the function and use the results to answer the question Though this isn't always perfect as it can decide that it cannt find that information (ignored functions exist) or if it disagrees with the function response will just completely ignore it. It also has issues with "find me the latest post and comment for ID 456" where it will call the function with "post" then state it doesn't have any comments data. It will find it after father prompting

getmeinrn 3y ago

This is solvable by augmenting the database schema with comments. When you integrate your database with an LLM, you'll notice the LLM will produce flawed queries based on wrinkles in your database schema. This is because the LLM relies on conventional understanding of how the schema is probably tied together. When you see the flawed queries, you augment the schema with a comment that explains why the schema has a wrinkle. The LLM takes that into consideration and the resulting queries are improved. A concrete example[1]: I found that when querying the Sakila movie rental database, the generated query would frequently attempt to join the `rental` table to the `film` table through a nonexistent `film_id` column on the rental table. By adding the linked comment, the LLM stopped doing that. 1. https://github.com/amoffat/HeimdaLLM/blob/dev/notebooks/sakila-schema.sql#L130-L131 https://github.com/amoffat/HeimdaLLM/blob/dev/notebooks/saki...