4 ms·

I'm on a (currently very experimental) project where we're trying to answer natural-language queries using data from an arbitrary database (given hypotheses on

by qsort 4y ago

I'm on a (currently very experimental) project where we're trying to answer natural-language queries using data from an arbitrary database (given hypotheses on the schema). The natural-language plaintext is fed into a language model that annotates it. That output is converted into an intermediate representation based on relational algebra, which we manipulate and finally compile to SQL. It's probably not scientifically groundbreaking, but there are a lot of interesting technical bits along the way, at least for me.

solardev 4y ago
Kinda like Wolfram Alpha?
- qsort 4y ago
  Sort of a Wolfram Alpha that hooks up to your own SQL DB, yeah.
omeysalvi 4y ago
Are you working on the Yale Semantic Parsing and Text-to-SQL Challenge? Are you already on the leaderboard? The topic has always fascinated me and got me learning natural language processing to understand how it works.
- qsort 4y ago
  We are going for a narrower scope than that. We are working with stricter restrictions on the database schema, where you are only allowed to join on synthetic keys and there is an enforced distinction between measures and dimension attributes. On the other hand we are aiming for reasonable response times on large scale datasets.
feoren 4y ago
So you're making big assumptions that the database's table names map to important concepts in the user's vocabulary, and that a thorough understanding of the data model can be ascertained from the schema alone? To be brutally honest: that's a silly assumption and likely a dead-end. (One thing I want to emphasize here is how abysmally, unbelievably, depressingly god-awful the vast majority of database schemas are.) You wouldn't expect any user-friendly feature to be automatically stamped on top of a database without the involvement of a programmer, and this is no different. You need some sort of descriptive layer between the database and the query engine (or beside it, helping out) that programmers are involved in building. AI is not going to magically solve this problem for you. Figure out what you need described to you and turn that into a meta language that application developers can write to help your engine out. Frankly I'm wary of any direct link between the user and the database. How do you deal with constraints like "users should only be able to see data associated with their organization in tables X, Y, and Z" if the programmer doesn't tell you? How could you possibly prevent exploits and security problems? Another reason to focus on querying an intermediate layer or relying on a programmer-provided description of what's available. (Or am I assuming way too much, and "hypotheses on the schema" is already this meta-language?)
- qsort 4y ago
  I haven't explained this very well. By "hypotheses on the schema" I mean we are assuming the schema is already in a very normalized form. I fully agree most database schemas are awful, this is a deliberate choice to avoid getting us into that kind of trouble. The "intermediate layer" role is played by the IR I was talking about in my previous comment. The user would never be able to see anything not provided by the IR because it's not even representable. Our scope is much narrower, really.

by qsort 4y ago

solardev 4y ago

Kinda like Wolfram Alpha?

qsort 4y ago
Sort of a Wolfram Alpha that hooks up to your own SQL DB, yeah.

omeysalvi 4y ago

Are you working on the Yale Semantic Parsing and Text-to-SQL Challenge? Are you already on the leaderboard? The topic has always fascinated me and got me learning natural language processing to understand how it works.

qsort 4y ago
We are going for a narrower scope than that. We are working with stricter restrictions on the database schema, where you are only allowed to join on synthetic keys and there is an enforced distinction between measures and dimension attributes. On the other hand we are aiming for reasonable response times on large scale datasets.

feoren 4y ago

So you're making big assumptions that the database's table names map to important concepts in the user's vocabulary, and that a thorough understanding of the data model can be ascertained from the schema alone? To be brutally honest: that's a silly assumption and likely a dead-end. (One thing I want to emphasize here is how abysmally, unbelievably, depressingly god-awful the vast majority of database schemas are.) You wouldn't expect any user-friendly feature to be automatically stamped on top of a database without the involvement of a programmer, and this is no different. You need some sort of descriptive layer between the database and the query engine (or beside it, helping out) that programmers are involved in building. AI is not going to magically solve this problem for you. Figure out what you need described to you and turn that into a meta language that application developers can write to help your engine out. Frankly I'm wary of any direct link between the user and the database. How do you deal with constraints like "users should only be able to see data associated with their organization in tables X, Y, and Z" if the programmer doesn't tell you? How could you possibly prevent exploits and security problems? Another reason to focus on querying an intermediate layer or relying on a programmer-provided description of what's available. (Or am I assuming way too much, and "hypotheses on the schema" is already this meta-language?)

qsort 4y ago
I haven't explained this very well. By "hypotheses on the schema" I mean we are assuming the schema is already in a very normalized form. I fully agree most database schemas are awful, this is a deliberate choice to avoid getting us into that kind of trouble. The "intermediate layer" role is played by the IR I was talking about in my previous comment. The user would never be able to see anything not provided by the IR because it's not even representable. Our scope is much narrower, really.