19 ms·

Natural language to SQL is a classical AI task at this point. The goal is not to make something for developers, who will likely prefer SQL or a DSL regardless,

by qsort 4y ago

Natural language to SQL is a classical AI task at this point. The goal is not to make something for developers, who will likely prefer SQL or a DSL regardless, but to craft tools that can be handed out to business people without developers having to fiddle with code for each request. (NL to SQL is actually one of my tasks at $work). Obviously, when you actually try to make such a system beyond a minimal demo to publish a paper, you start running into all kinds of issues, e.g. permissions for different users, performance, matching language to the concepts in the data. And that's not even taking into account adversarial prompts (injections, ddos, etc.) There's no chance in hell I'm hooking up my database to an undefined behavior generator! It's an interesting area that I follow closely, but the article is being a bit too optimistic.

graypegg 4y ago
That’s cool that this is your field! It could just be the optimism in the article that’s getting to me here. I just feel like natural language as an interface has a bit too much slop in the controls, even if the controls are easier. I’d like to imagine the non technical folks using something like the dashboard example in the article would be ok with some extra work formatting their query to ensure what they think they want, is precisely what they’re asking for. Adding a little layer in there that was mostly trained on things unrelated to your query, feels more like driving-by-wire. Doesn’t mean it doesn’t work but that precision is lost? I’m also: completely out of my field for talking on this. Thanks for the interesting insight into it!
- qsort 4y ago
  > I’d like to imagine the non technical folks using something like the dashboard example in the article would be ok with some extra work formatting their query to ensure what they think they want, is precisely what they’re asking for. The UX in this area is very much work in progress, but from what I have seen I agree that users likely prefer to be sure that the query is correct. What I would try to do -- and let's be clear, this is just a random opinion and my guess is as good as anybody's -- is to answer with two results, "this is my interpretation of what you asked" and "this is the answer according to the interpretation above". It's a bit too technical for the average user, but Wolfram Alpha nails it in my opinion, and I'm convinced that with some refinement it could be usable for business folk. Example of what I mean: https://www.wolframalpha.com/input?i=population+of+india+%2F+population+of+china https://www.wolframalpha.com/input?i=population+of+india+%2F... This is consistent with the state of the art in NL to SQL literature (example: https://arxiv.org/pdf/1905.08205.pdf https://arxiv.org/pdf/1905.08205.pdf), which tends to put an intermediate representation between pure English and pure SQL. It is especially useful for production systems because it gives additional control to the backend in addition to its benefits for the pure NLP task.

by qsort 4y ago

graypegg 4y ago

That’s cool that this is your field! It could just be the optimism in the article that’s getting to me here. I just feel like natural language as an interface has a bit too much slop in the controls, even if the controls are easier. I’d like to imagine the non technical folks using something like the dashboard example in the article would be ok with some extra work formatting their query to ensure what they think they want, is precisely what they’re asking for. Adding a little layer in there that was mostly trained on things unrelated to your query, feels more like driving-by-wire. Doesn’t mean it doesn’t work but that precision is lost? I’m also: completely out of my field for talking on this. Thanks for the interesting insight into it!

qsort 4y ago
> I’d like to imagine the non technical folks using something like the dashboard example in the article would be ok with some extra work formatting their query to ensure what they think they want, is precisely what they’re asking for. The UX in this area is very much work in progress, but from what I have seen I agree that users likely prefer to be sure that the query is correct. What I would try to do -- and let's be clear, this is just a random opinion and my guess is as good as anybody's -- is to answer with two results, "this is my interpretation of what you asked" and "this is the answer according to the interpretation above". It's a bit too technical for the average user, but Wolfram Alpha nails it in my opinion, and I'm convinced that with some refinement it could be usable for business folk. Example of what I mean: https://www.wolframalpha.com/input?i=population+of+india+%2F+population+of+china https://www.wolframalpha.com/input?i=population+of+india+%2F... This is consistent with the state of the art in NL to SQL literature (example: https://arxiv.org/pdf/1905.08205.pdf https://arxiv.org/pdf/1905.08205.pdf), which tends to put an intermediate representation between pure English and pure SQL. It is especially useful for production systems because it gives additional control to the backend in addition to its benefits for the pure NLP task.