4 ms·

> One team suggested that instead of text-to-SQL, we should build semantic business logic layers, “show me Q4 revenue” should map to a verified calculation, not

• tirumaraiselvan 1y ago
  A simple way is perhaps implement a text-to-metrics system where metrics could be defined as SQL functions.
• moomoo11 1y ago
  psychedelics
• dchftcs 1y ago
  On one side, you have an agent calculating the revenue. On the other side, you have an SQL that calculates the revenue Compare the two. If the two disagree, get the AI to try again. If the AI is still wrong after 10 tries, just use the SQL output.
  • mnky9800n 1y ago
    so you have an answer and then you throw compute at trying to produce the answer in a different way. What I hear is a billion dollar AI startup in the making!
    • encyclopedism 1y ago
      Well put...lol :-)
• meheleventyone 1y ago
  They're saying that someone should implement the CalculateQuarterRevenue(year, quarter) function somewhere in a manner that has been verified (e.g. run it against previous quarters to make sure it works correctly) then rather than using the LLM to generate SQL you use it to decide what domain function should be called. Which to me seems to mean that someone on the panel was gently taking the piss out of the idea. Since if you've done all the hardwork anyway presenting this in a deterministic way with a nice UX is straightforward bit of front end work.
  • moduspol 1y ago
    It also removes a lot of the value of the LLM. They're perceived as being smart, and the interface (open-ended text) implies they are capable of more than executing pre-defined functions. So if you have a "CalculateQuarterRevenue(year, quarter)" function, you'll soon find your users asking for the data per-month. Or just for the last six weeks. Or just for a specific client. And they'll be confused when it doesn't work.
    • esafak 1y ago
      The conversational user interface is misleading then, isn't it? It can't make you a sandwich either, though it allows you to submit this request.
      • moduspol 1y ago
        Yes. The sandwich example is contrived, but the basis is "discoverability." It's very opaque to the user what actually can be done and how reliable the result is. Compare this to basically any website you've ever been to. It's the "GUIs vs. CLIs" discussion all over again, except even CLIs had man pages for discoverability.
• esafak 1y ago
  In other words, there should be a list of predefined queries, or possibly subqueries, that the user can request. This is basically how products used to work before AI. The difference is now you can request which query you want verbally. edit: I'm serious. I'm just answering the question, not making a value judgement.
  • thr0w 1y ago
    So simple classification problem. Big deal.
  • lesuorac 1y ago
    I assume you're being tongue in check but I've watched a lot of people use software and they really just don't know anything about it. Being able to verbally request something is something they can learn to do while googling how do I normalize the scores in my rubric to add up to 100 is something they couldn't. Verbal queries is the solution for the world we have even if it's not optimal.
    • slfnflctd 1y ago
      Your last sentence sums it up. This is what users want. The main killer app, I think, boils down really expensive speech-to-text (and vice versa) with a reasonable number of seemingly authoritative querying details in fairly plain language. It's a new, 'better' search engine, just with different pitfalls people need to get up to speed on. And that may be enough, because employing humans to fill the same role as effectively is more expensive still.