4 ms·

Not to dismiss the effort here, but wondering if this should go even further than bridging natural language to SQL. With recent work on LLMs showing promising c

Not to dismiss the effort here, but wondering if this should go even further than bridging natural language to SQL. With recent work on LLMs showing promising capabilities in handling complex data structures, maybe the real play is to cut out the SQL altogether. Direct interaction with LLM-optimized data stores could yield more efficient, flexible, and fine-tuned results, eliminating the translational losses that might occur in the NL-to-SQL paradigm. Like instead of shoehorning LLMs into old paradigms, why not conceive data storage systems inherently designed for LLM interfacing? There's a latent inefficiency in using SQL as a bridge. It feels like trying to fit a new narrative into an old framework And yes, rethinking database architectures from the ground up is daunting and demands more background in storage systems but given where machine learning and data interaction are heading, maybe we should consider it.

datahead 3y ago
Curious what recent work you are referencing? Lots of new so I don’t want to miss things. In my experimentation I basically wrote off LLMs from doing any math directly. That includes aggregations or precise calculations. I’ve been playing with sql generation and allowing the db to do its thing.
aazo11 3y ago
SQL is definitely an old technology. However it is still the main language for interfacing with structured data, even for newer tools like Clickhouse. Down the road it is conceivable that it will be replaced with something else, but the cost of migrating existing data to those will be so high I imagine no-one will take the step to proactively move.
- canvascritic 3y ago
  SQL makes the most sense if humans are going to be programming deterministically, but if we're talking about natural language input I can't help but think that compiling natural language to SQL feels a little shortsighted as a long term strategy. Surely it makes sense for a MVP, but I'm curious about how much we're potentially forfeiting in terms of innovation by anchoring so deeply to it with all the possibilities afforded by generative AI. In holding onto SQL, might we be forestalling the development or adoption of a more evolved language? With Dataherald's proposition, how do you envision maintaining that cutting edge, ensuring you're not just prolonging the lifespan of a fading tech, but genuinely pushing the envelope forward? I suppose one intriguing possibility in my mind is that Dataherald could serve as a bridge or runtime. As you continue to develop and refine the tool, do you see a future where it might support more abstract querying languages or even emerging data interaction paradigms? This could potentially mitigate some transition pains when the tech ecosystem eventually shifts.
  - saigal 3y ago
    Do you mean that SQL will significantly fade away?
i_have_an_idea 3y ago
This is a rather superficial take that seems to only consider the aspect of querying a database, but, nevertheless, I'll jot down a few quick counterpoints: 1) A middle layer, such as SQL, offers a vital introspection point to understand and fix LLM mistakes. Without it, we will end up with a "black box" that could lead to errors in data queries and manipulations, making debugging and correction much more challenging. 2) Mathematical properties and data integrity: Databases are built around principles like 3NF and BCNF that are pivotal for efficiency and data integrity. It is highly doubtful that these essential principles could be as effectively replicated in a pure LLM-based system without losing critical aspects of data management and efficient query planning & execution. 3) You mentioned "translational losses" without clarity on what these might be. SQL is a specialized declarative language with clear structure and syntax, while human language often leaves room for ambiguity. Relying solely on an LLM could lead to interpretations that are likely but not necessarily correct. This ambiguity might actually increase "translational losses" rather than reduce them. I understand there's a lot of excitement around generative AI at the moment, but DBMS are an extremely complex topic and this feels like another case of "when you have a hammer everything looks like a nail".
- canvascritic 3y ago
  Appreciate your insights. A few comments/responses 1. Agree on the introspection point, but it's worth noting that the future of LLMs might involve self-awareness capabilities, which could provide an introspective mechanism similar to SQL's transparency. the aim would be to build upon this middle layer, not blindly supplant it. 2. While 3NF, BCNF, and other normalization forms have served us well, they are essentially tools to manage imperfections in our storage and retrieval systems. LLMs can be trained to understand these nuances intrinsically. additionally, database theories have evolved, with advancements like distributed databases, graph-based models, and nosql. so, it's not entirely outside the realm of possibility that we can pivot and adapt to new paradigms. 3. The "translational losses" referred to the semantic disconnect between natural language queries and their SQL representations. while SQL is unambiguous, the leap from a user's intent to the SQL formulation often introduces errors or inefficiencies. LLMs can be trained to improve over time, constantly refining based on feedback loops. not arguing that SQL or databases as we know them are obsolete today, just advocating for a more imaginative exploration about where the tech is headed.
mr_toad 3y ago
I doubt any current LLM is up to the job of bypassing SQL, even for the simplest of queries. The thing is, an SQL database is already a sort of AI - an old school algorithmic AI: “The code generator, and especially the logic in where*.c and in select.c, is sometimes called the query planner. For any particular SQL statement, there might be hundreds, thousands, or millions of different algorithms to compute the answer. The query planner is an AI that strives to select the best algorithm from these millions of choices.” https://www.sqlite.org/arch.html https://www.sqlite.org/arch.html Translating natural language into SQL is ironic, because SQL was intended to be a natural language that would be interpreted by an AI. In the future maybe an AI could be trained to bypass high-level languages. Perhaps it could generate intermediate representation (IR) code - skipping the tokenising and parsing. Perhaps it could generate byte code directly. Or it could even skip the VM and generate machine code to work with the back-end API.

datahead 3y ago

Curious what recent work you are referencing? Lots of new so I don’t want to miss things. In my experimentation I basically wrote off LLMs from doing any math directly. That includes aggregations or precise calculations. I’ve been playing with sql generation and allowing the db to do its thing.

aazo11 3y ago

SQL is definitely an old technology. However it is still the main language for interfacing with structured data, even for newer tools like Clickhouse. Down the road it is conceivable that it will be replaced with something else, but the cost of migrating existing data to those will be so high I imagine no-one will take the step to proactively move.

canvascritic 3y ago
SQL makes the most sense if humans are going to be programming deterministically, but if we're talking about natural language input I can't help but think that compiling natural language to SQL feels a little shortsighted as a long term strategy. Surely it makes sense for a MVP, but I'm curious about how much we're potentially forfeiting in terms of innovation by anchoring so deeply to it with all the possibilities afforded by generative AI. In holding onto SQL, might we be forestalling the development or adoption of a more evolved language? With Dataherald's proposition, how do you envision maintaining that cutting edge, ensuring you're not just prolonging the lifespan of a fading tech, but genuinely pushing the envelope forward? I suppose one intriguing possibility in my mind is that Dataherald could serve as a bridge or runtime. As you continue to develop and refine the tool, do you see a future where it might support more abstract querying languages or even emerging data interaction paradigms? This could potentially mitigate some transition pains when the tech ecosystem eventually shifts.
- saigal 3y ago
  Do you mean that SQL will significantly fade away?

i_have_an_idea 3y ago

This is a rather superficial take that seems to only consider the aspect of querying a database, but, nevertheless, I'll jot down a few quick counterpoints: 1) A middle layer, such as SQL, offers a vital introspection point to understand and fix LLM mistakes. Without it, we will end up with a "black box" that could lead to errors in data queries and manipulations, making debugging and correction much more challenging. 2) Mathematical properties and data integrity: Databases are built around principles like 3NF and BCNF that are pivotal for efficiency and data integrity. It is highly doubtful that these essential principles could be as effectively replicated in a pure LLM-based system without losing critical aspects of data management and efficient query planning & execution. 3) You mentioned "translational losses" without clarity on what these might be. SQL is a specialized declarative language with clear structure and syntax, while human language often leaves room for ambiguity. Relying solely on an LLM could lead to interpretations that are likely but not necessarily correct. This ambiguity might actually increase "translational losses" rather than reduce them. I understand there's a lot of excitement around generative AI at the moment, but DBMS are an extremely complex topic and this feels like another case of "when you have a hammer everything looks like a nail".

canvascritic 3y ago
Appreciate your insights. A few comments/responses 1. Agree on the introspection point, but it's worth noting that the future of LLMs might involve self-awareness capabilities, which could provide an introspective mechanism similar to SQL's transparency. the aim would be to build upon this middle layer, not blindly supplant it. 2. While 3NF, BCNF, and other normalization forms have served us well, they are essentially tools to manage imperfections in our storage and retrieval systems. LLMs can be trained to understand these nuances intrinsically. additionally, database theories have evolved, with advancements like distributed databases, graph-based models, and nosql. so, it's not entirely outside the realm of possibility that we can pivot and adapt to new paradigms. 3. The "translational losses" referred to the semantic disconnect between natural language queries and their SQL representations. while SQL is unambiguous, the leap from a user's intent to the SQL formulation often introduces errors or inefficiencies. LLMs can be trained to improve over time, constantly refining based on feedback loops. not arguing that SQL or databases as we know them are obsolete today, just advocating for a more imaginative exploration about where the tech is headed.

mr_toad 3y ago

I doubt any current LLM is up to the job of bypassing SQL, even for the simplest of queries. The thing is, an SQL database is already a sort of AI - an old school algorithmic AI: “The code generator, and especially the logic in where*.c and in select.c, is sometimes called the query planner. For any particular SQL statement, there might be hundreds, thousands, or millions of different algorithms to compute the answer. The query planner is an AI that strives to select the best algorithm from these millions of choices.” https://www.sqlite.org/arch.html https://www.sqlite.org/arch.html Translating natural language into SQL is ironic, because SQL was intended to be a natural language that would be interpreted by an AI. In the future maybe an AI could be trained to bypass high-level languages. Perhaps it could generate intermediate representation (IR) code - skipping the tokenising and parsing. Perhaps it could generate byte code directly. Or it could even skip the VM and generate machine code to work with the back-end API.