6 ms·

We've been chasing this rabbit since the beginning. It currently seems to be uncatchable for the use cases that would be most valuable to us - writing complex q

We've been chasing this rabbit since the beginning. It currently seems to be uncatchable for the use cases that would be most valuable to us - writing complex queries we've never seen before. Our use case seems to confound any notion of training or fine-tuning, since the cases we need the most help with are also the ones we have the fewest examples of. Instead of going for generative, few-shot models, I am starting to look at the other end of the spectrum: Binary classification into deterministic query building. With ChatGPT, you cannot realistically explain why you got some output in a way that anyone other than an AI/ML expert would find satisfying. With binary classifiers, you can precisely explain how some input resulted in some output in terms that a business person could easily digest - "You mentioned this table so it assumed you wanted to constrain on XYZ. Here's the trace from the classifiers...". I've proposed a few schemes where you define groups of classifiers for each SQL building concern - Which tables are involved, which columns, is a join or aggregate implied, general context of business use, etc. Clearly, there are holes with this scheme, but in our domain we could plausibly fine-tune our humans to be a little bit more verbose in their use of the automagic SQL vending machine. Several hours spent training humans is probably a lot cheaper & easier than getting ChatGPT, et. al. to consistently play by our rules.

sillysaurusx 3y ago
I think you've fallen into the trap of "AIs don't generalize, they memorize." But they do in fact generalize. The reason ChatGPT is so valuable is precisely because it can help out with situations that have never been seen before, not because it merely unlocks old preexisting knowledge. The fella who saved their dog with ChatGPT comes to mind. https://nypost.com/2023/03/27/chatgpt-saved-my-dogs-life-after-vet-couldnt-diagnosis-it/ https://nypost.com/2023/03/27/chatgpt-saved-my-dogs-life-aft...
- bob1029 3y ago
  > I think you've fallen into the trap of "AIs don't generalize, they memorize." Binary classifiers don't generalize? Just because my output is not generative does not mean we are cannot learn / generalize elsewhere. Think of it as a 2-stage process.
- Zircom 3y ago
  Did you read the article you posted? -ChatGPT initially gives the same diagnosis the vets did, Babesiosis -It notes that Babesiosis may have either been a misdiagnosis or there may be a secondary condition/infection causing the remaining symptoms after the Babesiosis treatment didn't resolve all of them -It suggests such a hypothetical secondary condition could be IMHA, which the article notes is an extremely common complication of Babesiosis with this specific dog breed -A quick Google search brings up a fair amount of literature about the association between Babesiosis and IMHA So in fact this is the opposite of a never before seen situation, ChatGPT was just regurgitating common comorbidities of Babesiosis and the vets in question are terrible at their job.
  - sillysaurusx 3y ago
    Are you suggesting that it was commonplace for machine learning models to be able to extrapolate medical case reports into an actual diagnosis? Even being able to read and understand a case report is a minor miracle in extrapolation, and it’s interesting how far the goal posts move.
    - lossolo 3y ago
      I have another example of ChatGPT not generalizing but just being a really good statistical model. I needed a solution to a problem that you can't find on Google, and a variation of a problem that also can't be found on Google. I attempted to obtain around 15 lines of code from ChatGPT that would solve the problem, but it consistently failed to produce the correct solution. I spent a few hours trying different prompts, indicating its errors and receiving apologies, only for it to generate another incorrect solution while acknowledging its mistake. Solving out of distribution problems correctly seems almost impossible for it.
      - sillysaurusx 3y ago
        GPT 3.5 or 4? Surprisingly it makes a huge difference. I think a lot of peoples’ impressions are with 3.5, but many startups couldn’t have been built on it, whereas with 4 they can. If it was 4 I’d be curious about the specific problem if you’d be willing to link to the chat.
- PheonixPharts 3y ago
  I think you have fallen into the trap of mistaking interpolation for generalization. Working with these models every day, it's clear that they can certainly interpolate between points in latent space and generate sensible answers to unseen questions, but it's pretty clear that they don't generalize. I've seen far to many examples of models failing to display any sense of generalization to believe otherwise. That's not to say that interpolation in a rich latent space of text isn't very useful. But it's not the same level of abstraction that comes from true generalization in this space.
  - riwsky 3y ago
    I think we’ve all fallen into the trap of mistaking pithiness for evidence
  - famouswaffles 3y ago
    >it's pretty clear that they don't generalize. I've seen far to many examples of models failing to display any sense of generalization to believe otherwise Failing to generalize on whatever you have in mind is not evidence that the models are incapable of generalization. If you really think so, just be prepared to write off a good chunk of humans as well lol. This is generalization. https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/ Just seems to me like you've taken examples of generalization and invented an alternate meaning just so you can't admit it generalizes. "Oh but you see this example of answering an unseen question is "insert meaningless distinction" so it doesn't really count"
  - johndough 3y ago
    With a sufficiently large dataset, every problem becomes interpolation.
  - nl 3y ago
    Given a multi-dimensional latent space with enough dimensions it's hard to imagine cases of generalization that aren't interpolation between points in latent space (given enough data). The one possible exception is logical inference, and this problem seems tractable with tool use or programming.
- mejutoco 3y ago
  > With ChatGPT, you cannot realistically explain why you got some output in a way that anyone other than an AI/ML expert would find satisfying IMHO OP is talking about "explainability" of the results, which is notoriously bad for current AI. For certain applications (idk if SQL would be one but mortgage application might be one) it is required to be able to explain how the computer got to the decision.
Nick_Molter 3y ago
Curious to learn more about your use case. If fine-tuning is only ineffective for your most complex queries (and presumably those are less frequent as well, since you mentioned you have few examples), then couldn't you use fine-tuning to handle the simpler queries (presumably the lion's share) and thus free up excess man hours to focus on the more complex queries? Is there any benefit to AI being able to answer 90% of queries vs 0%?
- bob1029 3y ago
  These tools are already fantastic at our 80% average case, even without fine tuning. We are seeing some value, but the real pain is in that other 20%.
MuffinFlavored 3y ago
> Binary classification into deterministic query building. I have no clue what this means.
- philipodonnell 3y ago
  I tried something similar to this so I assume it’s to classify a table as being part of the resulting query or not, and using the schemas of the resulting tables in the prompt for the question. I found it just as useful to give it the full schema and ask it for a list of relevant tables, certainly more cost effective.
lmeyerov 3y ago
Yeah louie.ai query generation internals feel more like classic program synthesis, where's it's more about staged compilation, iterative refinement, etc, and we use gpt3/gpt4 for each stage rather than a traditional solver. "Which tables would help this query, which parts should go to text search vs vector search, ...", and we iteratively rewrite and pass along until we reconstitute. At the same time, we're also moving to make a lot more of that process AI controlled, just across LLM calls (e.g., LLM-dictated ones), so it's a funny maturity process.

sillysaurusx 3y ago

I think you've fallen into the trap of "AIs don't generalize, they memorize." But they do in fact generalize. The reason ChatGPT is so valuable is precisely because it can help out with situations that have never been seen before, not because it merely unlocks old preexisting knowledge. The fella who saved their dog with ChatGPT comes to mind. https://nypost.com/2023/03/27/chatgpt-saved-my-dogs-life-after-vet-couldnt-diagnosis-it/ https://nypost.com/2023/03/27/chatgpt-saved-my-dogs-life-aft...

bob1029 3y ago
> I think you've fallen into the trap of "AIs don't generalize, they memorize." Binary classifiers don't generalize? Just because my output is not generative does not mean we are cannot learn / generalize elsewhere. Think of it as a 2-stage process.
Zircom 3y ago
Did you read the article you posted? -ChatGPT initially gives the same diagnosis the vets did, Babesiosis -It notes that Babesiosis may have either been a misdiagnosis or there may be a secondary condition/infection causing the remaining symptoms after the Babesiosis treatment didn't resolve all of them -It suggests such a hypothetical secondary condition could be IMHA, which the article notes is an extremely common complication of Babesiosis with this specific dog breed -A quick Google search brings up a fair amount of literature about the association between Babesiosis and IMHA So in fact this is the opposite of a never before seen situation, ChatGPT was just regurgitating common comorbidities of Babesiosis and the vets in question are terrible at their job.
- sillysaurusx 3y ago
  Are you suggesting that it was commonplace for machine learning models to be able to extrapolate medical case reports into an actual diagnosis? Even being able to read and understand a case report is a minor miracle in extrapolation, and it’s interesting how far the goal posts move.
  - lossolo 3y ago
    I have another example of ChatGPT not generalizing but just being a really good statistical model. I needed a solution to a problem that you can't find on Google, and a variation of a problem that also can't be found on Google. I attempted to obtain around 15 lines of code from ChatGPT that would solve the problem, but it consistently failed to produce the correct solution. I spent a few hours trying different prompts, indicating its errors and receiving apologies, only for it to generate another incorrect solution while acknowledging its mistake. Solving out of distribution problems correctly seems almost impossible for it.
    - sillysaurusx 3y ago
      GPT 3.5 or 4? Surprisingly it makes a huge difference. I think a lot of peoples’ impressions are with 3.5, but many startups couldn’t have been built on it, whereas with 4 they can. If it was 4 I’d be curious about the specific problem if you’d be willing to link to the chat.
PheonixPharts 3y ago
I think you have fallen into the trap of mistaking interpolation for generalization. Working with these models every day, it's clear that they can certainly interpolate between points in latent space and generate sensible answers to unseen questions, but it's pretty clear that they don't generalize. I've seen far to many examples of models failing to display any sense of generalization to believe otherwise. That's not to say that interpolation in a rich latent space of text isn't very useful. But it's not the same level of abstraction that comes from true generalization in this space.
- riwsky 3y ago
  I think we’ve all fallen into the trap of mistaking pithiness for evidence
- famouswaffles 3y ago
  >it's pretty clear that they don't generalize. I've seen far to many examples of models failing to display any sense of generalization to believe otherwise Failing to generalize on whatever you have in mind is not evidence that the models are incapable of generalization. If you really think so, just be prepared to write off a good chunk of humans as well lol. This is generalization. https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/ Just seems to me like you've taken examples of generalization and invented an alternate meaning just so you can't admit it generalizes. "Oh but you see this example of answering an unseen question is "insert meaningless distinction" so it doesn't really count"
- johndough 3y ago
  With a sufficiently large dataset, every problem becomes interpolation.
- nl 3y ago
  Given a multi-dimensional latent space with enough dimensions it's hard to imagine cases of generalization that aren't interpolation between points in latent space (given enough data). The one possible exception is logical inference, and this problem seems tractable with tool use or programming.
mejutoco 3y ago
> With ChatGPT, you cannot realistically explain why you got some output in a way that anyone other than an AI/ML expert would find satisfying IMHO OP is talking about "explainability" of the results, which is notoriously bad for current AI. For certain applications (idk if SQL would be one but mortgage application might be one) it is required to be able to explain how the computer got to the decision.

Nick_Molter 3y ago

Curious to learn more about your use case. If fine-tuning is only ineffective for your most complex queries (and presumably those are less frequent as well, since you mentioned you have few examples), then couldn't you use fine-tuning to handle the simpler queries (presumably the lion's share) and thus free up excess man hours to focus on the more complex queries? Is there any benefit to AI being able to answer 90% of queries vs 0%?

bob1029 3y ago
These tools are already fantastic at our 80% average case, even without fine tuning. We are seeing some value, but the real pain is in that other 20%.

MuffinFlavored 3y ago

> Binary classification into deterministic query building. I have no clue what this means.

philipodonnell 3y ago
I tried something similar to this so I assume it’s to classify a table as being part of the resulting query or not, and using the schemas of the resulting tables in the prompt for the question. I found it just as useful to give it the full schema and ask it for a list of relevant tables, certainly more cost effective.

lmeyerov 3y ago

Yeah louie.ai query generation internals feel more like classic program synthesis, where's it's more about staged compilation, iterative refinement, etc, and we use gpt3/gpt4 for each stage rather than a traditional solver. "Which tables would help this query, which parts should go to text search vs vector search, ...", and we iteratively rewrite and pass along until we reconstitute. At the same time, we're also moving to make a lot more of that process AI controlled, just across LLM calls (e.g., LLM-dictated ones), so it's a funny maturity process.