3 ms·

The thing is that this test data generation does not work if you don't account for the schema. Author did so, well done. Been following the same algo for an yea

by larodi 2y ago

The thing is that this test data generation does not work if you don't account for the schema. Author did so, well done. Been following the same algo for an year, and it works as long, as context big enough to keep ids generated. or otherwise you feed ids for the FKs missing. But this is really not a breakthrough, anyone with fair knowledge of LLMs and E/R should be able to devise it. the fact not many people have interdisciplinary knowledge is very much evident from all text2sql papers for example which is a similar domain.

Version467 2y ago
> anyone with fair knowledge of LLMs and E/R should be able to devise it. While this may be true, I think it overlooks a really important aspect. Current LLMs could be very useful in many workflows if someone does the grunt work of properly integrating it. That’s not necessarily complicated, but it is quite a bit of work. I don’t think we’ll hit a capabilities wall anytime soon, but if we do, we’ll still have years of work to do, to properly make use of everything llms have to offer today.
- larodi 2y ago
  This grunt wall is these 10-20% that the model gets wrong, or just misbehaves. And it can be lotta struggle. I'm not talking about easy stuff like writting a letter to customer, but classification and text2code, which is hard Stating this after just finished my long overdue masters and the topic was text2sql with a pinch of my own thing. Hundreds of papers are written on this topic, and only when complex agents, multi-prompting + actual discreete systems play together things start ot work. So just tossing all in the context is not a solution. In practice, I agree, the llms have their role in software, as classifier, graphics segmentation, code assist, etc. But it is very wrong to put all eggs in the same basket, and this basket is very very very shade one.

by larodi 2y ago

Version467 2y ago

> anyone with fair knowledge of LLMs and E/R should be able to devise it. While this may be true, I think it overlooks a really important aspect. Current LLMs could be very useful in many workflows if someone does the grunt work of properly integrating it. That’s not necessarily complicated, but it is quite a bit of work. I don’t think we’ll hit a capabilities wall anytime soon, but if we do, we’ll still have years of work to do, to properly make use of everything llms have to offer today.

larodi 2y ago
This grunt wall is these 10-20% that the model gets wrong, or just misbehaves. And it can be lotta struggle. I'm not talking about easy stuff like writting a letter to customer, but classification and text2code, which is hard Stating this after just finished my long overdue masters and the topic was text2sql with a pinch of my own thing. Hundreds of papers are written on this topic, and only when complex agents, multi-prompting + actual discreete systems play together things start ot work. So just tossing all in the context is not a solution. In practice, I agree, the llms have their role in software, as classifier, graphics segmentation, code assist, etc. But it is very wrong to put all eggs in the same basket, and this basket is very very very shade one.