4 ms·
Great summary - thanks. And definitely closer to the LLM api surface we need to really start to use these things. There’s definitely a danger with this kind of
by jameshart 3y ago
Great summary - thanks. And definitely closer to the LLM api surface we need to really start to use these things.
There’s definitely a danger with this kind of code that the first thing this is going to generate to complete the “1.” prompt will be something like “ Truck; 2. Sedan; 3. Minivan”, though.
- mmoskal 3y agoYou generally have to tell the LLM what you want it to generate and then enforce it, i.e, the LLM has to be somewhat aligned with the constraints. Otherwise, for example if you ask for JSON it will keep generating (legal) white-space or when you ask for C code it will say: int sureICanHelpYouHereIsAnExampleOfTheCodeYouWereAskingFor; In this case however, you can just do: await aici.gen_text(regex=r"[a-zA-Z\n]+" stop_at = "\n") Also note that there still a lot of work to figure out how it's easiest for the programmer of an LLM-enabled app to express these things - AICI is meant to make it simple to implement different surface syntaxes.
- giovannibonetti 3y agoGood point. I believe it could be solved with backtracking, just like it is done in compilers/lexers.