3 ms·

Frankly, this shouldn't be necessary. There are so many easy gains to be had in implementing an LLM-based chat app which do not require any theoretical advances

by prvc 3y ago

Frankly, this shouldn't be necessary. There are so many easy gains to be had in implementing an LLM-based chat app which do not require any theoretical advances compared to what we have now. All that's needed is a bit of elbow grease from the implementers.

spenczar5 3y ago
Could you give an example?
- prvc 3y ago
  Planning can be implemented (at high computational cost) trivially by generating hidden responses. New models can be trained to natively query "authoritative" sources of information, such as databases and computer algebra systems. New models can be used to transform prompts into more effective ones (along the lines of TFA).
ianbicking 3y ago
I don't know... some of these are about being clear about what you want, and would work with people just like they work with the LLM. Or a lot of what happens in a conversational chat interface is what could happen in a one-shot full prompt; and maybe that's fine for a casual user but if you are programming something you should put in the effort to get that initial prompt right so the conversation isn't as necessary. I do agree about planning; one of the disappointments of Custom GPTs (among many!) is that you can't do this planning without letting it all hang out for the end user. That is, it would be great if you could tell the Custom GPT to put its plans inside <plan>...</plan> tags and have those filtered out (or at least hidden by default; they shouldn't be _secret_, but they are distracting). But even so in that case deciding that you need a plan, and what kind of plan, is something that can and probably should go in the prompt. Not all "plans" are the same, just as not all "summaries" are the same – and part of prompt engineering is getting past these rather lazy descriptions and being specific. Most summaries are a kind of extraction, and asking for a "summary" is deferring to the LLM to figure out what information is interesting entirely based on its sort-of-common-sense assessment. You can always do better than that! Plans are similar, it's an opportunity to give the LLM a template for planning, to specify goals, things to watch out for, etc. You can usually do better than "think step by step".

by prvc 3y ago

spenczar5 3y ago

Could you give an example?

prvc 3y ago
Planning can be implemented (at high computational cost) trivially by generating hidden responses. New models can be trained to natively query "authoritative" sources of information, such as databases and computer algebra systems. New models can be used to transform prompts into more effective ones (along the lines of TFA).

ianbicking 3y ago

I don't know... some of these are about being clear about what you want, and would work with people just like they work with the LLM. Or a lot of what happens in a conversational chat interface is what could happen in a one-shot full prompt; and maybe that's fine for a casual user but if you are programming something you should put in the effort to get that initial prompt right so the conversation isn't as necessary. I do agree about planning; one of the disappointments of Custom GPTs (among many!) is that you can't do this planning without letting it all hang out for the end user. That is, it would be great if you could tell the Custom GPT to put its plans inside <plan>...</plan> tags and have those filtered out (or at least hidden by default; they shouldn't be _secret_, but they are distracting). But even so in that case deciding that you need a plan, and what kind of plan, is something that can and probably should go in the prompt. Not all "plans" are the same, just as not all "summaries" are the same – and part of prompt engineering is getting past these rather lazy descriptions and being specific. Most summaries are a kind of extraction, and asking for a "summary" is deferring to the LLM to figure out what information is interesting entirely based on its sort-of-common-sense assessment. You can always do better than that! Plans are similar, it's an opportunity to give the LLM a template for planning, to specify goals, things to watch out for, etc. You can usually do better than "think step by step".