3 ms·
This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com
by simonw 4y ago
This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here.
For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/chat.ipynb https://github.com/microsoft/guidance/blob/main/notebooks/ch...
create_plan = guidance('''{{#system~}}
You are a helpful assistant.
{{~/system}}
{{#block hidden=True}}
{{#user~}}
I want to {{goal}}.
{{~! generate potential options ~}}
Can you please generate one option for how to accomplish this?
Please make the option very short, at most one line.
{{~/user}}
{{#assistant~}}
{{gen 'options' n=5 temperature=1.0 max_tokens=500}}
{{~/assistant}}
{{/block}}
{{~! generate pros and cons and select the best option ~}}
{{#block hidden=True}}
{{#user~}}
I want to {{goal}}.
''')
How about something like this instead?
create_plan = guidance([
system("You are a helpful assistant."),
hidden([
user("I want to {{goal}}."),
comment("generate potential options"),
user([
"Can you please generate one option for how to accomplish this?",
"Please make the option very short, at most one line."
]),
assistant(gen('options', n=5, temperature=1.0, max_tokens=500)),
]),
comment("generate pros and cons and select the best option"),
hidden(
user("I want to {{goal}}"),
)
])
- itake 4y agoMy guess is you can store the DLS as a file (or in a db). With your example, you have to execute the code stored in your db.
- slundberg 4y agoYou can serialize and ship the DSL to a remote server for high speed execution. (without trusting raw Python code)
- marcotcr 4y agoI think the DSL is nice when you want to take part of the generation and use it later in the prompt, e.g. this (in the same notebook). --- prompt = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#user~}} From now on, whenever your response depends on any factual information, please search the web by using the function <search>query</search> before responding. I will then paste web results in, and you can respond. {{~/user}} {{#assistant~}} Ok, I will do that. Let's do a practice round {{~/assistant}} {{>practice_round}} {{#user~}} That was great, now let's do another one. {{~/user}} {{#assistant~}} Ok, I'm ready. {{~/assistant}} {{#user~}} {{user_query}} {{~/user}} {{#assistant~}} {{gen "query" stop="</search>"}}{{#if (is_search query)}}</search>{{/if}} {{~/assistant}} {{#if (is_search query)}} {{#user~}} Search results: {{#each (search query)}} <result> {{this.title}} {{this.snippet}} </result>{{/each}} {{~/user}} {{#assistant~}} {{gen "answer"}} {{~/assistant}} {{/if}}''') --- You could still write it without a DSL, but I think it would be harder to read.
- PeterisP 4y agoYour example assumes a nested, hierarchical structure while the former example is strictly linear. IMHO that's the key difference there, as the former can (and AFAIK is) be directly encoded and passed to the LLM, which inherently receives only a flat list of tokens. Your example might be nicer to edit, but then it would still have to be translated to the actual 'guidance language' which would have to look (and be) flat.
- emehex 4y agoWe could write a python package that could? A codegen tool that generates codegen that will then generate code? <insert xzibit meme here>
- netdur 4y agoI think chatgpt4 can easily write the python code... wait a second!
- quickthrower2 4y agoWould love to hear your opinion on guidance, in the context of prompt injection attacks :-)