4 ms·

I am using a hacky one I wrote myself. I looked at llm but it doesn't appear to have a mechanism for multi-shot prompting, where you provide both prompts and r

by evmar 2y ago

I am using a hacky one I wrote myself. I looked at llm but it doesn't appear to have a mechanism for multi-shot prompting, where you provide both prompts and responses within your query. (Ref https://platform.openai.com/docs/guides/prompt-engineering/tactic-provide-examples https://platform.openai.com/docs/guides/prompt-engineering/t... .) Maybe take this as a feature request? It feels like the 'template' system in llm might be able to encompass this but the docs don't appear to provide a reference for the yaml format, only examples. I guess that is another feature request, sorry! (BTW if you haven't seen https://docs.divio.com/documentation-system/ https://docs.divio.com/documentation-system/ it really changed how I think about documentation)

simonw 2y ago
Yeah, I've been thinking a bit about the multi shot thing. I've had great results from Claude by "faking" the previous conversation to include example question/answer pairs. With LLM you can do that using the undocumented Python Conversation API, but it's undocumented for a reason (I don't think it's good enough yet). You could also fake a previous conversation through the CLI tool but that is VERY undocumented - you would have to write fake rows into the SQLite database! I also want to support the Claude thing where you can prefill the start of the response - amazing for things like forcing HTML by refilling an HTML doctype.
- evmar 2y ago
  In case it helps any, here are some more details about what I used it for. (Summary: providing examples of a specific translation task, to make it use a particular API in the output.) https://inuh.net/@evmar/112001414385042731 https://inuh.net/@evmar/112001414385042731 And what I expected of llm is for the template file to (optionally) contain an array of prompt/response pairs. You could even imagine the --save flag constructing one from an ongoing conversation of llm -c maybe.
  - simonw 2y ago
    Yeah, doing this with templates is a great idea.

by evmar 2y ago

simonw 2y ago

Yeah, I've been thinking a bit about the multi shot thing. I've had great results from Claude by "faking" the previous conversation to include example question/answer pairs. With LLM you can do that using the undocumented Python Conversation API, but it's undocumented for a reason (I don't think it's good enough yet). You could also fake a previous conversation through the CLI tool but that is VERY undocumented - you would have to write fake rows into the SQLite database! I also want to support the Claude thing where you can prefill the start of the response - amazing for things like forcing HTML by refilling an HTML doctype.

evmar 2y ago
In case it helps any, here are some more details about what I used it for. (Summary: providing examples of a specific translation task, to make it use a particular API in the output.) https://inuh.net/@evmar/112001414385042731 https://inuh.net/@evmar/112001414385042731 And what I expected of llm is for the template file to (optionally) contain an array of prompt/response pairs. You could even imagine the --save flag constructing one from an ongoing conversation of llm -c maybe.
- simonw 2y ago
  Yeah, doing this with templates is a great idea.