5 ms·
I expected to read about the methods used by the libraries to get the structured output and not a comparison of the language compatibility for each. Fortunatel
by resiros 2y ago
I expected to read about the methods used by the libraries to get the structured output and not a comparison of the language compatibility for each.
Fortunately the same author have a blog post (https://www.boundaryml.com/blog/type-definition-prompting-baml https://www.boundaryml.com/blog/type-definition-prompting-ba...) explaining how their approach works and how it compares to instructor (https://github.com/jxnl/instructor https://github.com/jxnl/instructor).
Basically these libraries provide two things:
1. A way to prompt the LLM
2. A way to get a valid JSON
For 1. instructor does it through the json schema definition, BAML's innovation is that they use a simplified lossless schema definition that uses less tokens.
For 2. instructor does it through reprompting until they receive a valid JSON. BAML's innovation is a fuzzy parser able to to parse non-perfect JSON.
Personally I think that there is no need to all these abstractions to get structured outputs from LLMs. A simple .to_prompt() function that takes a pydantic and translate it into some prompt block you can add to your prompt and a retry is sufficient to get the same results.
- Jayakumark 2y agoWill you be able to share an example code or gist ?
- vladsanchez 2y agohttps://agenta.ai/ https://agenta.ai/
- resiros 2y agoThanks for sharing the link, but no agenta is not a library that can help with getting structured outputs from LLMs (at least not in the way discussed in the parent comment). It's a prompt management, evaluation and observability platform for LLM apps.
- vladsanchez 2y agoJust stumbled with https://controlflow.ai/ https://controlflow.ai/ today. Perhaps it serves to structure outputs as "agentic" workflows in pursuit of LLM autonomy. Let us know your opinion.
- resiros 2y agoIf you look into the instructor code(https://github.com/jxnl/instructor/blob/06a49e7824729b8df1f7253f7cdf3de8240aa0bf/instructor/process_response.py#L177 https://github.com/jxnl/instructor/blob/06a49e7824729b8df1f7...). Here is the core code snippet they use: message = dedent( f""" As a genius expert, your task is to understand the content and provide the parsed objects in json that match the following json_schema:\n {json.dumps(response_model.model_json_schema(), indent=2)} Make sure to return an instance of the JSON, not the schema itself """ ) Then depending on the mode, either they add another message `Return the correct JSON response within a ```json codeblock. not the JSON_SCHEMA` or they set the response format to json.