3 ms·

> The tokens that are valid at the beginning of the output include things like {, {“, {\n, etc. However, once the model has already sampled {“val, then { is no

> The tokens that are valid at the beginning of the output include things like {, {“, {\n, etc. However, once the model has already sampled {“val, then { is no longer a valid token Oops, this is incorrect. {“val{“:2} is valid json. (modulo iOS quotes lol)

jhgg 2y ago
Valid JSON, sure, but that key does not conform to the schema provided in the example. The LLM must generate valid JSON that also conforms to the provided schema.
jeffhuys 2y ago
FYI you can just hold down the “ button and get a ".