3 ms·
I think he's proposing this kind of API: POST /openai/gpt4 { "prompt": "The address of the White House", "sampler_wasm": "base64 encode
by simonw 3y ago
I think he's proposing this kind of API:
POST /openai/gpt4
{
"prompt": "The address of the White House",
"sampler_wasm": "base64 encoded WASM binary blob here"
}
That WASM would be a program that you write yourself that is run as part of the tokenizer - so it could be a grammar but it could be anything else too.
It's WASM which means it can be safely and performantly run in a sandbox by the OpenAI servers as part of their execution of your prompt.
- newhouseb 3y agoI think you mean "run as part of the sampler," the tokenizer (and tokenization) is fixed for a given model. The sampler blob would basically: 1. Modify the output token probabilities to fit any arbitrary use case 2. Perhaps do trigger some sort of backtracking / beam-search (I'm not Grant but we've chatted on twitter and built similar things)
- simonw 3y agoYes, I meant sampler, not tokenizer.