2 ms·
I don’t understand the comment about server send events not being async friendly. What is unfriendly about this? import OpenAI from 'openai'; const opena
by ec109685 2y ago
I don’t understand the comment about server send events not being async friendly.
What is unfriendly about this?
import OpenAI from 'openai';
const openai = new OpenAI();
async function main() {
const stream = await openai.chat.completions.create({
model: 'gpt-4',
messages: [{ role: 'user', content: 'Say this is a test' }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
}
main();
It’s easy to collect the streaming output and return it all when the llm’s response is done.
- deleted 2y ago[deleted]
- wokwokwok 2y agoThey’re referring to the: > assistant.on("textDelta”, () => … Callbacks, which are not async and can’t be streamed that way directly without wrapping it in some helper function. (Which does seem obvious; I’m also not sure why they called it out specifically as not being async friendly? I guess most callback style functions these days have async equivalents in popular libraries and these ones don’t)
- arresin 2y ago> const stream = await… Is this right? Aren’t you prematurely unwrapping the promise here?
- ec109685 2y agoI believe that is what gets the call started so awaiting there is okay. There isn’t anything to stream at that point.