The standard set by openai and shared by everyone else is to use SSE when “stream: true” is set. Personally I like to strip all the excess data as soon as poss

The standard set by openai and shared by everyone else is to use SSE when “stream: true” is set. Personally I like to strip all the excess data as soon as possible and expose the raw response as a plain “transfer-encoding: chunked” stream. That way the client can decide if they want to accept each chunk as they come or simply await the whole response with limited code change. Either way, being able to easily pass the response writer around and set headers, write, close, as needed without being tied to a single function’s control flow is nice. It can be done in the more modern frameworks ofc, but ends up being a bit more machinery required at each endpoint.