3 ms·
> Do you use an agent SDK or is it home grown? It's about 260 lines of home-grown code. Here's a snippet of how it looks: https://gist.github.com/stopachka/a3
by stopachka 1y ago
> Do you use an agent SDK or is it home grown?
It's about 260 lines of home-grown code. Here's a snippet of how it looks:
https://gist.github.com/stopachka/a38f00545d048661cf15a3cf4d7e1dcd https://gist.github.com/stopachka/a38f00545d048661cf15a3cf4d...
Two interesting things there:
1. Since Instant is reactive, we use it to convey the stream to the client. This way all chats are "resumable" by default, even if the user refreshes the page.
2. To make sure Instant writes don't block the stream, we use a "natural" buffer approach. Basically tokens from the LLM buffer up during every save.