4 ms·

hi dan! really interesting post. do you think a new data serialization format built around easier generation/parseability and that also happened to be streamab

by tough 1y ago

hi dan! really interesting post. do you think a new data serialization format built around easier generation/parseability and that also happened to be streamable because its line based like jsonld could be useful for some?

danabramov 1y ago
I don’t know! I think it depends on whether you’re running into any of these problems and have levers to fix them. RSC was specifically designed for that so I was trying to explain its design choices. If you’re building a serializer then I think it’s worth thinking about the format’s characteristics.
- dgb23 1y ago
  I've used React in the past to build some applications and components. Not familiar with RSC. What immediately comes to mind is using a uniform recursive tree instead, where each node has the same fields. In a funny way that would mimic the DOM if you squint. Each node would encode it's type, id, name, value, parent_id and order for example. The engine in front can now generically put stuff into the right place. I don't know whether that is feasible here. Just a thought. I've used similar structures in data driven react (and other) applications. It's also efficient to encode in memory, because you can put this into a flat, compact array. And it fits nicely into SQL dbs as well.
- tough 1y ago
  Awesome, thanks! I do keep running on the issues, but the levers as you say make it harder to implement. As of right now, I could only replace the JSON tool calling on LLM's on something I fully control like vLLM, and the big labs probably are happy to over-charge a 20-30% tokens for each tool call, so they wouldn't really be interested on replacing json any time soon) also it feels like battling against a giant which is already an standard, maybe there's a place for it on really specialized workflows where those savings make the difference (not only money, but you also gain a 20-30% extra token window, if you don't waste it on quotes and braces and what not Thanks for replying!

by tough 1y ago

danabramov 1y ago

I don’t know! I think it depends on whether you’re running into any of these problems and have levers to fix them. RSC was specifically designed for that so I was trying to explain its design choices. If you’re building a serializer then I think it’s worth thinking about the format’s characteristics.

dgb23 1y ago
I've used React in the past to build some applications and components. Not familiar with RSC. What immediately comes to mind is using a uniform recursive tree instead, where each node has the same fields. In a funny way that would mimic the DOM if you squint. Each node would encode it's type, id, name, value, parent_id and order for example. The engine in front can now generically put stuff into the right place. I don't know whether that is feasible here. Just a thought. I've used similar structures in data driven react (and other) applications. It's also efficient to encode in memory, because you can put this into a flat, compact array. And it fits nicely into SQL dbs as well.
tough 1y ago
Awesome, thanks! I do keep running on the issues, but the levers as you say make it harder to implement. As of right now, I could only replace the JSON tool calling on LLM's on something I fully control like vLLM, and the big labs probably are happy to over-charge a 20-30% tokens for each tool call, so they wouldn't really be interested on replacing json any time soon) also it feels like battling against a giant which is already an standard, maybe there's a place for it on really specialized workflows where those savings make the difference (not only money, but you also gain a 20-30% extra token window, if you don't waste it on quotes and braces and what not Thanks for replying!