3 ms·
This record and replay trick is very similar to what I recently used to implement the query DSL for Tanstack DB (https://tanstack.com/db/latest/docs/guides/live
by samwillis 1y ago
This record and replay trick is very similar to what I recently used to implement the query DSL for Tanstack DB (https://tanstack.com/db/latest/docs/guides/live-queries https://tanstack.com/db/latest/docs/guides/live-queries). We pass a RefProxy object into the where/select/join callbacks and use it to trace all the props and expressions that are performed. As others have noted you can't use js operators to perform actions, so we built a set of small functions that we could trace (eq, gt, not etc.). These callbacks are run once to trace the calls and build a IR of the query.
One thing we were surprisingly able to do is trace the js spread operation as that is a rare case of something you can intercept in JS.
Kenton, if you are reading this, could you add a series of fake operators (eq, gt, in etc) to provide the capability to trace and perform them remotely?
- kentonv 1y agoYes, in principle, any sort of remote compute we want to support, we could accomplish by having a custom function you have to call for it. Then the calls can be captured into the record. But also, apps can already do this themselves. Since the record/replay mechanism already intercepts any RPC calls, the server can simply provide a library of operations as part of its RPC API. And now the mapper callback can take advantage of those. I think this is the approach I prefer: leave it up to servers to provide these ops if they want to. Don't extend the protocol with a built-in library of ops.
- samwillis 1y agoAh, yes, obviously. This is all very cool!
- svieira 1y agoJust a side note - reading https://tanstack.com/db/latest/docs/guides/live-queries#reusable-callback-functions https://tanstack.com/db/latest/docs/guides/live-queries#reus... I see: const isHighValueCustomer = (row: { user: User; order: Order }) => row.user.active && row.order.amount > 1000 But if I'm understanding the docs correctly on this point, doesn't this have to be: const isHighValueCustomer = (row: { user: User; order: Order }) => and(row.user.active, gt(row.order.amount, 1000))
- samwillis 1y agoYep, that's an error in the docs..
- IshKebab 1y agoAs I understand it it's basically how Pytorch works. A clever trick but also super confusing because while it seems like normal code, as soon as you try and do something that you could totally do in normal code it doesn't work: let friendsWithPhotos = friendsPromise.map(friend => { return {friend, photo: friend.has_photo ? api.getUserPhoto(friend.id) : default_photo}; } Looks totally reasonable, but it's not going to work properly. You might not even realise until it's deployed.