3 ms·
There seems to be an obvious choice to make here, should you give the users to decrypt and use the COT that they did not generate themselves? This is only requ
by myworkaccount2 2mo ago
There seems to be an obvious choice to make here, should you give the users to decrypt and use the COT that they did not generate themselves?
This is only required if you want users to be able to share things with everyone and you are going for the simplest implementation.
If not you could try to keep a record of keys associated with a user, then when a new request comes in look through to see if the user has a valid key to decrypt the COT.
For explicit shares, just add the key used in that one conversation to the users valid keys. For global shares use the global keys. But that's adding more complexity to the system.
- supriyo-biswas 2mo agoIt’s about being able to change models mid-task. For example, I want to be able to plan using Fable but implement the plan using Sonnet, and that won’t work if this is implemented.
- SaltyBackendGuy 2mo agoOr even my fable credits run out mid task and need to switch back to opus >.<
- QuercusMax 2mo agoPrior to LLMs I never considered that I might have to make a resource-usage decision between hiring Star Trek's Data vs. his stupider brother B4... https://memory-alpha.fandom.com/wiki/B-4 https://memory-alpha.fandom.com/wiki/B-4
- TeMPOraL 2mo agoStar Trek is a post-scarcity society, those problems don't exist there unless you're in the middle of a crisis and on emergency power. LLMs briefly seemed like this too, after subscriptions made the SOTA models too cheap to meter, but before they walked back on that and introduced quotas...
- gs17 2mo agoActually, that brings up a good reason they can't fix it. Fable falls back to Opus when the topic is too "unsafe". That behavior requires traces than can move between models!
- crabette 2mo agoFor plan it's relatively easy, just make the plan the artifact. The point is to ingest knowledge with one model and use it in another, and that is not necessarily easily expressible in natural language.
- miki123211 2mo agoI suspect that there are companies with internal proxies that load-balance across keys, and they didn't want to break that when adding encrypted reasoning.