2 ms·
Makes me curious. What context is sent? Whole story, including later chapters? Just up to that paragraph? Planning and character description documents? Any way
by jtbayly 1mo ago
Makes me curious. What context is sent? Whole story, including later chapters? Just up to that paragraph? Planning and character description documents? Any way to control all of that?
- refsab 1mo agoWhole story up to the context limit. Then it starts dropping stuff. But I didn't want auto dropping or auto summaries like what other tools do. I like to be strategic summarize the correct places when the context gets full. There is a request inspector that shows everything that is sent. I'm experimenting with prompt ordering strategies as well. this is surprisingly complext when you take into account the different model providers. For example local models usually do a different form of prompt caching than for example openai and anthropic