3 ms·
Do any providers do this level of granularity? Anthropic require explicit cache markers, for example.
by IanCal 10mo ago
Do any providers do this level of granularity? Anthropic require explicit cache markers, for example.
- jgeralnik 10mo agoAnthropic requires explicit cache markers but will “look backwards” some amount, so you don’t need to fall on the exact split to get cached tokens