2 ms·

Our codex on AWS Bedrock read / write cache ratio was less than 5%. Cache writes are very expensive and they were never being used. This results in codex on Bed

Our codex on AWS Bedrock read / write cache ratio was less than 5%. Cache writes are very expensive and they were never being used. This results in codex on Bedrock causing ~10x what it should due to no caching and massive writes. The workaround in issue resolved for me: web_search = "disabled"

otterley 1mo ago
If you’ve got a workaround, I’d suggest updating the issue description to have it up top there so similarly impacted users can spot it quickly and benefit.
- the_duke 1mo ago
  It's already mentioned in the issue...
  - otterley 1mo ago
    It’s in a comment halfway down the page. It’s not in the issue description. If enough comments are added to the discussion, it might end up being collapsed.
yablak 1mo ago
Way to bury the lede..
chrisweekly 1mo ago
"causing" -> "costing", right?
- vorticalbox 1mo ago
  In this case yeah. If it’s not reading the cache then it has to compute all the context window again and not just the newest tokens.