I do not understand your first paragraph. It's more expensive than s3 and it's entirely different. It's cheaper than writing on vellum, and faster! > If you us

I do not understand your first paragraph. It's more expensive than s3 and it's entirely different. It's cheaper than writing on vellum, and faster! > If you use the Elasticache pricing, which is $0.125/gb per hour, it's still eight times more expensive. So even if a million tokens is a full gigabyte of data Is it a gigabyte of data and is it fast enough? You've guessed 4mb and 1gb. What's the actual data size here? What speed do you need to get it into the GPU ram? The entire point here is to lower latency and costs so it has to be close and fast. Guessing at sizes isn't helping anything here.