It is also caching requests on-disk. See for instance https://github.com/ianb/infinite-ai-array/blob/main/iaia/gptclient.py#L76-L87 https://github.com/ianb/infi

It is also caching requests on-disk. See for instance https://github.com/ianb/infinite-ai-array/blob/main/iaia/gptclient.py#L76-L87 https://github.com/ianb/infinite-ai-array/blob/main/iaia/gpt... The problem* is that if a prompt changes slightly then it won't be a cache hit. * okay, one problem.