Not everything should persist, such as in a classic web architecture, analogous to redis vs postgres Ex: For some of our use cases for more-text data sources a

Not everything should persist, such as in a classic web architecture, analogous to redis vs postgres Ex: For some of our use cases for more-text data sources and APIs we are helping folks index for Louie.ai, we do ingest-time embedding into persistent vector DBs for the raw, and are looking at llmindex and others for more ephemeral run-time caching of post-proccessed results. Ex: Pre-embed 1GB of text for fast search, but then only cache ad-hoc genAI task results on those, like classifications. We may still want to persist the embedded task results for other reasons, so not obvious. If figuring out that kind of thing for use cases like analysts talking to DBs for cyber investigations, supply chain, emergencies, etc, we are actively hiring in backend engineering (remote): https://www.graphistry.com/careers https://www.graphistry.com/careers, who build louie.ai