3 ms·
I'd say that querying data from S3 is not ideal when low-latency queries are required. Generally, there could be a few roundtrip requests to fetch metadata (JSO
by exAspArk 2y ago
I'd say that querying data from S3 is not ideal when low-latency queries are required. Generally, there could be a few roundtrip requests to fetch metadata (JSON, Avro) and data (Parquet) files, which may lead to around 1s or so latency. However, we have caching on our roadmap (it could be just a simple TTL for the fetched data or some more sophisticated caching depending on the synced & queried data)
- anentropic 2y agoHow tied is it to S3? Could I easily point it to Iceberg tables on another storage target?
- exAspArk 2y agoYes! BemiDB natively supports two storage layers, a local disk and S3 (we assumed that most people would choose this in production environments to simplify management). When I query Iceberg tables stored on SSD, it works superfast.
- anentropic 2y agoawesome, thanks