3 ms·

I didn't know that neon supports using S3 as cold storage. That's seriously amazing. How does that work with indexes and queries? It's there a single index whic

I didn't know that neon supports using S3 as cold storage. That's seriously amazing. How does that work with indexes and queries? It's there a single index which will always grow and stored on disk or is the index split?

nikita 4y ago
The storage works at the page level. Pageservers provide low latency reads and safekeepers low latency writes. Then we move cold data to S3. Think about safekeepers + S3 being the system of record for most recent + historical data. And pageservers is a scratch space for updating data and service reads. Pages are also on compute nodes. The problem that S3 doesn't love small files. So we organize pages in LSM trees on the pageservers and offload layers to S3.
- zinclozenge 4y ago
  Do you guys do compaction on the levels that live in S3?
  - nikita 4y ago
    Compaction is performed only by pageserver and only for L0 layers. But L0 layers can be offloaded to S3 and loaded back. So we actually perform compaction of S3 layers (but we download them to the pageserver).
- jrochkind1 4y ago
  > The problem that S3 doesn't love small files. I hadn't heard it before and am not finding references googling. I'm curious what problems you have with small files on S3; or if someone wants to feel free to point me to a link to discussion of this apparently known fact!
  - nikita 4y ago
    Small as in 8kb page. A sizable Postgres database would have millions of them. Lots of stuff on the web. Quick googling found: https://www.upsolver.com/blog/small-file-problem-s3 https://www.upsolver.com/blog/small-file-problem-s3