4 ms·

It is time for Postgres to care about customers

9 points by code_reader 2y ago

ezekg 2y ago
> Nile’s goal is simple - To be the best platform to build and scale multi-tenant AI applications. So Nile is focused solely on the AI niche now? I stopped reading pretty soon after that was mentioned, so perhaps you should retune your messaging here if that's not the case. I was actually looking at Nile awhile back as a contender for when I eventually move away from Heroku Postgres. But now... I'm confused on what Nile is. ...or maybe I was always confused?
- gwen-shapira 2y ago
  We are still solving problems for all multi-tenant applications. But AI companies, that tend to scale faster, also seem to benefit the most. So we wanted to make this clear.
- code_reader 2y ago
  Nile is general purpose but we probably do well at this point for new use cases. All the new use case apps we have spoken to are building AI into it and have needs from the database. It made sense for us to focus on this market first and expand over time. You can see this in the YC directory as well. Almost 200 B2B companies in the latest batch and they are all trying to build it in an AI-native way https://www.ycombinator.com/companies?batch=W24&industry=B2B https://www.ycombinator.com/companies?batch=W24&industry=B2B
  - ezekg 2y ago
    And are you going to pivot back when the AI bubble bursts? We also saw an influx of blockchain and web3 companies a couple years back -- and that bubble burst. All I'm saying is that from the point of somebody who was keeping tabs on and evaluating Nile, I find the new messaging weird and exclusive. It makes me reconsider Nile because my business has nothing to do with AI at this point. Just my $0.02.
    - code_reader 2y ago
      Yeah, fair point. We will have nothing to pivot on. It is more of a focus question for us and being true to our users. Where do we see demand, and what do we prioritize? We could market as a general-purpose DB and then not prioritize needs from non-AI use cases. This would leave users frustrated. Instead, the problems we are solving for AI have a massive overlap. This helps us mature faster and onboard more non-AI use cases over time.
deleted 2y ago
[deleted]
al2o3cr 2y ago
Strange to see an article about Postgres multitenancy that doesn't even mention schemas: https://www.postgresql.org/docs/current/ddl-schemas.html https://www.postgresql.org/docs/current/ddl-schemas.html I assume the article's solution involves some different set of tradeoffs versus native schemas, but that would have been good to include in the writeup...
- code_reader 2y ago
  We mention that there is a spectrum of solutions to this in the blog but plan to go into it in a separate blog. The main thing is that users no longer have to think about any of these options. Nile does it natively by having a page per tenant architecture and can move tenant's data between computes.
- gwen-shapira 2y ago
  (co-founder of Nile here) You are right, it is hard to cover everything in one blog... But the gist is that schemas and all the objects in them have few significant drawbacks: 1. Catalog overhead. Each schema has all the objects. PG Catalog doesn't scale to huge numbers. If you test anything that needs to work with the entire catalog, you'll run into issues in the 4-5 digits range (depends on number of tables, indexes, etc). 2. If you need to write a query that executes on all schema and aggregates results (for a report or something), you need to build your own tools. 3. Schema migrations and DDLs are increasingly hard to coordinate (I gave a whole talk about that here: https://www.youtube.com/watch?v=Nnz4VesXyUA https://www.youtube.com/watch?v=Nnz4VesXyUA We basically didn't just build the distributed query and DDL layer, we also did some smart things in Postgres itself (blog TBD) that allow us to reduce the catalog overhead significantly.

9 points by code_reader 2y ago

> Nile’s goal is simple - To be the best platform to build and scale multi-tenant AI applications. So Nile is focused solely on the AI niche now? I stopped reading pretty soon after that was mentioned, so perhaps you should retune your messaging here if that's not the case. I was actually looking at Nile awhile back as a contender for when I eventually move away from Heroku Postgres. But now... I'm confused on what Nile is. ...or maybe I was always confused?

gwen-shapira 2y ago
We are still solving problems for all multi-tenant applications. But AI companies, that tend to scale faster, also seem to benefit the most. So we wanted to make this clear.
code_reader 2y ago
Nile is general purpose but we probably do well at this point for new use cases. All the new use case apps we have spoken to are building AI into it and have needs from the database. It made sense for us to focus on this market first and expand over time. You can see this in the YC directory as well. Almost 200 B2B companies in the latest batch and they are all trying to build it in an AI-native way https://www.ycombinator.com/companies?batch=W24&industry=B2B https://www.ycombinator.com/companies?batch=W24&industry=B2B
- ezekg 2y ago
  And are you going to pivot back when the AI bubble bursts? We also saw an influx of blockchain and web3 companies a couple years back -- and that bubble burst. All I'm saying is that from the point of somebody who was keeping tabs on and evaluating Nile, I find the new messaging weird and exclusive. It makes me reconsider Nile because my business has nothing to do with AI at this point. Just my $0.02.
  - code_reader 2y ago
    Yeah, fair point. We will have nothing to pivot on. It is more of a focus question for us and being true to our users. Where do we see demand, and what do we prioritize? We could market as a general-purpose DB and then not prioritize needs from non-AI use cases. This would leave users frustrated. Instead, the problems we are solving for AI have a massive overlap. This helps us mature faster and onboard more non-AI use cases over time.

deleted 2y ago

[deleted]

al2o3cr 2y ago

Strange to see an article about Postgres multitenancy that doesn't even mention schemas: https://www.postgresql.org/docs/current/ddl-schemas.html https://www.postgresql.org/docs/current/ddl-schemas.html I assume the article's solution involves some different set of tradeoffs versus native schemas, but that would have been good to include in the writeup...

code_reader 2y ago
We mention that there is a spectrum of solutions to this in the blog but plan to go into it in a separate blog. The main thing is that users no longer have to think about any of these options. Nile does it natively by having a page per tenant architecture and can move tenant's data between computes.
gwen-shapira 2y ago
(co-founder of Nile here) You are right, it is hard to cover everything in one blog... But the gist is that schemas and all the objects in them have few significant drawbacks: 1. Catalog overhead. Each schema has all the objects. PG Catalog doesn't scale to huge numbers. If you test anything that needs to work with the entire catalog, you'll run into issues in the 4-5 digits range (depends on number of tables, indexes, etc). 2. If you need to write a query that executes on all schema and aggregates results (for a report or something), you need to build your own tools. 3. Schema migrations and DDLs are increasingly hard to coordinate (I gave a whole talk about that here: https://www.youtube.com/watch?v=Nnz4VesXyUA https://www.youtube.com/watch?v=Nnz4VesXyUA We basically didn't just build the distributed query and DDL layer, we also did some smart things in Postgres itself (blog TBD) that allow us to reduce the catalog overhead significantly.