9 ms·
Quicker serverless Postgres connections
- nikita 4y agoI'm CEO of Neon. Happy to answer any questions you might have.
- amerine 4y agoIs your geo-distributed read replicas going to lean on WAL replication?
- nikita 4y agoYes, we are working on this at the moment. Since we control storage we will be implementing this at the storage level. From there we are thinking to expose <project_id>.read.neon.tech and <project_id>.write.neon.tech and for read queries route traffic to the local replica. It's not set in stone yet, but that's the gist of it.
- amerine 4y agoExcellent. That work’s wonderfully for us too ;-)
- evangow 4y agoAny plans to introduce a way to connect to a Neon database without making it publicly accessible via the internet?
- MuffinFlavored 4y agohow would they do this? allow it to be added to a wireguard VPN or something?
- nikita 4y agoYes. What's your use case? Security or egress costs (those are 0 if you are in the same region).
- r3trohack3r 4y agoNeon looks like everything I've ever wanted out of a serverless database. I send a vendor pg compatible queries, vendor takes care of everything, charges me for usage - and connections work with 3rd party serverless runtimes. I have a number of workloads I'd migrate to you over the next few weeks if you're ready for it. What is the state of Neon? Is it ready for small-volume but business critical loads? I'm struggling to find support guarantees and SLAs on your website, do you have them yet? What is your off-site backup story? Can I export my backups to something like R2, S3, or B2? Many of my customers are running in Cloudflare Workers. Their volume is low, but it's business critical (DB downtime means business downtime - business will fail if we lose all the data).
- nikita 4y ago> I have a number of workloads I'd migrate to you over the next few weeks if you're ready for it. What is the state of Neon? Please do we will support it! We have 22K hosted databases on the platform and are quite stable. > Is it ready for small-volume but business critical loads? Depends on business criticality. But generally yes. We are debating internally when we are going to announce it. > I'm struggling to find support guarantees and SLAs on your website, do you have them yet? Good feedback. It's in the works. > What is your off-site backup story? Can I export my backups to something like R2, S3, or B2? Since our storage is integrated with s3 and supports branching we can treat branches as backups. > Many of my customers are running in Cloudflare Workers. Their volume is low, but it's business critical (DB downtime means business downtime - business will fail if we lose all the data). We haven't lost data yet. There are a lot of redundancy in the system. Safekeepers, S3, branches (backups).
- slig 4y agoDue to latency, Neon is only for AWS/GCP/Azure datacenters or can we use it from smaller providers, such as DO/Linode/etc?
- nikita 4y agoYou can use in anywhere. We will keep working to drive GLOBAL latency down. As in connect to Neon from anywhere.
- codegeek 4y agoHow do you compare against planetscale (I understand that they are mysql vs you postgres but I am looking for serverless db right now so business/product/support comparison is important)
- manigandham 4y agoPlanetscale[1] is MySQL that is automatically sharded and proxied with a system called Vitess to scale horizontally. Because it's real MySQL, most functionality still works without the operations overhead. Neon[2] is Postgres that has a surgically enhanced data layer that reads from a distributed set of nodes and object storage, but is still a single Postgres instance that's started and stopped on-demand and not automatically sharded. Because it's real Postgres, all functionality works including extensions without dealing with issues from sharded datasets, but also without the horizontally scalability and instant start (so far). However because the data layer is improved, the actual compute node is much more efficient in both startup and processing so it works well in many scenarios. CockroachDB[3] is a proprietary distributed database built to Postgres wire/data protocol with natively separated storage and compute. Their serverless plan also starts and stops instances as traffic comes in but with fast startup because of their specific architecture. Because it's not real Postgres though, there's a good bit of missing functionality. There's also TiDB[4] from PingCap which is similar to a MySQL version of CockroachDB, and Yugabyte[5] which is somewhat like Neon but with both distributed compute and data layers also using real Postgres components - however neither has a serverless offering. 1. https://planetscale.com/ https://planetscale.com/ 2. https://neon.tech/ https://neon.tech/ 3. https://www.cockroachlabs.com/ https://www.cockroachlabs.com/ 4. https://www.pingcap.com/tidb/ https://www.pingcap.com/tidb/ 5. https://www.yugabyte.com/ https://www.yugabyte.com/
- codegeek 4y agoThank you. This is helpful.
- sebk 4y agoI'm glad to have an answer to the question I asked you 7 months ago: https://news.ycombinator.com/item?id=32472443 https://news.ycombinator.com/item?id=32472443 (What's Neon's point of view about transient state in nodes? Is there a world where serverless client connections are stateless, or is the set up overhead not expected to be worth the cost?). This is a good analysis of network time and optimization. I'd love to see a followup exploring the impact on the server side; Considering how Postgres was historically not very efficient at setting up new connections but has been steadily getting better with each major release. It would also be great to see what portion of the request is taken up by the setup versus the query itself, to have some idea of cost.
- irq-1 4y agoWhy do web services not discuss the metrics that were common for cloud servers? Like average response time and distribution, throughput, error rates, etc... and what can you tell us about neon?
- Normal_gaussian 4y agoI have a non-tech business approaching limited release. I'm very interested in two things: - Data safety and privacy. As I'm UK based - can my data be restricted to the UK only (no magic round the world trips inside your backend)? do you comply with the GDPR? - Can I set up postgres (WAL) replication to a non-neon database?
- sarb 4y agoIs the websocket connection opened directly from the client to the Postgres server? Or is the other end of the websocket tunnel terminated some sort of reverse proxy? In that case, do you manage the possibility of multiple clients opening a websocket connection to the same to “the same” server (which may actually be a different instance of the same proxy)
- boundlessdreamz 4y agoI didn't know that neon supports using S3 as cold storage. That's seriously amazing. How does that work with indexes and queries? It's there a single index which will always grow and stored on disk or is the index split?
- nikita 4y agoThe storage works at the page level. Pageservers provide low latency reads and safekeepers low latency writes. Then we move cold data to S3. Think about safekeepers + S3 being the system of record for most recent + historical data. And pageservers is a scratch space for updating data and service reads. Pages are also on compute nodes. The problem that S3 doesn't love small files. So we organize pages in LSM trees on the pageservers and offload layers to S3.
- zinclozenge 4y agoDo you guys do compaction on the levels that live in S3?
- nikita 4y agoCompaction is performed only by pageserver and only for L0 layers. But L0 layers can be offloaded to S3 and loaded back. So we actually perform compaction of S3 layers (but we download them to the pageserver).
- jrochkind1 4y ago> The problem that S3 doesn't love small files. I hadn't heard it before and am not finding references googling. I'm curious what problems you have with small files on S3; or if someone wants to feel free to point me to a link to discussion of this apparently known fact!
- nikita 4y agoSmall as in 8kb page. A sizable Postgres database would have millions of them. Lots of stuff on the web. Quick googling found: https://www.upsolver.com/blog/small-file-problem-s3 https://www.upsolver.com/blog/small-file-problem-s3
- janpio 4y agoThat is some serious optimization. Nice! The article ends with the statement that you are pretty much done here for now. Would optimizing your TLS termination not maybe offer some more ways to speed this up? Or is that also already fully optimized? I did not realize before that your approach with Websockets actually meant that there was no application/client side pooling of connections. What made you choose this approach over an HTTP API (as for example PlanetScale did) anyway?
- kelvich 4y ago> The article ends with the statement that you are pretty much done here for now. Would optimizing your TLS termination not maybe offer some more ways to speed this up? Or is that also already fully optimized? No, we don't do early termination yet, but it makes sense to try it out too. Here we mostly concentrated on how far we can get in terms of reducing number of round-trips. > I did not realize before that your approach with Websockets actually meant that there was no application/client side pooling of connections. What made you choose this approach over an HTTP API (as for example PlanetScale did) anyway? To keep compatibility with current code using postgres.js.
- carlsverre 4y agoPretty insightful observation on authentication. Using a password method that requires the client to compute a brute force resistant hash that takes ~100ms of cpu time is a non-starter for most serverless platforms. I'm curious if Cloudflare, or other serverless platforms offer the ability to generate short lived JWTs assigned to each worker. Combined with some configuration (pub key exchange, claim setup, etc) a platform like Neon could use these JWTs to establish identity. Sorta SSO for workers without the CPU overhead. Seems like a safer approach than basic auth with fixed pws.
- nikita 4y agoWe are working with the Cloudflare team and see if they can build it. This is a great idea.
- tyingq 4y agoPostgres also supports auth via SSL client certs, so that's perhaps another option. I don't know how well that maps to apis in Cloudflare and the server side of Neon.
- carlsverre 4y agoThis is a good idea... how much compute does SSL client cert auth require on the client side? I would suppose it has to produce a signature of some kind, perhaps signing a challenge from the server?
- deleted 4y ago[deleted]
- e12e 4y agoClient signs previous handshake messages with private key to prove it has possession of private key for client certificate. It should be a lot faster than a purposefully slow kdf AFAIK.
- chatmasta 4y agoThis is basically how we do authentication in the Splitgraph DDN [0], which is kind of like a multi-tenant serverless Postgres. We implement the Postgres frontend with a forked version of PgBouncer, and we changed the authentication method such that when the user authenticates, we issue them a JWT which we store as a session variable. That session variable has the same security properties as a cookie in a web browser (the user can change/manipulate it, but if it's signed by us we can trust its claims). That's the simple explanation that skips over the multi-tenant part. I don't want to derail from the thread - Neon is very cool, and we are actually experimenting with it right now, for storing the Seafowl [1] catalog [2] when deploying to "scale to zero" services like Google Cloud Run or AWS Lambda, which don't have persistent storage. [0] https://www.splitgraph.com/connect/query https://www.splitgraph.com/connect/query [1] https://seafowl.io https://seafowl.io [2] https://seafowl.io/docs/learning/architecture https://seafowl.io/docs/learning/architecture
- throwaway290 4y agoTo spare someone a click, yes there is still a postgres server. Can't stop being mad at the misuse of "serverless". Even worse than using crypto for cryptocurrencies. Words have meaning...
- chungy 4y agoAt least there are cryptographic functions backing cryptocurrency. It's not _as_ inaccurate as "serverless".
- halayli 4y agoAnd what's your definition of serverless?
- throwaway290 4y agoSomething from peer to peer world for example (or postgres embedded in the same process perhaps)
- hot_gril 4y agoThe title made me think there's, uh, no server. So Postgres running locally on a client. I'm not being obtuse or something, it's a reasonable guess.
- throwaway936x 4y agoFrom what I can tell, you still need to think of it in terms of hardware, as the prices are based on instance sizes. A true serverless solution, is for me something that takes care of this automatically by autoscaling / sharding. I should just have to point my database connection to an URL and it would not require any thoughts on my side. If I suddenly get a peek during Black Friday it should just handle it transparently.
- manigandham 4y agoIt is a terrible term but in this case it's referring to the client application, not the database instance.
- kccqzy 4y ago> A key feature of these environments is that state is not generally persisted from one request to the next. That means we can’t use standard client-side database connection pooling. So we introduced so many optimizations just because we can't persist state. I can't help but think this is being penny wise pound foolish; the problems this article is solving wouldn't have been problems in the first place if you choose a boring architecture.
- nikita 4y agoServerless environments are a matter of fact. They exist, they are super convenient, and people want to use Postgres in them. This is a bridge for such people. Yes, it requires some sophisticated engineering, but it also enables to have less boilerplate code for a lot of people.
- Cthulhu_ 4y agoI think if this particular logic is hidden away behind a server, it doesn't really matter - people want convenience, and for someone else to hide the complexity. Serverless postgres-as-a-service isn't a stretch at all. This thread does remind me of a previous thread, I believe it was something like optimizing a basic string operation (concatenation?) by going down to assembly and SIMD. Few people have a need to know the details of that particular implementation, and mere mortals cannot comprehend the wizardry going on there either, but that's just fine because if it gets incorporated into standard libraries or the underlying implementations that said standard libraries use, everybody benefits without ever having to understand the inner workings.
- interstice 4y agoUnless it breaks one day and no one knows why
- hot_gril 4y agoSomeone knows why, but that someone is not me. And if it had to be me, I'd go figure it out given some time. Point is, I don't need SIMD knowledge hot-loaded in my head.
- spullara 4y agoThis isn't just useful for serverless. Any process-per-request based architecture can also benefit (PHP, Ruby, etc).
- spapas82 4y agoA simple optimization in these cases is to keep the connection open (see https://docs.djangoproject.com/en/4.1/ref/databases/#persistent-connections https://docs.djangoproject.com/en/4.1/ref/databases/#persist...)
- chippiewill 4y agoPHP isn't generally process per request these days (for precisely this reason). Each request is "fresh", but extensions can persist state between requests, including connections to databases etc.
- armatav 4y agoSo how do you do development with this? Is the recommendation to have a per-dev branch off the main DB?
- 0xbadcafebee 4y agoWith a persistent OS-level network tunnel (ex. wireguard) that exists when the app starts, you wouldn't need HTTP, WebSockets, TLS, etc. The app could open a single TCP connection to a sql connection pool and send its request, and it would be tunneled securely from point to point, with no additional round trips or protocols. The idealist in me would rather this kind of thing become a standardized extension of address + transport protocols. But every person I talk to would rather stack 12 protocols on top of each other than work on improving the existing status quo. It feels like a truism of human societies; everyone wants improvement, but nobody wants fundamental reform.
- flumpcakes 4y agoThis article is mainly focusing on serverless architectures where there can be no persistent connection for pooling.
- deleted 4y ago[deleted]
- e12e 4y agoThey most definitely run some network hardware/software in front though - you could definitely spin up workers behind a vpn? They are assigned a network interface, after all? Ed: oh, i see - the usecase is going from a serveless app to pg? I suppose any vm-like serveless node could be made to speak witeguard, too...
- chatmasta 4y agoIt's worth noting that Wireguard doesn't use any persistent connection (although you can configure clients to send Keepalive packets). It sends UDP packets only as long as you're opening connections to remote endpoints over its network interface, and lazily sends handshakes only when it determines it hasn't sent one in a while. In the normal setup, that network interface is "persistent," but only in the sense it has an entry in the routing table, etc.; by default there is no daemon sending any packets over it (unlike say, OpenVPN for example), because the nature of UDP means there is no connection to maintain.
- eashish93 4y agoneon.tech compute units looks costly as compared to mongodb atlas and cloud postgres
- moderation 4y agoThe end of the post mentions HTTP/3 and QUIC. Planetscale have blogged about this [0]. 0. https://planetscale.com/blog/faster-mysql-with-http3 https://planetscale.com/blog/faster-mysql-with-http3