6 ms·
HTTP vs. WebSockets: Which one is the fastest for Postgres queries at the edge?
- nikita 3y agoCEO of Neon here. We have done a lot of latency work. Feel free to ask questions.
- bryzaguy 3y agoWhat’s the difference in performance between websockets and http with large datasets. I would imagine http might be faster? Do you encode your data with something like messagepack when using web sockets?
- raoufchebri 3y agoBlog post author here. Let me know if you have any questions!
- Lichtso 3y agoI assume by "http" you mean http1.1 and http2, not http3? Would be nice to differentiate these in the testing as well. WebSockets still seem to be stuck with TCP as rfc9220 [0] is not implemented I think. [0]: https://datatracker.ietf.org/doc/rfc9220/ https://datatracker.ietf.org/doc/rfc9220/
- raoufchebri 3y agoYeah! these are just normal websockets. We're definitely interested in supporting and testing more protocols and ways of accessing postgres and neon as we can. We're also super excited to see the development and adoption of new protocols!
- lfmunoz4 3y ago[dead]
- com2kid 3y agoThe axis on all your graphs are unlabeled and have the same scale in both the X and the Y, accordingly I am not sure exactly what is being shown on the graph.
- gmac 3y agoFair point! We'll fix this. In the meantime, the graphs show latency distributions, so the X axis is time (in milliseconds) and the Y axis is a simple count.
- raoufchebri 3y agoThank you. We fixed it now.
- FlyingSnake 3y agoPardon my ignorance as I'm not aware of these recent developments, but what is a valid use case for this? Why would I put my database into an ephemeral serverless DB[1]? 1: https://news.ycombinator.com/item?id=36684121 https://news.ycombinator.com/item?id=36684121
- gmac 3y agoWell, the data is absolutely not ephemeral: it's safely tucked away in S3. But to crib from https://neon.tech https://neon.tech, you might choose Neon because it's scalable, cost-effective, and easy to deploy and manage. Or you might choose it for its copy-on-write branching and point-in-time recovery, all open-source, and all on top of the solid, performant, feature-rich foundation that is Postgres. :)
- raoufchebri 3y agoNeon separates storage and compute. The compute part is serverless and can scale up and down on demand. The storage has multiple layers: Safekeepers for durability and Pageservers to reconstruct pages from WAL. The data is also offloaded to S3. Here is a diagram that describes the architecture: https://neon.tech/_next/image?url=https%3A%2F%2Fneon-hwp.dreamhosters.com%2Fwp-content%2Fuploads%2F2023%2F07%2Fimage-8.png&w=3840&q=85 https://neon.tech/_next/image?url=https%3A%2F%2Fneon-hwp.dre...
- tristan957 3y agoTo follow up on Raouf's comment, here is a blog post[0] that describes Neon's architecture in more detail. [0]: https://neon.tech/blog/architecture-decisions-in-neon https://neon.tech/blog/architecture-decisions-in-neon
- redm 3y ago@raoufchebri & @nikita, What about comparisons to CloudFlare Workers using Workers Sock API to connect to PG? [1] I know CloudFlare workers support Neon directly through API so how does that compare to the alternative PG implementations? [1] https://developers.cloudflare.com/workers/learning/integrations/databases/ https://developers.cloudflare.com/workers/learning/integrati...
- raoufchebri 3y agoWe have only compared on Vercel Edge and Serverless environments. I think it makes a lot of sense to compare our driver to pg on CF with connect.
- bluefishinit 3y agoIt seems like if speed was a concern you wouldn't make the extra hop from a serverless function to their websocket server, then on to the db. Instead you could serve your api directly off the app server which would then connect to the db without the intermediary step.
- raoufchebri 3y agoWe use a proxy that redirects the request to the compute node. The proxy uses WebSockets in this case. https://neon.tech/blog/serverless-driver-for-postgres https://neon.tech/blog/serverless-driver-for-postgres
- gmac 3y agoEvery connection to a Neon database (traditional TCP, WebSockets, https) now goes via the same custom (open-source) proxy server that's close to the database compute node. So there's technically an additional hop irrespective of the transport type, but not one that should add meaningfully to the latency, or that would make WebSockets slower than the others.
- deleted 3y ago[deleted]
- bob1029 3y agoThe diagram is confusing me too. We are using Az Functions to serve webapps via HTTP triggers and they just take a direct connection to SQL. Our average trip to the database is ~5 milliseconds for the trivial views. No web sockets, middleware, etc. What value is added with this extra middleware/ceremony/etc?
- gmac 3y agoRight. If TCP is available, any common-or-garden Postgres driver will do (for Neon or any other Postgres DB). But not all serverless compute environments support TCP connections.
- mwcampbell 3y ago
- rubenfiszel 3y agoWhat I would absolutely love is to have a proxy server that would act as postgres, and use this new driver to connect to a remote postgres. This would avoid us having to rewrite our remote agents (which require a postgres connection to the main Database) to use an http api. Instead, we could just keep them as it is, using rust's sqlx queries, point them to the local proxy. Essentially pgBouncer but through http. Postgres <> Neon HTTP Proxy <---HTTP---> Local Fake Postgres Server <--unix socket--> Agents We need it for the open-source project windmill.dev and we will build this if no one else does it.
- gmac 3y agoThis sounds cool (but it's not something we've built at Neon). In case it's useful, I can clarify: * Every Neon database is accessible over an ordinary TCP Postgres connection (as well as WebSockets and http). * Our JavaScript serverless driver is based on node-postgres and is a drop-in replacement for it. So for JS, no rewriting is needed. Assuming that doesn't help, do you specifically need http (rather than WebSockets), and a proxy that speaks the pg protocol? If not, the simplest thing might be to (1) use Neon, or run a WebSocket proxy next to your own database (e.g. https://github.com/neondatabase/wsproxy); https://github.com/neondatabase/wsproxy); and (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
- rubenfiszel 3y agoWindmill is written in Rust so cannot use the javascript serverless driver. I love sqlx and sqlx would not be easy to replace so short of making it an extension for sqlx, the only route is to make believe the sqlx connection pool that it's talking to a pg server, when it's actually just talking to the local proxy. If we were using javascript/typescript, then yes we would probably just be able to use that library. > (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents. It's that part that I would love to see. My assumption is that this proxy next to my agents would need to speak pg protocol and then transcribe it to wss. If there is a simpler way, let me know :)
- KingMob 3y agoCool article, thanks for sharing. You don't mention which version of HTTP you were using, which makes some of my comments here conjecture, but if I had to guess, I'd say you were using HTTP/1.1. Is that right? It would really help if you could break down where the time is being spent. I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc. Did you consider HTTP/2 or 3? I would expect them to have lower setup overhead, and be more competitive with Websockets. In particular, if you were using HTTP/2+, did you look into CONNECT proxying? It could replace Websockets while having lower connection setup overhead, albeit at the cost of a little payload unwrapping overhead in the proxy. But theoretically, this could offer both the faster startup of HTTP/2+, combined with the lower processing overhead for the db of a TCP stream. Comparing the graphs of 10 queries, it _appears_ that processing HTTP imposes a ~15 ms overhead over raw bytes. (Theoretically, the CONNECT proxy has to obey the host and port in the :authority pseudoheader, but in your case, the proxy would just ignore it, and hook up the edge function to whatever compute node it pleased.)
- deleted 3y ago[deleted]
- raoufchebri 3y ago> I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc. We experimented with SELECT 1, so most of the latency is transit
- KingMob 3y agoThat's good to rule out db differences, but that doesn't explain where in transit the 15 ms difference is between HTTP and WS for established connections. I'm assuming the most likely difference is in Pg processing an HTTP request over processing raw bytes, but there are other explanations, too. (Older/slower code pathway for HTTP in proxy, different intermediate node behavior for HTTP vs TCP from proxy to db, etc. Much less likely, but not impossible.)
- kelvich 3y agoYes, exploring http/3 is on the roadmap. One of the learnings for us was the fact that v8 is quite good with caching open TCP connection; For consecutive queries the same TCP+TLS session is being reused. Even when queries are sent from unrelated V8 isolates.
- Dhruva23 3y agohttps://news.ycombinator.com/item?id=33909616 https://news.ycombinator.com/item?id=33909616 history. Why were sockets used in the first place? Neon is slow and flaky I am thankful you are making changes but strongmanning your technical choice when you are wrong is concerning as a user. I still use Neon due to not being able to move back to heroku. I also you keep making open source more.
- PaulWaldman 3y agoIs there anything like this to expose Postgres over gRPC?
- tristan957 3y agoHow popular is gRPC in a serverless world?
- raoufchebri 3y agoHey @PaulWaldman, can you tell us more about your use case with gRPC and Postgres?
- PaulWaldman 3y agoPerformant streaming of structured telemetry data from Edge networks. Edge in this case meaning on-prem installations, not necessarily from a remote DC. These networks have limited network bandwidth and potentially intermittent connectivity, minimizing the message size and overhead as much as possible is ideal.
- dathinab 3y agoBut why? The edge is in general not the users browser. So you can do a TLS/postgres connection instead of adding additional layers. Sure you want to probably run a patched pg client/server which has various optimizations to reduce the number of round trips when opening a connection, but that also applies to any HTTP/WebSocket wrapper.
- gmac 3y ago> So you can do a TLS/postgres connection instead of adding additional layers. Sure, sometimes you can. But a key use-case for a serverless database is connecting from a serverless compute environment, and not all serverless compute environments support TCP connections. That's why Neon offers this serverless driver, which supports WebSockets and now also http.
- l5870uoo9y 3y agoHere is the actual package used [1]. I thought it would have been similar to serverless-pg[2] with initiation outside the handler as with the node implementation. Any insights into how they achieve connection caching (similar to pooling?) on the edge? [1]: https://github.com/neondatabase/serverless https://github.com/neondatabase/serverless [2]: https://github.com/MatteoGioioso/serverless-pg https://github.com/MatteoGioioso/serverless-pg
- raoufchebri 3y agoConnection caching is achieved on proxy side. Edge <-> Proxy<->Compute(Postgres)
- raoufchebri 3y agoThanks, I didn't know about serverless-pg. I'll have a look.
- CuriouslyC 3y agoThe biggest selling point of websockets over http is that you can stream the results back easily and quickly, which enables stuff like canceling slow queries easily, a transfer progress bar, etc.
- Camisa 3y agoI tried making a realtime multiplayer game network based on HTTP, sending movement data every 100ms and individual world changes on each HTTP requests by that was taking 50ms each on a wifi local network, I thought that was good enough but then I tested websockets and the round-trip for websocket was around 4ms.