3 ms·

(Cockroach Labs CTO) Yes, the commit identifiers sound like something we could use, as long as they're not tied too closely to pg's WAL LSNs. For us, the token

(Cockroach Labs CTO) Yes, the commit identifiers sound like something we could use, as long as they're not tied too closely to pg's WAL LSNs. For us, the token would be a 96-bit hybrid logical timestamp, and we'd want semantics roughly similar to HTTP cookies (the server sends it back with the response, and then that client will include it on any future requests. With our current implementation the client would only need to send the token on new connections, but including room for it on a per-request basis seems like a good idea). Also on the subject of the protocol, we've run into a need for a (server-initiated) ping message (https://github.com/cockroachdb/cockroach/pull/10188 https://github.com/cockroachdb/cockroach/pull/10188).

anarazel 9y ago
> (Cockroach Labs CTO) Yes, the commit identifiers sound like something we could use, as long as they're not tied too closely to pg's WAL LSNs. There was some discussion around this before, and the idea was basically to just include some text-payload after COMMIT. PG would produce something like "COMMIT 343/3f0037a0", but the spec for would just be text. Some drivers might do something with the knowledge that it looks like an LSN, but that's hard to avoid. > Also on the subject of the protocol, we've run into a need for a (server-initiated) ping message (https://github.com/cockroachdb/cockroach/pull/10188 https://github.com/cockroachdb/cockroach/pull/10188). Maybe I'm missing something here (I admittedly only skimmed ticket and things it references), but shouldn't server triggered tcp keepalives be sufficient for this case? In postgres that's configured using tcp_keepalives_idle/tcp_keepalives_interval/tcp_keepalives_count. Several drivers (including at least libpq and pgjdbc) can do the same to protect against vanishing servers. Doing keepalives on the pg wire protocol level has some disadvantages, but also some issues. Namely you've to be ready to send ping/pongs forth/back in almost any protocol state. Not necessarily easy if there's partial writes and such.
- bdarnell 9y ago
  Ah, right. We did turn on TCP keepalives so we probably don't need a protocol-level ping (although we often deploy behind haproxy, so tcp keepalives need to be configured at each hop)
  - anarazel 9y ago
    Similarly, tcp level keepalive not necessarily sufficient, if there's intermediate "pgwire" protocol level connection poolers like pgbouncer. Nor do they protect against applications that are effectively hung somewhere. So I do think "pgwrite" level pings make some sense, it's just not exactly straightforward to add them :/

anarazel 9y ago

> (Cockroach Labs CTO) Yes, the commit identifiers sound like something we could use, as long as they're not tied too closely to pg's WAL LSNs. There was some discussion around this before, and the idea was basically to just include some text-payload after COMMIT. PG would produce something like "COMMIT 343/3f0037a0", but the spec for would just be text. Some drivers might do something with the knowledge that it looks like an LSN, but that's hard to avoid. > Also on the subject of the protocol, we've run into a need for a (server-initiated) ping message (https://github.com/cockroachdb/cockroach/pull/10188 https://github.com/cockroachdb/cockroach/pull/10188). Maybe I'm missing something here (I admittedly only skimmed ticket and things it references), but shouldn't server triggered tcp keepalives be sufficient for this case? In postgres that's configured using tcp_keepalives_idle/tcp_keepalives_interval/tcp_keepalives_count. Several drivers (including at least libpq and pgjdbc) can do the same to protect against vanishing servers. Doing keepalives on the pg wire protocol level has some disadvantages, but also some issues. Namely you've to be ready to send ping/pongs forth/back in almost any protocol state. Not necessarily easy if there's partial writes and such.

bdarnell 9y ago
Ah, right. We did turn on TCP keepalives so we probably don't need a protocol-level ping (although we often deploy behind haproxy, so tcp keepalives need to be configured at each hop)
- anarazel 9y ago
  Similarly, tcp level keepalive not necessarily sufficient, if there's intermediate "pgwire" protocol level connection poolers like pgbouncer. Nor do they protect against applications that are effectively hung somewhere. So I do think "pgwrite" level pings make some sense, it's just not exactly straightforward to add them :/