3 ms·

> Typical usecase: a large fleet of IOT devices with a write-only access to a central database, feeding new data every second. No visibility of the current tran

by taffer 6y ago

> Typical usecase: a large fleet of IOT devices with a write-only access to a central database, feeding new data every second. No visibility of the current transactions is needed. Can't you queue and serialize those writes in a middle layer so that you only need a small amount of connections?

anarazel 6y ago
Yea, normally you'd want some type of broker / middle-layer to accept the incoming data from the devices and then send it onward into the database. That broker would also authenticate / validate the devices / data etc. And it can do stuff like batch the input, for efficiency. If you actually need that many incoming connections, for now you'd probably want to use pgbouncer (or oddysey or ...) in transaction or statement pooling mode.
- 1996 6y ago
  Indeed, that's what I currently have, but for this database I'm looking at alternatives (ex: clickhouse) that could get everything and "send it onward" on a schedule. It's just a glorified bulk insert with extra moving parts, but if it works around the issue, why not? BTW Have you played with oddysey? It doesn't seem essentially different (or better) from pgbouncer.
  - anarazel 6y ago
    > BTW Have you played with oddysey? It doesn't seem essentially different (or better) from pgbouncer. pgbouncer is single threaded, oddysey isn't.
    - 1996 6y ago
      that is enough of a reason to try it, thanks!
skrtskrt 6y ago
Seems like a Kafka use case.