3 ms·

38TB of large objects stored in Postgres right here

• Rapzid 3y ago
  A hero appears! The client I use currently, npgsql, supports proper streaming so I've created a FS->BLOB->PG storage abstraction. Streamy, dreamy goodness. McStreamy.
  • iamcreasy 3y ago
    I am confused and curious. Can you please elaborate on how streaming is related to pg storage?
    • Rapzid 3y ago
      If you can't stream data the user is sending through the server to the database you have to buffer the entire payload in memory. This doesn't scale well but can work for smaller payloads. You CAN stream bytea but: * Good luck finding a client that supports it(npgsql does actually) * It doesn't support support start the read/write at any location other than the start. So you'd have to basically build the Lob(Large Object) functionality yourself on top to support upload resume and other use cases. With Lob even if your client doesn't support streaming, you can mimick it by writing chunks into the Lob.
      • iamcreasy 3y ago
        What kind of streaming from user are we talking about? User interaction, such as mouse move, clicks? Does each interaction essentially create a SQL update statement? And are you trying to avoid buffering those statements?