3 ms·
I’ve been following that pattern in PostgreSQL and love it. It cleanly decouples the ingest and keeps the stack simple.
by klysm 4y ago
I’ve been following that pattern in PostgreSQL and love it. It cleanly decouples the ingest and keeps the stack simple.
- KingOfCoders 4y agoCan you elaborate?
- allan_s 4y agoNot the author, but if you have a write intensive workflow and I/O starts to be your blottleneck , compressing the data application-side allow your database to write less bytes, and thus to handle more load for the same capacity.
- OnlyMortal 4y agoBe aware of the number of iops. That’s a killer.
- rovr138 4y agoLook into ETL vs ELT ETL you grab the data from somewhere, transform it to clean it/format it, then load it (in this case to postgres). ELT, you grab the data, load it as is. Then you use another step to extract what you need or transform it. You keep the original data in case you need it there too.
- masklinn 4y agoDo you mean gencols or compression?
- klysm 4y agoI meant consuming the raw json into a jsonb column and then using views or triggers to maintain the structured version of it.