anarazel

23 ms·

241.
by anarazel 4y ago
My point is just that you can't set FETCH_COUNT with ALTER etc (as the post I was replying to suggested), because the server doesn't know anything about the parameter, as it just affects psql.
242.
by anarazel 4y ago
FETCH_COUNT is a psql side setting, not something you can configure server side.
243.
by anarazel 4y ago
That's not a panacea either. You either need the client connection file descriptor in another process, or incur the overhead of marshalling the query + query results over IPC.
244.
by anarazel 4y ago
> Right. One of several bits of architectural friction that didn’t matter while your large DBMS was I/O constrained. Yep. Lots of architectural design out there based on IO latency being higher by an order of magnitude or two than n
245.
by anarazel 4y ago
Yea, it's not something that's going to stop you dead in your tracks. But it does show up noticeably in profiles. There is stuff that you can do to reduce the pain, like using gigantic pages for the shared memory and remapping you
246.
by anarazel 4y ago
Postgres also uses a multi process architecture. But I think that turned out to be a mistake for something like a database, on modern systems. There are other reasons, but the biggest problem is that inter process context switches are consi
247.
by anarazel 4y ago
s/slowing/altering/ Damn you autocorrect.
248.
by anarazel 4y ago
We also batch the insertion into the table and the WAL logging, if possible. There's cases where that optimization has to be disabled though (presence of some types of triggers, for example).
249.
by anarazel 4y ago
They do have backing files - after all, unlogged tables can end up many times the size of memory. We need to reserve space for them, so there's a very high likelihood we could evict the buffers / shut down. But they're just w
250.
by anarazel 4y ago
Unlogged table data commonly won't even have been written to disk. And what is on disk is completely inconsistent after a crash. So really, the data is gone once postgres crashed (itself, due to hw issues, kill -9, ...)
251.
by anarazel 4y ago
Session state is a very common one, where the journaling overhead also can be particularly pronounced (much easier to hide in batch workloads).
252.
by anarazel 4y ago
Not yet clear. A lot of the infrastructure needed to support JSON_TABLE et al in a cleaner fashion, has now been added.
253.
by anarazel 4y ago
Wait events are perhaps inspired by oracle's stuff, at least indirectly. It was developed by Robert Haas and Amit Kapila, working for EDB. Traditionally a lot of their customers have migrated from Oracle. The redesigned way of storing
254.
by anarazel 4y ago
Android doesn't run highly concurrent workloads, so it's not surprising that you'd not fall over. But I haven't used it, so I really can't say anything well qualified. I've only used ETW a little, but I've
255.
by anarazel 4y ago
IME that doesn't work, because the un-aggregated data volumes are far too high. The whole point of using something like eBPF is that you don't need to log all that data.
256.
by anarazel 4y ago
Strongly agreed. In fact, I found an over-dimensioned workstation to be one of the best tools improving performance of non-trivial programs. There's a lot of performance issues that don't stand out sufficiently on today's com
257.
by anarazel 4y ago
> While it's true that user threads can't reset their signal masks without a system call, 1) SA_NODEFER means we don't have to, and 2) in theory, we could make an rseq field that would allow us to implement sigprocmask(2)
258.
by anarazel 4y ago
Re signals, see https://man7.org/linux/man-pages/man2/sigreturn.2.html - you'd at least end up with that. Re bpf: I don't think you can just defer the insertion into maps, that'd block executio
259.
by anarazel 4y ago
"Skewed" was perhaps not a good descriptor. What I mean is workloads where most of the time is spent in a small part of the program. Many programs have more flat profiles though. Profiling those with gdb doesn't work well. It
260.
by anarazel 4y ago
Which blog post is yours? You posted https://lists.fedoraproject.org/archives/list/devel@lists.fe... earlier - but I don't know if that's yours. Based on the approach there I do think you'd end up w
261.
by anarazel 4y ago
From memory, gcc's switch to omitting frame pointers by default on x86 was in the late 2000s - when amd64 was at least clearly on the winning path, if it hadn't already. I understand changing it on 32bit x86, but...
262.
by anarazel 4y ago
Shrug. Our experiences differ drastically then. IME profiling using the gdb style manual sampling method work kinda OK if you have a very skewed profile, but basically not at all after that. Sure, lots of people don't know how to use
263.
by anarazel 4y ago
> A similar "tuning best practice" that I still see applied in the field is database servers with dedicated data, log, temp, and templog drives... on cloud VMs where this is guaranteed to have worse performance than simply pool
264.
by anarazel 4y ago
> And if look what's going inside core postgres you will discover that almost all APIs of PG are duplicated in Rust so in future we'll see quite rusty PG What do you mean by this?
265.
by anarazel 4y ago
Sure, but that's not really counter to my point. An interrupt processed in the kernel isn't comparable to sending a signal to a userspace process, scheduling it (might not even be possible immediately!), walking the stack there, r
266.
by anarazel 4y ago
I suspect the dwarf issue from the post is just missing debug info for some library, or too deep stacks. But leaving that aside, for profiling, dwarf is just very expensive (perf, where the data volume explodes, due to copying stacks) or no
267.
by anarazel 4y ago
Heh. Looking for discussions in debian around this, I found a bug requesting a glibc build with frame pointers enabled. Reported by me. 8 years ago ...
268.
by anarazel 4y ago
Introducing a context switch, performing unwinding in process, with a relatively expensive method, would distort profiles sufficiently to make them not that useful anymore.
269.
by anarazel 4y ago
I did vaguely follow the effort - last time I checked nothing seems to have been merged yet. Did it get merged since? I'd expect it to benefit postgres substantially.
270.
by anarazel 4y ago
I only looked curiously at the relevant Linux code. But my understanding is that compound pages don't show up in the page table of a process. Yes, there's an associated data structure with each physical page, but that's separ

More ›