anarazel

8 ms·

61.
by anarazel 1y ago
> There are known scenarios in the literature that will cause Postgres to lose data, which TigerBeetle can detect and recover from. What are you referencing here?
62.
Compiler Explorer and the promise of URLs that last forever(xania.org)
357 points by anarazel 1y ago|189 comments
63.
by anarazel 1y ago
The numbers in the post highlight an issue I have with Samsung consumer SSDs - they've slowed down FUA writes to an absurd degree. open_datasync 249.578 ops/sec 4007 usecs/op fdatasy
64.
by anarazel 1y ago
> So PostgreSQL evolved to the point that it has a stable API for extensibility? Not across major versions, no. I seriously doubt we will ever make promises around that. It would hamper development way too much.
65.
by anarazel 1y ago
> Postgres offers strictly serializable isolation indeed, but IIUC it's basically a form of read locking so will tank performance. Postgres' SSI [1] does not block reads. If you have lots of transactions reading and updating a
66.
PostgreSQL 18 Beta 1 Released(postgresql.org)
2 points by anarazel 1y ago|0 comments
67.
by anarazel 1y ago
FWIW, there actually are some ongoing efforts towards that - including several preparatory changes in PG 18. Still lots more work, but we are working towards it. https://wiki.postgresql.org/wiki/Multithreading
68.
by anarazel 1y ago
It turns out to be the other way round, curiously. The bigger the reads (i.e. how much to read in one syscall) and the bigger the target area of the reads (how long before a target memory location is reused), the bigger the overhead of SMAP
69.
by anarazel 1y ago
From what I've seen a surprisingly large part of the overhead is due to SMAP when doing larger reads from the page cache - i.e. if I boot with clearcpuid=smap (not for prod use!), larger reads go significantly faster. On both Intel a
70.
by anarazel 1y ago
> Do you think there's a possibility of Direct IO being adopted at some point in the future now that AIO is available? Explicitly a goal. You can turn it on today, with a bunch of caveats (via debug_io_direct=data). If you have the
71.
by anarazel 1y ago
FWIW, I played with that - unfortunately it seems that the the overhead of doing twice the page cache lookups is a cure worse than the disease. Note that we do not offload IO to workers when doing I/O that the caller will synchronously
72.
by anarazel 1y ago
FWIW, there are prototype patches for an IOCP based io_method. We just couldn't get them into an acceptable state for PG 18. I barely survived getting in what we did...
73.
by anarazel 1y ago
They sped up things for a long time - but only when using unbuffered IO. The new thing with io_uring is that it also accelerates buffered IO. In the initial version it was all through kernel worker threads, but these days several filesyste
74.
by anarazel 1y ago
Intrusive lists are often used to enqueued pre-existing structures onto lists. And often the same object can be in different lists at different times. That's not realistically dealt with by the compiler re-organizing the struct layout.
75.
by anarazel 2y ago
Link to the option in https://news.ycombinator.com/item?id=43598217
76.
by anarazel 2y ago
> One gotcha about Postgres queries is that running queries do not get cancelled when the client disconnects You can configure that these days, at least if the server is running on some common platforms: https://www.postgresql
77.
by anarazel 2y ago
FWIW, you can parametrize statements on postgres without preparing them.
78.
by anarazel 2y ago
> Today you should do the same trick as a Linux futex although you spell it differently in Windows. It's also optionally smaller, which is nice for this sort of job, a futex costs 4 bytes but in Windows we can spend just one byte.
79.
by anarazel 2y ago
> That Craig person, the OP in the thread. Imagine doing all that work, having other people say, hey, that's an issue; and then all the people saying "so what" or "nothing can be done" For context - Craig's
80.
by anarazel 2y ago
> As such I'm occasionally bemused at the sort-of monoculture here around Postgres, where if Postgres doesn't have it, it may as well not exist. FWIW, I, as a medium-long term PG developer, are also regularly ... bemused by tha
81.
by anarazel 2y ago
It works even with that setting at zero! Just requires a bit more concurrency.
82.
by anarazel 2y ago
Concurrent commits can group commit with one WAL flush (i.e. one fsync/fdatasync).
83.
by anarazel 2y ago
Correct. Async commit transactions add their changes to the WAL the same way normal transactions do. The only relevant change is that the WAL is not synchronously flushed to disk before COMMIT completes.
84.
by anarazel 2y ago
Hah, I found something similar in gcc a few years back. Affecting postgres' expression evaluation interpreter: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71785
85.
by anarazel 2y ago
I have two Google accounts, one in the us, one in Germany. So far that has divided to get around this for the apps I encountered it. But I'm not a heavy phone user...
86.
by anarazel 2y ago
> I will try this on my own hardware as well FWIW, my results corresponded to: cpupower frequency-set --governor powersave && cpupower idle-set -E cpupower frequency-set --governor performance && cpupower idle-set
87.
by anarazel 2y ago
> Intermittent workloads on the order of 2 milliseconds, you mean? Yea. Most of the cases I was looking at were with postgres, with fully cached simple queries. Each taking << 10ms. The problem is more extreme if the client takes s
88.
by anarazel 2y ago
It performs terrible if you have an intermittent workload. Like e.g. a request response workload where request processing is cheap (so that the time to increase the frequency matters). I've seen cases it's a more than 2x request l
89.
by anarazel 2y ago
FWIW, the set of procedural languages in postgres is runtime extensible: https://www.postgresql.org/docs/current/sql-createlanguage.h...
90.
by anarazel 2y ago
The absurd thing about it is that IO bound code IME often is way easier to optimize than CPU bound code. Adding batching or pipelinining is often almost mechanical work and can give huge speedups.

More ›