anarazel

41 ms·

481.
by anarazel 6y ago
There's been a lot of statements about the efficacy of parts of PRISM, just not about the metadata collection. Obviously not the end-all-be-all but the PCLOB reports about 215 and 702 are worth skimming. 215: https://documen
482.
by anarazel 6y ago
> The title is misleading The HN title certainly seems to be. My understanding is that 9th circuit opinion was talking about the bulk phone metadata collection program (which used to be enabled under section 215 and even done before (&qu
483.
by anarazel 6y ago
That doesn't actually address the "overhead of an idle connection" bit meaningfully: https://techcommunity.microsoft.com/t5/azure-database-for-po...
484.
by anarazel 6y ago
> I can completely imagine the corner cases this ends up touching, as there must be an insane amount of logic in Postgres that assumes immutability of data on disk: if I read data block X for transaction version 1234, it will always rema
485.
by anarazel 6y ago
> must be crazy if they're not aggressively using pgbouncer to keep the number of connections down. With the limitations around PG poolers that's not always that easy :( I've worked on a number of very large postgres insta
486.
by anarazel 6y ago
> I thought process forking was a fundamental aspect of Posgres' architecture. Is it correct to read above as effectively rearchitecting Postgres? Is it a small change? Certainly not. But it'd also not touch the majority of the
487.
What’s new in pg_auto_failover 1.4 for Postgres high availability(citusdata.com)
5 points by anarazel 6y ago|0 comments
488.
by anarazel 6y ago
> A fork() call can be relatively cheap, but is rarely as cheap as an accept(). Agreed, obviously. While I personally am not that concerned with the cost of connection establishment, and much more concerned with the context switches in a
489.
by anarazel 6y ago
I really didn't want to give the impression that I think poolers are useless - I don't, I think they're often crucial. Sorry if I did so. > while simply copying the shared buffers table into a new process represents a real
490.
by anarazel 6y ago
Pooling has a huge effect in this regard when using statement level pooling. Which isn't suitable for all that many workloads. With transaction level pooling it's still significant, but latency / processing effects become sig
491.
by anarazel 6y ago
The < 2MB figure is with huge pages (2MB ones in this case) enabled. I used 16GB of shared_buffers. The measurement at the end shows how much of a difference in memory usage that makes: The page tables use 6.5MB per process without huge
492.
by anarazel 6y ago
> Oltp databases typically cannot process more than 5 x cpu cores without having to time-slice among cpus I think this is entirely wrong in reality. In most read-mostly OLTP workloads, due to client<->server latency and application
493.
by anarazel 6y ago
> a) You can't hold 5000 open connections out of which 4900 are idle without performance impact. Author here. I agree. In fact this blog post only exists because a blog post on analyzing the various bottleneck around connection scal
494.
by anarazel 6y ago
If anybody has a good idea of how to accurately subtract memory that is shared between processes... It's doable with /proc/[pid]/pagemap + /proc/kpagecount, but that's fairly expensive...
495.
by anarazel 6y ago
Author here, if you have questions...
496.
Measuring the Memory Overhead of a Postgres Connection(blog.anarazel.de)
4 points by anarazel 6y ago|0 comments
497.
by anarazel 6y ago
Amazon is not exactly a large contributor to postgres...
498.
by anarazel 6y ago
Note that the postgres core team aren't all developing postgres itself, and that there's a vastly larger number of people working on postgres than the core team. They're basically "just" a body to escalate things to
499.
by anarazel 6y ago
Probably not helpful, and obviously one of the more archaic parts of postgres, but here's the concrete encoding: https://github.com/postgres/postgres/blob/master/src/include...
500.
by anarazel 6y ago
For on-disk storage (in memory largely is different, although the on-disk format is used in a few places too), variable length datums are stored as 'varlenas'. Basically there's a 'varlena header' indicating the len
501.
by anarazel 6y ago
I can't sit upright in the back of a Prius. And I'm not that tall.
502.
by anarazel 6y ago
You don't necessarily need to. You might just transparently use them in a lot of cases. In others you'll just read the docs when you need to do something specific.
503.
by anarazel 6y ago
>> Maybe you don't live in California but it's still full draconian lockdown here Also, draconian? What? In SF restaurants with outside seating are open and populated. State and national parks are largely open (unless closed
504.
by anarazel 6y ago
> Sweden looks great. They have higher death rates than their neighboring countries. While the economic outcome doesn't appear to be better than the surrounding countries. > South Dakota looks great. https://covidtrack
505.
by anarazel 6y ago
Yes, I am desperate trying to prove Trump wrong by not putting a temporary ~3 month Trump administration regulation on the same level as all the post 9/11 measures. Absolutely. I don't think this is particularly likely to hold u
506.
by anarazel 6y ago
I think this is pretty over the top.
507.
by anarazel 6y ago
The claim is that it derives from public health emergency powers. Arguing that evicting people increases the infection risk, therefore falling within CDCs powers. But that indeed seems to be stretching things to somewhere around the breakin
508.
by anarazel 6y ago
Thought the same at first, but there's a second clearer paragraph more in line with the title: > In order to keep SQLite completely free and unencumbered by copyright, the project does not accept patches
509.
by anarazel 6y ago
There's plenty performance work going on in postgres. Most major releases had significant improvements in some aspect of that.
510.
by anarazel 6y ago
Well, there are branches that just apply bug fixes. We have five years of support for our (yearly) releases. Optimizations kind of are features (sometimes kinda bugs too). Why should those be singled out vs other features? The primary &quot

More ›