anarazel

40 ms·

451.
by anarazel 6y ago
That's the price of a heavily patched postgres fork. It's really expensive to maintain them. Far from the first time such forks fell behind quickly (e.g. redshift, greenplum).
452.
by anarazel 6y ago
Main Koch PAC: https://www.opensecrets.org/outsidespending/detail.php?cmte=... Total Independent Expenditures: $41,016,106 For Democrats: $92,936 Against Democrats: $8,887,525 For Republicans: $32,035,6
453.
by anarazel 6y ago
Can be changed using SET application_name = ...
454.
by anarazel 6y ago
Should have dug out the link earlier... The relevant README is https://github.com/postgres/postgres/blob/master/src/backend...
455.
by anarazel 6y ago
"It would be great if Postgres could emit a trace per query, showing in real-time which internal components were hit by this query. A sort of continuous query explain service."
456.
by anarazel 6y ago
Have you considered to add the span ID to the application name instead? Combined with including the application name in the log_line_prefix that will make the logs easier to inspect in quite a few cases. Also makes querying pg_stat_activity
457.
by anarazel 6y ago
> Are these usually used for monitoring (like a read-only api)? Yes. > I guess I'm asking if you can take postgres and "turn it inside out" by cherry-picking parts of it to build other storage software with it. Unfortun
458.
by anarazel 6y ago
Unfortunately some of the data for that isn't cheap to collect. Postgres' EXPLAIN (ANALYZE, SETTINGS) <query>, which shows a good bit of what you want, can slow queries down substantially (in some workloads). Primarily becau
459.
by anarazel 6y ago
What kind of in information would you like to see in that trace? Not planning to work on it directly, but I am planning to do some larger executor work as one of the next bigger projects, and having ideas about what kind of information peop
460.
by anarazel 6y ago
We were off by few in one of the crosschecks for a while. Turned out to just be voided mail-in ballots that in-person voters dropped off, which aren't supposed to be counted in the respective field. Which the manual doesn't really
461.
by anarazel 6y ago
> BTW Have you played with oddysey? It doesn't seem essentially different (or better) from pgbouncer. pgbouncer is single threaded, oddysey isn't.
462.
by anarazel 6y ago
Yea, normally you'd want some type of broker / middle-layer to accept the incoming data from the devices and then send it onward into the database. That broker would also authenticate / validate the devices / data etc. A
463.
by anarazel 6y ago
I think it would be neat to have - but personally I think there are higher priorities (asynchronous IO, faster executor, better buffer mapping, different connection model, built in automatic failover). And I don't think any of my colle
464.
by anarazel 6y ago
> Did Microsoft follow up your research with some Postgresql patches? Yes. I had committed them to the development branch before starting to write the post. There's a second post describing the concrete improvements that were commit
465.
by anarazel 6y ago
Author here, happy to answer questions & criticism.
466.
by anarazel 6y ago
> By “mirroring the includes” do you just mean listing the .h files in the build files? It's not just listing them once, so they get installed or such. You need to list them as dependencies for objects that get built. > I just fo
467.
by anarazel 6y ago
> can you elaborate? On what exactly? The freeze map bit I referenced above is the following commit: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... Commit a892234f830e832110f63fc0a2afce2fb21d1
468.
by anarazel 6y ago
> Surprised they didn't cover transaction id wraparound, which has caused more than a few public outages at scale. There have been some significant improvements in the last couple years - most importantly the introduction of the fre
469.
by anarazel 6y ago
I have never seriously used bazel. Just looking at it, I found the the thought of manually mirroring all #includes to the build configuration in a big project pretty aggravating. Is that not really an issue in practice? Do people automate t
470.
by anarazel 6y ago
> But even for 4500 connections idle you'd still want 500 cores and a monster IO subsystem? Entirely depends on your workload: These days it's very common to have the hot working set fit into the database server's buffer c
471.
by anarazel 6y ago
I've not seen any real efforts to add support for different protocols to postgres. Do you really think just adding protocol level support for e.g. TDS would be useful, if the the SQL dialect still was postgres's? If so - why? Whil
472.
by anarazel 6y ago
> Think you're mixing up parallelism and concurrency. Other databases with connections that support pipelining can easily get to high concurrency even with a single core. The key is the cpu can kick off 100 queries to a single conne
473.
by anarazel 6y ago
> I always see people recommending external pools like this here on HN. I'm wondering a bit about that as many frameworks/libraries that use Postgres already implement an internal connection pool. Is this recommendation general
474.
by anarazel 6y ago
> This suggests that the various companies employing people to work on Postgres-related stuff (like Microsoft, perhaps due to their purchase of Citus) have more lucrative work they'd rather do instead of improve this at the design l
475.
by anarazel 6y ago
TBH, early this year I tried pretty hard to get these changes into PG 13 (released a few weeks ago). Missed the mark by a week or two. So this amount of time between complex commits and release isn't quite the norm. Still would have be
476.
by anarazel 6y ago
You might be thinking of > In POSTGRES they are run as subprocesses managed by the POSTMASTER. A last aspect of our design concerns the operating system process structure. Currently, POSTGRES runs as one process for each active user. Thi
477.
by anarazel 6y ago
It's extremely rare for workloads that need high connection counts to utilize every connection to the extent that they're practically never idle. To the contrary, usually the majority are idle - latency alone leads to that, given
478.
by anarazel 6y ago
The postgres protocol does support pipelining of queries - and it can be a huge boon in latency sensitive workloads. They'll get processed in-order on the server side, with results being sent back while the next query is being processe
479.
by anarazel 6y ago
> I do not understand why people aren't clamouring against postgres's connection model. There are people wanting to change that - including me, the author of the blog post. I explained in an earlier blog post ([1]) why I chose
480.
Improving Postgres Connection Scalability: Snapshots(citusdata.com)
4 points by anarazel 6y ago|0 comments

More ›