felixge

11 ms·

91.
by felixge 9y ago
Are you aware of pg_repack [1]? I haven't had a chance to use it myself yet, but it's designed for reclaiming bloated space without downtime. [1] https://github.com/reorg/pg_repack/
92.
by felixge 9y ago
I guess you're just trying to show how to use MapD rather than showing when it makes sense to use it, which is cool. For the latter, I found the results over here fairly interesting: http://tech.marksblogg.com/benchmark
93.
by felixge 9y ago
100ms for this query is good compared to PostgreSQL (~4s when running the query as-is, ~275ms when using a window function) considering that MapD doesn't seem to utilize an index for it. That being said a 146MB CSV with 5 million rows
94.
by felixge 9y ago
Indeed. The actual story behind this research can be found here: http://tinyclouds.org/residency/ . JavaScript folks should also recognize the name of the researcher ;).
95.
by felixge 9y ago
> 2. WAN sucks. Where I work, I've seen mysql slaves get hours out of sync due to packet loss. Have you tried the new BBR congestion control for TCP? We've been seeing similar issues with PostgreSQL WAN replication and found th
96.
by felixge 9y ago
Good point. But the other questions didn't have a "Not enough information" choice. So I think the question should be improved to avoid false negatives in the results.
97.
by felixge 9y ago
I got 4/5 on PostgreSQL. I selected "Not enough information" on question 5, which is supposed to be the wrong answer. But I think it's the correct answer. Depending on the distribution of the data in the table (e.g. all
98.
by felixge 9y ago
I think the term you're looking for is cargo cult programming. Sure, applying keyset pagination prematurely can be a form of cargo cult programming, but OTOH I'd rather work with somebody who is aware of the different pagination o
99.
by felixge 9y ago
Yes, I think different companies invested that over the years. But perhaps more importantly it’s been 20+ years of development.
100.
by felixge 9y ago
Check out https://www.pgexercises.com/ . If you like learning by doing, it's a great resource.
101.
by felixge 9y ago
I admire SQLite as well. It's a fantastic piece of software with many use cases. You're correct that there are some mostly-ready-only situations where a file-lock isn't a problem (perhaps a small CMS, blog or similar). But us
102.
by felixge 9y ago
AFAIK SQLite doesn't achieve atomicity by virtue of its global write lock. It does so by using a rollback journal (default) or WAL. So it's not really all that different from PostgreSQL in that regard and the authors point remains
103.
by felixge 9y ago
I wasn't really proposing to change how visibility information is maintained, my grasp on the internals is far to weak for that :). I just tried to make sure I understood your comment, which I think I do now. So thanks again for anothe
104.
by felixge 9y ago
Thanks for the clarification about the visibility map. That being said, I don't understand why HOT would interfere with a hypothetical mechanism for marking index pages as all-visible. If a tuple gets a HOT update, I think the index va
105.
by felixge 9y ago
Not OP, but I think the answer is that PostgreSQL updates the the index right away, which points to the heap tuple (and it's visibility information). This index value can already be accessed by concurrent transactions, but they decide
106.
by felixge 9y ago
My e-mail is in my profile.
107.
by felixge 9y ago
Thanks for writing this up. As somebody who also likes to look into the internals of the PostgreSQL black box from time to time, posts like this are much appreciated :).
108.
by felixge 9y ago
Our actual system has the requirements of an existing DB schema that can't be changed, that's already using an events table, and that needs to be analyzed. I wrote some more about it here [1]. The FSMs are also cyclic, which can&#
109.
by felixge 9y ago
Hi, thanks for taking the time to explain your ideas. There are certainly many ways to model this problem in the database. But to me the main idea of the article is implementing a FSM as a user defined aggregate. This is technique is actual
110.
by felixge 9y ago
Apple | Backend (Go, PostgreSQL) engineer | Shanghai, China | ONSITE | Full Time We're a small team that is having a huge impact on the mass production of Apple's products. If you're an expert in Go or PostgreSQL (ideally bot
111.
by felixge 9y ago
I don't understand your alternative solution, nor your negativity. Would you still keep the events table, or just have a single orders table with the latest state?
112.
by felixge 9y ago
Yeah :).
113.
by felixge 9y ago
TBH I'm not sure if I follow your argument. You can certainly apply the main idea of this article (modeling a FSM as a user defined aggregate) to automatically materialize the latest state of each order. In fact, that's what we&#x
114.
by felixge 9y ago
Yeah, I think that'd be reasonable. My main motivation for using ENUMs is type safety and small column width. What you're proposing sounds like it would keep both of these advantages :)
115.
by felixge 9y ago
Good question. I'm working remotely myself and can't relocate easily because my wife is a doctor and it's very difficult for her to move between medical systems. I was lucky to join this team when this wasn't considered
116.
by felixge 9y ago
Sweet. That's certainly a nice improvement :). If removing ENUM values will be supported at some point as well, I can see myself recommending them unconditionally.
117.
by felixge 9y ago
Hah, I hear you :). Pushing this much logic into the DB can feel weird at times. And the application this is part of definitely has a lot of very complicated and large SQL queries. That being said, I think things can be somewhat tamed by ap
118.
by felixge 9y ago
Yeah, the idea of having a SQL standard is great. But in practice a "standard" is somewhat meaningless without rigorous compliance testing, and unfortunately SQL hasn't had this for 20 years [1]. But I'll take SQL over a
119.
by felixge 9y ago
Despite being pretty familiar with window functions, I couldn't figure out how to do it for that example in the time I was willing to spend on this post :). If you have a better query, I'm happy to update the article and give cred
120.
by felixge 9y ago
Author here: If the article above has you excited, come and join my team at Apple. We're hiring Go and PostgreSQL developers in Shanghai, China right now. Relocation is possible, just send me an e-mail to find out more about this role.

More ›