malisper

34 ms·

301.
My Approach to Getting Dramatically Better as a Programmer(malisper.me)
2 points by malisper 8y ago|0 comments
302.
by malisper 9y ago
That's reasonable. I usually avoid the higher isolation levels because they can cause queries to spontaneously fail. You need to be able to put in the extra work for handling the cases where they fail. If data correctness is an absolut
303.
by malisper 9y ago
Except mutex locking the rows is also vulnerable to read skew! For example, even if you lock the rows with SELECT ... FOR UPDATE: > CREATE TABLE reward_members (id bigint, member_status text); > INSERT INTO reward_members (v
304.
by malisper 9y ago
Head's up. Depending on the circumstances, multiple updates are vulnerable to a race condition known as "read skew". If a user's status changes while the multiple update transaction is executing, their status can wind up
305.
Testing Database Changes the Right Way(heapanalytics.com)
1 points by malisper 9y ago|0 comments
306.
Analyzing Millions of Postgres Query Plans(blog.heapanalytics.com)
1 points by malisper 9y ago|0 comments
307.
Analyzing Millions of SQL Queries When Each One Is a Special Snowflake(heap.engineering)
1 points by malisper 9y ago|0 comments
308.
There Is More to Programming Than Programming Languages(malisper.me)
2 points by malisper 9y ago|0 comments
309.
How to Get a Programming Job Straight Out of High School(malisper.me)
2 points by malisper 9y ago|0 comments
310.
How to Get a Programming Job Straight Out of High School(malisper.me)
3 points by malisper 9y ago|0 comments
311.
The Bw-Tree: A B-tree for New Hardware Platforms (2013)(microsoft.com)
2 points by malisper 9y ago|0 comments
312.
by malisper 9y ago
AFAICT the behavior of NOWAIT has always been the same. The earliest mention of NOWAIT I could find was in Postgres 8.1[0]: > To prevent the operation from waiting for other transactions to commit, use the NOWAIT option. SELECT FOR UPDAT
313.
by malisper 9y ago
I don't see how NOWAIT gives you what you want here. NOWAIT and SKIPPED LOCKED are completely different.
314.
by malisper 9y ago
Uh... NOWAIT will error if the query encounters a locked row and rollback the transaction. SKIP LOCKED will make locked rows invisible to the current transaction, but will not cause the transaction to be rolled back. SKIP LOCKED effectively
315.
The Missing Postgres Scan: The Loose Index Scan(malisper.me)
1 points by malisper 9y ago|0 comments
316.
by malisper 9y ago
We create one row per event logged in Heap. We currently have a default 3-year retention policy. We will keep your old data for up to three years. We have a redshift integration which you can use to export your data from Heap into redshift.
317.
by malisper 9y ago
While our data is ordered, a BRIN index doesn't support our use case. We need to perform non-zero amount of deletions/updates to existing data. This leaves holes in our table which will be filled in by new data, which makes a BRIN
318.
by malisper 9y ago
A single multi-column index would only work for the one specific query in the example. We have many queries that contain filters that are not easy to index such as regular expression filters. Additionally, a multi-column index would be sign
319.
by malisper 9y ago
One of the nice advantages of partial indexes is that Postgres will automatically fallback to the next best possible plan. If we created separate tables for each event, either we would have to handle the logic for falling back ourselves, or
320.
by malisper 9y ago
Yes, we have. The problem a lot of our keys and values are numeric. GIN/GiST only support equality for JSON fields. Additionally the GIN/GiST index winds up being significantly larger and is a bit slower than partial indexes.
321.
by malisper 9y ago
You can add more conditions and Postgres will use the index to first find the rows that match the partial index condition and then use the additional condition to postfilter the rows.
322.
by malisper 9y ago
I know Postgres really well because my job is mostly scaling Postgres. I got a good feeling of the tools of available by reading the Postgres manual. Then I got to learn when each tool is effective and the tradeoffs between each of them whi
323.
by malisper 9y ago
What do you mean by "filter out only interesting events"? The Heap philosophy is to capture all of the data up front and then make it easy for our customers to specify and query the subsets of the data they care about. That way ou
324.
by malisper 9y ago
> To index those effectively, you'd need to create a trigram index in Postgres, which takes up a TON of space. I actually ran some experiments on using a naive trigram index and found it doesn't currently don't work for ou
325.
by malisper 9y ago
My manager wrote a blog post explaining partial indexes and some of the advantages of them: https://blog.heapanalytics.com/speeding-up-postgresql-querie...
326.
by malisper 9y ago
> What sort of performance impact do these indexes have on writes? I'm guessing PostgreSQL has to look at evaluate each index predicate for the inserted data to see if the index needs to have an entry added. When we first deployed p
327.
by malisper 9y ago
Author here. Feel free to ask me any questions. By the way, the 10 million number in the title is outdated. It's now about 30 million indexes. Also, I've been publishing one blog post a weekday on my personal blog about one Postgr
328.
Transaction Id Wraparound in Postgres(malisper.me)
1 points by malisper 9y ago|0 comments
329.
The Story Behind Memsql's Skiplist Indexes(blog.memsql.com)
3 points by malisper 9y ago|0 comments
330.
by malisper 9y ago
Author here. We were under the assumption that CPU was mostly being used for evaluating the partial index predicates. Under this assumption, we figured batching was unlikely to yield much of a benefit. It wasn't until we actually profi

More ›