3 ms·
Just wondering if this problem could have been solved by a properly indexed table? The article says: “Eight minutes and 4+ terabytes of data queried later I had
by fpbarthel 7y ago
Just wondering if this problem could have been solved by a properly indexed table? The article says: “Eight minutes and 4+ terabytes of data queried later I had my results“. 4+ TB seems way too much for 60k patients and sounds like an inefficient table scan was performed.
- fpbarthel 7y agoAlso, wouldn’t partitioning only make sense if there is a sensible way to separate data that is more likely to be accessed vs data less likely to be accessed? Like is common with date data, since recent entries are often more relevant compared to old entries. For example, you could categorize SNPs by priority and eg. partition SNPs of high importance (frequently accessed) vs medium importance (sometimes accessed) vs low importance (rarely accessed).