anarazel

66 ms·

721.
by anarazel 8y ago
It's near trivial to exhaust resources when you have SQL access. That's not information disclosure, but it's darn painful if the set of users includes a sizable enough group of bad faith actors.
722.
by anarazel 8y ago
Postgres has offered F_FULLFSYNC on OSX since 2005?
723.
by anarazel 8y ago
Turns out people don't like one baseless accusation after another. I provided you with concrete references where review comments where made, where people agreed it was necessary. Yet you deny that happened, without refuting anything co
724.
by anarazel 8y ago
> Also nobody has mention data checksum added v9.3, do you know if this helps avoid this kind of fsync related corruption? Not really, I think. Page-level checksums don't protect against entire writes going missing, unfortunately.
725.
by anarazel 8y ago
Please cite which parts of the thread you're referring to. Again, several people +1'd the idea, and reviewed the code.
726.
by anarazel 8y ago
Paper based voting also excludes me, a Swiss person living abroad, from reliably voting. I don't understand why these discussions are done in this black/white manner. There's valid policy discussions to be had, but treating t
727.
by anarazel 8y ago
Meh. Mail voting shares that part, so removing it from the scope of testing for electronic voting makes plenty sense.
728.
by anarazel 8y ago
I have serious concerns about electronic voting. But: One of the predecessor systems allows me to vote in referenda from abroad, and it significantly increases the likelihood I vote successfully. Having to send back the paper ballot weeks i
729.
by anarazel 8y ago
> Isn't one of the difficulties the inability to decide when the I/O error is transient (e.g. running out of space with thin provisioning) vs. permanent (e.g. drive eaten by a velociraptor)? I don't really think so. The t
730.
by anarazel 8y ago
I think the "big project" angle is that maintainability is of higher priority than for smaller project. So more minor things need to to be addressed than in smaller projects. And that then either leads to being very nitpicky in re
731.
by anarazel 8y ago
You're basically just making shit up at this point. The most critical message in that thread contained pieces like https://news.ycombinator.com/item?id=19128154
732.
by anarazel 8y ago
Newer versions of linux (but not plenty of other OSs) guarantee that each write failure will be signalled once to each file descriptor open before the error occurred. So that ought to be handled, unless the FD is closed inbetween.
733.
by anarazel 8y ago
Yea, but the largest portion of databases these days is not deployed on dedicated hardware that's been carefully capacity planned, and has a dedicated DBA.
734.
by anarazel 8y ago
Is that really justification for incurring unsignalled dataloss? If that's actually common enough, count the number of uncleared errors on per-mount basis, and shut down the filesystem if the memory pressure gets too high while signif
735.
by anarazel 8y ago
> I don't think the on-disk size matters all that much, TBH, it's likely just a tiny fraction of data stored in the cluster. I've seen pg_attribute take up very significant fractions of the database numerous times, so I do
736.
by anarazel 8y ago
I think the only real defense of the linux et al behaviour here is that: a) there's no standardized way to recover from write errors when not removing dirty data. I personally find that more than an acceptable tradeoff, but obviously
737.
by anarazel 8y ago
> Is it ~140 bytes? pahole says it's 112 (without CATALOG_VARLEN). Well, but on-disk varlena data is included. pg_column_size() averages 144 bytes for pg_attribute on my system. > The impact of doubling NameData size would be qu
738.
by anarazel 8y ago
Why are: > 1. If writes need to be retried, why not reads? (No, that's not an invitation to expand the scope of the patch; it's a question about NFS implementation.) and > 4. As coded, the patch behaves incorrectly if you g
739.
by anarazel 8y ago
> It's random because pretty much anyone could volunteer that opinion. By diagnosing the problem, making a patch and reaching out the author has already made their intention of fixing the problem clear. Unless it has been establishe
740.
by anarazel 8y ago
> > Review in the PostgreSQL community does tend to be on the "harsh" side. > Can you share an example of a review that you consider harsh? (feel free to share privately, I'm simply interested in what you consider h
741.
by anarazel 8y ago
Yea, I think pg_attribute is likely to be the main issue here. For one, it obviously exists many times per table, and there are workloads with a lot of tables. But also importantly it's included in all tuple descriptors, which in tur
742.
by anarazel 8y ago
> Agreeing that there is something worthy of fixing is a first step. It should have happened with this NFS patch and imo some other stuff. The considerations for how and when should be dear with separately. But there were like multiple p
743.
by anarazel 8y ago
I don't think that's really true. It worked well enough, true, but I think it allowed us to not fix deficiencies in a number of areas that we should just have fixed. IOW, I think we survived despite not offering DIO (because other
744.
by anarazel 8y ago
Yea, that's definitely something that happens. Partially due to some conservatism, partially due to having way more patches than review bandwidth, ... > Another example: NAMEDATALEN restricts every identifier to 63 bytes - a limit t
745.
by anarazel 8y ago
Hm? I explicitly said that retries would be better?
746.
by anarazel 8y ago
Note that that patch doesn't really fix the issue. You can write();close();open();fsync(); and you'll miss the issue if the OS failed during writeback before the open(). That's worse than on new-ish linux, where at least it&#
747.
by anarazel 8y ago
I'm a bit confused by this perception of the thread. There was one person doubtful the patch is the right approach (Tom), but still offered concrete review comments (of actual issues that'd at the very least need to be documented
748.
by anarazel 8y ago
Turns out sometimes people other than Linus have more experience with IO than Linus. I think there's pretty good reasons to go for DIO for a database. But only when there's a good sysadmin/DBA and when the system is dedicated
749.
by anarazel 8y ago
The relevant thread: https://www.postgresql.org/message-id/46ED7555-9383-45D2-A5C... There's literally one person saying that NFS is crapshot anyway, and several +1s for retries. And the former was accompanied wit
750.
by anarazel 8y ago
That justification is bogus however. There's already separate logic for the case the entire underlying device vanishes.

More ›