4 ms·
It’s too bad that the code and correction seem to be locked down, but the linked Twitter discussion has an explanation [1]: the code expects Python’s glob to re
by boulos 7y ago
It’s too bad that the code and correction seem to be locked down, but the linked Twitter discussion has an explanation [1]: the code expects Python’s glob to return a sorted list of files, but it doesn’t guarantee this (the main fix is to sort the result). I had thought reading the Vice article that this would be about case-sensitive vs insensitive filesystems.
[1] https://mobile.twitter.com/bmarwell/status/1181821143835201536 https://mobile.twitter.com/bmarwell/status/11818211438352015...
- contravariant 7y agoTo me this just raises more questions like "Why does the order of the files even matter?", and "If it does why wouldn't you make sure it was sorted, or better yet sort it in some reliable fashion?".
- lupire 7y agoIf you are writing a paper about the order in which a sequence of events happens, the order matters. It's the difference between cause and effect, the heart of science. Filename is often used as part of the data, as a timestamp. Obviously reliable sorting is important, which is why people are upset that python's glob() turned out to be unreliable
- luckylion 7y ago> Obviously reliable sorting is important, which is why people are upset that python's glob() turned out to be unreliable A quick glance at the documentation didn't mention it, does glob even promise sorting the result? Relying on that to happen when it's not even promised seems negligent.
- magicbuzz 7y agoMost developers look at the result from a function call rather than looking at the documentation.
- perl4ever 7y agoI mean, I have learned that people simply won't accept that SQL doesn't guarantee the order of a result set if you don't use "order by". I tried to argue it and eventually gave up. It seems vastly easier to change a language than human nature. People are hardwired to ignore claims that something should not be relied on, because in normal human relations, that is a way of pushing away undue responsibility, not something to be taken literally.
- bmc7505 7y ago> A quick glance at the documentation didn't mention it What do you mean? It is the very first sentence of the documentation: "results are returned in arbitrary order" https://docs.python.org/2/library/glob.html https://docs.python.org/2/library/glob.html
- luckylion 7y agoAh, see, I missed that, I had just skimmed, then grepped for "sort" and "order" (but mistyped that as oder).
- emmelaich 7y agoIt doesn't sort. But people assume it does especially if the files are returned in the order that they were created, which typically will be sorted. I think this is why it doesn't get noticed; the files are written say result-20190901-1303 result-20190901-1305 result-20190901-1310 etc...
- magicbuzz 7y agoIIRC glob.glob() is a wrapper around os.listdir(). I used to develop code on an OS X laptop then push to a Linux box for the actual computational result (big dataset). And discovered at one point that one OS returns a sorted result, the other does not. Sorting is important if you want track/analyse instrument data over the days in a year.
- smnrchrds 7y agoAre they locked down? The article links to the paper [0], which has two zip files to download, first of which is the code, including an explanation of the corrections made [1]. [0] https://pubs.acs.org/doi/10.1021/acs.orglett.9b03216 https://pubs.acs.org/doi/10.1021/acs.orglett.9b03216 [1] https://pubs.acs.org/doi/suppl/10.1021/acs.orglett.9b03216/suppl_file/ol9b03216_si_002.zip https://pubs.acs.org/doi/suppl/10.1021/acs.orglett.9b03216/s...
- boulos 7y agoAhh, I failed to click those. They seemed large enough to be “not the code, but the full dataset”. Thanks for the correction!