4 ms·

"Python does not cater to my favourite edge case" != fundamental problem In the days before UTF-8-everywhere, file names with anything besides alphanumerics an

by mikl 7y ago

"Python does not cater to my favourite edge case" != fundamental problem In the days before UTF-8-everywhere, file names with anything besides alphanumerics and safe symbols like dashes or underscores were always a problem. If you had special characters in your filenames, you were almost certain to run in to problems, since their meaning varied greatly depending on how the system was set up - codepages and whatnot. But this is only a problem if you have such files, and unless you’ve kept files around for decades, you don’t. So young programmers can grow up never having problems with this. Everything will be UTF-8 and it’ll just work. And as for broken old file names, who cares? Fix your file names and move on. There’s no reason that Python 3 should have workarounds for problems that were solved over a decade ago.

KaiserPro 7y ago
> And as for broken old file names, who cares? Fix your file names and move on. what if you are trying to process "foreign"(as in, not created by you) filenames, trying to validate/conform with python? I mean its a great DoS vector, which is difficult to protect against with python. it'll crash. Which the point the article is trying to make. > unless you’ve kept files around for decades, you don’t. Thats both untrue and not very helpful. Its perfectly possible to bump into files like this.
- viraptor 7y ago
  > trying to validate/conform with python Then you can use the 'bytes' variant of calls. If you process arbitrary data, there's lots of things you need to think about. This is just one of them. And if you'd crash otherwise, you should have exception handling in place. I understand it's a problem and it's not trivial to support this, but it's not difficult either.
- oefrha 7y ago
  > what if you are trying to process "foreign"(as in, not created by you) filenames, trying to validate/conform with python? I mean its a great DoS vector, which is difficult to protect against with python. 1. Have catch-all exception handling. Exception may not be your fault, exception-caused "crashing" whatever that means is entirely your fault. 2. Use os.fsdecode. https://docs.python.org/3/library/os.html#os.fsdecode https://docs.python.org/3/library/os.html#os.fsdecode 3. Don't process random untrusted filenames. Sanitize if it's some sort of HTTP upload, for instance.
- mikl 7y ago
  > Its perfectly possible to bump into files like this. Possible, sure, but highly unlikely. Unless you consider black-hat hackers, it’s pretty rare for modern software to be fed decades-old files.
coldtea 7y ago
>"Python does not cater to my favourite edge case" != fundamental problem Broken logic for a very common case != "Python does not cater to my favourite edge case" >And as for broken old file names, who cares? People with actual products / production code. >Fix your file names and move on. Is this satire?

by mikl 7y ago

KaiserPro 7y ago

> And as for broken old file names, who cares? Fix your file names and move on. what if you are trying to process "foreign"(as in, not created by you) filenames, trying to validate/conform with python? I mean its a great DoS vector, which is difficult to protect against with python. it'll crash. Which the point the article is trying to make. > unless you’ve kept files around for decades, you don’t. Thats both untrue and not very helpful. Its perfectly possible to bump into files like this.

viraptor 7y ago
> trying to validate/conform with python Then you can use the 'bytes' variant of calls. If you process arbitrary data, there's lots of things you need to think about. This is just one of them. And if you'd crash otherwise, you should have exception handling in place. I understand it's a problem and it's not trivial to support this, but it's not difficult either.
oefrha 7y ago
> what if you are trying to process "foreign"(as in, not created by you) filenames, trying to validate/conform with python? I mean its a great DoS vector, which is difficult to protect against with python. 1. Have catch-all exception handling. Exception may not be your fault, exception-caused "crashing" whatever that means is entirely your fault. 2. Use os.fsdecode. https://docs.python.org/3/library/os.html#os.fsdecode https://docs.python.org/3/library/os.html#os.fsdecode 3. Don't process random untrusted filenames. Sanitize if it's some sort of HTTP upload, for instance.
mikl 7y ago
> Its perfectly possible to bump into files like this. Possible, sure, but highly unlikely. Unless you consider black-hat hackers, it’s pretty rare for modern software to be fed decades-old files.

coldtea 7y ago

>"Python does not cater to my favourite edge case" != fundamental problem Broken logic for a very common case != "Python does not cater to my favourite edge case" >And as for broken old file names, who cares? People with actual products / production code. >Fix your file names and move on. Is this satire?