4 ms·

I remember ages ago getting in an argument with the kernel hackers on unix-wizards. My position was that filenames shouldn't be allowed to have meta-characters

I remember ages ago getting in an argument with the kernel hackers on unix-wizards. My position was that filenames shouldn't be allowed to have meta-characters in them because it made shell programming difficult and dangerous. Their argument was that the spirit of Unix was that the kernel should be as simple as it possibly could be, and that you should "Live free or die". Therefore it was not the kernel's business to dictate what characters should be able to be in a filename. They were wrong of course. The true spirit of Unix was to enable programmer productivity by being able to easily piece together new programs out of reusable components. E.g., via shell scripting, and the kernel hackers' notions about filenames and living free was directly at odds with this true spirit. The OP claims that they've solved this dilemma in Julia. I await with bated breath.

rcthompson 15y ago
How can the kernel possibly dictate what filenames are acceptable if it wants to interoperate with filesystems created by other operating systems?
- nessus42 15y ago
  if it wants to interoperate with filesystems created by other operating systems? That, of course, was not an issue back in the day, as Unix did not in fact support anything such thing. As for today, please do tell me how Unix can interoperate with filesystems that allow nulls and/or forward slashes in filenames. I'm sure that you might come up with an existing or potential solution, and well, there's your answer!
perlgeek 15y ago
Note that different shells have different meta characters. Which one would you forbid? All of them? Then I'm sure somebody comes up with a shell where nearly all characters are meta characters, leaving only one or two to make up file names.
- nessus42 15y ago
  Back in the day, there were only two shells: The Bourne Shell and the C Shell. And before that there was only whatever the Bourne Shell was based on. In any case, the answer to this is that the kernel developers and the shell developers get together and decide on something, and then future shells have to live within those constraints. At the point that they should have done this, they should have made sure that this was flexible enough for future needs. Personally, I would have been fine if filenames were restricted to more or less to the same degree that variables names are restricted in, let's say, C. Why can't I name my variables whatever I want to? I thought it was "Live free or die!"?!?!?!?!
  - jarman 15y ago
    YES! And while we are at it, let's get rid of SQL injections by forbidding spaces and quotes in [VAR]CHAR fields! Limiting valid file names won't protect from injections, won't make such protection easier (asserting lack of special characters isn't significantly easier than escaping same characters), but will cripple system functionality. Personally_comma_I_would_hate_seeing_such_file_names.
    - nessus42 15y ago
      This discussion has ABSOLUTELY NOTHING to do with SQL injection. That is a completely different issue, as you may indeed want and need to put all of War and Peace into a char field, but there is no valid use case for putting all of War and Peace into a filename. Furthermore, it was quite traditional for Unix filenames to be written exactly has you have done so above. (Although dashes would have been more commonly used than underscores.) And, in fact, it was considered fairly anti-social to put spaces, carriage returns, etc., in your filenames, as Unix utilities could not cope with them. The fact that for decades Unix utilities could only cope with filenames written as you have above, proves my point, not yours. Even now, when Unix utilities are finally getting to the point where they might be able to cope with any combination of characters in a filename (other than null or "/"), this has come at great extra complexity and requiring much extra care on the part of the programmer. Which again proves my point, not yours. Now, of course, whenever anyone asserts that something requires too much care, someone else will quite wrongly assert that programmers should not be encouraged to be lazy. That real programmers should work for a living. This is, of course, absurd. The best programmers will always be lazy... in the right way. They will want to solve a problem that accomplishes the most, in the least amount of time, with the least amount of code. I.e., they will want to be productive. People who make quips about lazy programmers are doing a huge disservice to the world by promulgating a world with less productive programmers who consequently accomplish less. Which means that, for all you know, cancer won't be cured when otherwise it might have been. As to your proposed solution of having the shell escape special characters, rather than having the kernel disallow them, that could probably work too, at the cost of a consistency. I.e., when you look at a filename in the shell, it's going to look different from how it will look in your GUI browser. E.g., the shell is going to display your filename in the manner that you have done, only it probably won't do as good a job as you would at encoding your intent into a limited character set. And, if you are going to take this approach, it doesn't have to be by allowing special characters in filenames. It can be done either by encoding special characters in non-special characters, or by having a display name that is different from the filename. Additionally, I find it amusing when people assert that having limitations in filenames is crippling system functionality, and yet they don't make the same assertion about identifier names in programming languages. Oh no! Having a limited set of characters in variable names has crippled my ability to program... NOT! In summary, history has proved me right.
tesseract 15y ago
> The OP claims that they've solved this dilemma in Julia. I await with bated breath. The obvious solution would seem to be to provide an easy way to directly create a pipeline from within Julia. I.e. you use syntax similar to shell syntax, but rather than spawning a shell Julia spawns the desired processes directly. That way it can directly pass arguments to execv (avoiding escaping issues altogether), wait on all of the child processes individually (allowing clean and detailed error detection), and avoid the overhead of actually starting a shell.
- deleted 15y ago
  [deleted]

rcthompson 15y ago

How can the kernel possibly dictate what filenames are acceptable if it wants to interoperate with filesystems created by other operating systems?

nessus42 15y ago
if it wants to interoperate with filesystems created by other operating systems? That, of course, was not an issue back in the day, as Unix did not in fact support anything such thing. As for today, please do tell me how Unix can interoperate with filesystems that allow nulls and/or forward slashes in filenames. I'm sure that you might come up with an existing or potential solution, and well, there's your answer!

perlgeek 15y ago

Note that different shells have different meta characters. Which one would you forbid? All of them? Then I'm sure somebody comes up with a shell where nearly all characters are meta characters, leaving only one or two to make up file names.

nessus42 15y ago
Back in the day, there were only two shells: The Bourne Shell and the C Shell. And before that there was only whatever the Bourne Shell was based on. In any case, the answer to this is that the kernel developers and the shell developers get together and decide on something, and then future shells have to live within those constraints. At the point that they should have done this, they should have made sure that this was flexible enough for future needs. Personally, I would have been fine if filenames were restricted to more or less to the same degree that variables names are restricted in, let's say, C. Why can't I name my variables whatever I want to? I thought it was "Live free or die!"?!?!?!?!
- jarman 15y ago
  YES! And while we are at it, let's get rid of SQL injections by forbidding spaces and quotes in [VAR]CHAR fields! Limiting valid file names won't protect from injections, won't make such protection easier (asserting lack of special characters isn't significantly easier than escaping same characters), but will cripple system functionality. Personally_comma_I_would_hate_seeing_such_file_names.
  - nessus42 15y ago
    This discussion has ABSOLUTELY NOTHING to do with SQL injection. That is a completely different issue, as you may indeed want and need to put all of War and Peace into a char field, but there is no valid use case for putting all of War and Peace into a filename. Furthermore, it was quite traditional for Unix filenames to be written exactly has you have done so above. (Although dashes would have been more commonly used than underscores.) And, in fact, it was considered fairly anti-social to put spaces, carriage returns, etc., in your filenames, as Unix utilities could not cope with them. The fact that for decades Unix utilities could only cope with filenames written as you have above, proves my point, not yours. Even now, when Unix utilities are finally getting to the point where they might be able to cope with any combination of characters in a filename (other than null or "/"), this has come at great extra complexity and requiring much extra care on the part of the programmer. Which again proves my point, not yours. Now, of course, whenever anyone asserts that something requires too much care, someone else will quite wrongly assert that programmers should not be encouraged to be lazy. That real programmers should work for a living. This is, of course, absurd. The best programmers will always be lazy... in the right way. They will want to solve a problem that accomplishes the most, in the least amount of time, with the least amount of code. I.e., they will want to be productive. People who make quips about lazy programmers are doing a huge disservice to the world by promulgating a world with less productive programmers who consequently accomplish less. Which means that, for all you know, cancer won't be cured when otherwise it might have been. As to your proposed solution of having the shell escape special characters, rather than having the kernel disallow them, that could probably work too, at the cost of a consistency. I.e., when you look at a filename in the shell, it's going to look different from how it will look in your GUI browser. E.g., the shell is going to display your filename in the manner that you have done, only it probably won't do as good a job as you would at encoding your intent into a limited character set. And, if you are going to take this approach, it doesn't have to be by allowing special characters in filenames. It can be done either by encoding special characters in non-special characters, or by having a display name that is different from the filename. Additionally, I find it amusing when people assert that having limitations in filenames is crippling system functionality, and yet they don't make the same assertion about identifier names in programming languages. Oh no! Having a limited set of characters in variable names has crippled my ability to program... NOT! In summary, history has proved me right.

tesseract 15y ago

> The OP claims that they've solved this dilemma in Julia. I await with bated breath. The obvious solution would seem to be to provide an easy way to directly create a pipeline from within Julia. I.e. you use syntax similar to shell syntax, but rather than spawning a shell Julia spawns the desired processes directly. That way it can directly pass arguments to execv (avoiding escaping issues altogether), wait on all of the child processes individually (allowing clean and detailed error detection), and avoid the overhead of actually starting a shell.

deleted 15y ago
[deleted]