4 ms·

Ignoring the fact that non-terminal users regularly want spaces, hyphens and newlines in their filenames since they have visual purpose... The argument in favo

Ignoring the fact that non-terminal users regularly want spaces, hyphens and newlines in their filenames since they have visual purpose... The argument in favor of newlines is the same argument in favor of spaces in filenames or filenames starting with hyphen: the filesystem should not be restricted by bash and other shell scripts continual confusion between data and instructions. We should fix/replace lazy programs, don't try to fix lazy programming by neutering the system. I think the correct action should be to design shell scripting languages that are robust about data versus instructions. The lack of robustness in shell scripting languages has been a burden since their inception. Restricting the filesystem won't stop problems with shell scripts making the same mistakes on other kinds of data.

overgard 13y ago
So one solution is to replace all the popular shells from the ground up, throwing away about 30 years of history because they're "not elegant". (Ignoring the fact that almost every attempt to do this in the past has been a resounding failure). The other solution is to agree not put overly weird shit in filenames. Which seems more likely to happen in our lifetime? Which would you rather code against? I'm baffled that people are so hell bent on what "should" be allowed that they're ignoring the fact that allowing any possible character introduces exceptional complexity while offering extremely marginal benefits. I mean, imagine trying to make a gui for a filesystem browser that allows for characters that flow both up and down along with left and right. And newlines all over the place. It'd be a mess. I mean.. why? Is filename expressivity really such a major deal?
- gilgoomesh 13y ago
  If you're looking at "30 years of history" then shells have endured "weird shit in filenames" for their entire existence and civilisation hasn't collapsed. Clearly, we don't need to change anything for them so I don't think filesystem are going to recant on their longstanding policies by making any changes to further accommodate scripting languages in our lifetimes. Meanwhile, new shells (or new versions of shells) come and go all the time. Could one of these implement a foolproof (or at least, foolresistant) way of escaping arguments? Maybe.
- toast0 13y ago
  > The other solution is to agree not put overly weird shit in filenames. If you don't put weird shit in your filenames, I won't put weird shit in mine. Let's not bother the kernel with our contract. :)
  - Flimm 13y ago
    That doesn't solve the security vulnerabilities caused by weird filenames.
- cnvogel 13y ago
  I think it shows that contemporary shell programming never was exposed to as much malicious misuse as common web-apps. I mean "program ★foo★" is basically equivalent to "SELECT ★ FROM table WHERE foo = '$bar'" when provided with arbitrary, user-provided input. And you see a lot of this in typical shell-scripts. And for output messing up our terminal, no one ever uses "<? print($input) ?>" to put things in HTML pages without escaping it. So, yes, if you are writing scripts that should be resilient to all malicious or accidentaly provided input, it's a lot of work, and you better have code for these common operations (input, output of user-provided filenames) in all applications that try to do useful things with filenames. And probably it means that you cannot use programs not providing a "--" parse-stopper or put the data on the end of a "ssh" command-line. And there will always be an application confused by some random other seemingly innocuous character that the kernel "still" allows in filenames. So the solution probably is not to limit the allowed characters, and also not to try to fix the useful and working parts of shells, but to augment them with the equivalents of html_escape() and mysql_prepare("...?...")/mysql_escape() where needed (to stay in the realm of SQL and web-apps).
- jfb 13y ago
  Who owns the filename? If the user owns the filename, than it is incumbent on the system to be able to handle arbitrary names. If the user doesn't own the name, then the system must not expose the name to the user. c.f. filename extensions. If a human script is read top-to-bottom, then the computer should handle that without complaint. This may not be possible in all cases immediately -- there's far too much Unix braindamage weighting the world down to throw it all away at once -- but we should all be trying to liberate humans from the dead hand of New Jersey.
vacri 13y ago
Ignoring the fact that non-terminal users regularly want spaces, hyphens and newlines in their filenames since they have visual purpose... I'll happily agree with the spaces, because people like filenames to look like short phrases. But I've never seen a human make filenames with a leading hyphen (plenty with hyphen within the word) nor with a newline. I could imagine that somewhere there is a person or two that likes to make hyphen-led filenames to try and sort things, but this is catering to the outliers. I can't imagine the use case for anyone to intentionally make a filename with a newline in it, and haven't seen such a thing, human or program-made. I think it's not common or valuable enough a thing to hamstring everyone else. Edit: solution in the traditional linux way: off by default, but you can always compile them back in if you love them that much :)
- jlgreco 13y ago
  I once saw humans making filenames like --splitwork --verbose -tdxh input.txt And by "once", I mean "yesterday". Now, is it reasonable to design a system that takes input flags through the names of input files? No. It is clever, yes, but it's not reasonable. The fact of the matter is however that these usecases, for better or worse, do exist. We should not change filesystems to break these systems (broken as they are already) just because bourne-related shells aren't fond of them.
einhverfr 13y ago
I have done a lot with users of all sorts. I see a lot of spaces and hyphens inside files, but I have never seen either a leading hyphen or an embedded newline. Are there cases where these would be desired in GUI environments? Why and what is the use case?
- gilgoomesh 13y ago
  I've regularly seen: -- Myfile With Emphasis --.txt Basically, users will try to use any punctuation they see on their keyboard to adorn their filenames.
  - deleted 13y ago
    [deleted]
  - einhverfr 13y ago
    Interesting. So maybe this should be configurable.

overgard 13y ago

So one solution is to replace all the popular shells from the ground up, throwing away about 30 years of history because they're "not elegant". (Ignoring the fact that almost every attempt to do this in the past has been a resounding failure). The other solution is to agree not put overly weird shit in filenames. Which seems more likely to happen in our lifetime? Which would you rather code against? I'm baffled that people are so hell bent on what "should" be allowed that they're ignoring the fact that allowing any possible character introduces exceptional complexity while offering extremely marginal benefits. I mean, imagine trying to make a gui for a filesystem browser that allows for characters that flow both up and down along with left and right. And newlines all over the place. It'd be a mess. I mean.. why? Is filename expressivity really such a major deal?

gilgoomesh 13y ago
If you're looking at "30 years of history" then shells have endured "weird shit in filenames" for their entire existence and civilisation hasn't collapsed. Clearly, we don't need to change anything for them so I don't think filesystem are going to recant on their longstanding policies by making any changes to further accommodate scripting languages in our lifetimes. Meanwhile, new shells (or new versions of shells) come and go all the time. Could one of these implement a foolproof (or at least, foolresistant) way of escaping arguments? Maybe.
toast0 13y ago
> The other solution is to agree not put overly weird shit in filenames. If you don't put weird shit in your filenames, I won't put weird shit in mine. Let's not bother the kernel with our contract. :)
- Flimm 13y ago
  That doesn't solve the security vulnerabilities caused by weird filenames.
cnvogel 13y ago
I think it shows that contemporary shell programming never was exposed to as much malicious misuse as common web-apps. I mean "program ★foo★" is basically equivalent to "SELECT ★ FROM table WHERE foo = '$bar'" when provided with arbitrary, user-provided input. And you see a lot of this in typical shell-scripts. And for output messing up our terminal, no one ever uses "<? print($input) ?>" to put things in HTML pages without escaping it. So, yes, if you are writing scripts that should be resilient to all malicious or accidentaly provided input, it's a lot of work, and you better have code for these common operations (input, output of user-provided filenames) in all applications that try to do useful things with filenames. And probably it means that you cannot use programs not providing a "--" parse-stopper or put the data on the end of a "ssh" command-line. And there will always be an application confused by some random other seemingly innocuous character that the kernel "still" allows in filenames. So the solution probably is not to limit the allowed characters, and also not to try to fix the useful and working parts of shells, but to augment them with the equivalents of html_escape() and mysql_prepare("...?...")/mysql_escape() where needed (to stay in the realm of SQL and web-apps).
jfb 13y ago
Who owns the filename? If the user owns the filename, than it is incumbent on the system to be able to handle arbitrary names. If the user doesn't own the name, then the system must not expose the name to the user. c.f. filename extensions. If a human script is read top-to-bottom, then the computer should handle that without complaint. This may not be possible in all cases immediately -- there's far too much Unix braindamage weighting the world down to throw it all away at once -- but we should all be trying to liberate humans from the dead hand of New Jersey.

vacri 13y ago

Ignoring the fact that non-terminal users regularly want spaces, hyphens and newlines in their filenames since they have visual purpose... I'll happily agree with the spaces, because people like filenames to look like short phrases. But I've never seen a human make filenames with a leading hyphen (plenty with hyphen within the word) nor with a newline. I could imagine that somewhere there is a person or two that likes to make hyphen-led filenames to try and sort things, but this is catering to the outliers. I can't imagine the use case for anyone to intentionally make a filename with a newline in it, and haven't seen such a thing, human or program-made. I think it's not common or valuable enough a thing to hamstring everyone else. Edit: solution in the traditional linux way: off by default, but you can always compile them back in if you love them that much :)

jlgreco 13y ago
I once saw humans making filenames like --splitwork --verbose -tdxh input.txt And by "once", I mean "yesterday". Now, is it reasonable to design a system that takes input flags through the names of input files? No. It is clever, yes, but it's not reasonable. The fact of the matter is however that these usecases, for better or worse, do exist. We should not change filesystems to break these systems (broken as they are already) just because bourne-related shells aren't fond of them.

einhverfr 13y ago

I have done a lot with users of all sorts. I see a lot of spaces and hyphens inside files, but I have never seen either a leading hyphen or an embedded newline. Are there cases where these would be desired in GUI environments? Why and what is the use case?

gilgoomesh 13y ago
I've regularly seen: -- Myfile With Emphasis --.txt Basically, users will try to use any punctuation they see on their keyboard to adorn their filenames.
- deleted 13y ago
  [deleted]
- einhverfr 13y ago
  Interesting. So maybe this should be configurable.