32 ms·

Dot Dot Considered Harmful

345 points by rohan1024 8y ago

devit 8y ago
That makes it impossible to properly implement POSIX on top of it and seems like a poor design. The correct design is to have both a "root" and a "current object" associated with every "file descriptor object" and allow ".." to work up to the "root" (and thoughtfully handle cases where the "current object" is moved outside the "root"). You can't do it with paths because that doesn't track directories being renamed, that would cause the descriptor to suddenly become inaccessible in the middle of an operation.
- empath75 8y ago
  > That makes it impossible to properly implement POSIX on top of it so don't implement posix. I don't know why people get so hung up on new oses being POSIX compliant.
  - vertex-four 8y ago
    Mostly because being POSIX compliant makes it far easier to port a lot of command-line and server-side software quickly. Most new operating system projects don’t have the manpower to rewrite the world - I think the last major one was probably Android.
    - empath75 8y ago
      Well. They’re google. Also the redox team knocked out rust versions of most of the standard utilities in a few months.
    - vlovich123 8y ago
      You'd be surprised how generic most code is or how easy it is to port if it's POSIX-like. Out of the major consumer OSes, only OSX is POSIX certified (in 10.5 which is only 11 years ago). Hell, the majority of Android code is written in Java which has no POSIX roots at all so avoiding POSIX isn't a death knell you seem to imply. iOS, while it supports POSIX, most of the developers use iOS-specific APIs unless they're in shared code with another OS & not using something like Boost. Have you actually tried using POSIX networking APIs? They're god awful. POSIX threading primitives are terrible. etc etc. POSIX is sometimes useful for writing cross-platform C/C++ code. It's extremely limiting though, out-of-date, & doesn't actually offer the write-once run anywhere you'd like with POSIX. You can write POSIX code that will fail to build, fail at runtime, or even behave incorrectly when you run it on another "POSIX" system (at least as far as Windows/Linux/OSX/Android are concerned). Certainly a far cry from how a standard is supposed to behave. POSIX also, for the most part, targets the lowest common denominator of platform features which means the POSIX API isn't as rich as makes sense for the majority of applications, doesn't have the same performance/security, and/or isn't as easy-to-use. POSIX also leaves many subtle decisions to implementations' discretion which means that even if everything works in the happy path, it'll break in subtle corner cases. For example, PATH_MAX is defined as 256, _XOPEN_PATH_MAX is 1024, but Linux & OSX both have unbounded limits so a POSIX program can easily fail to be able to open all files on an OS; making this a build-time constant was the stupidest decision in the world & endemic of how POSIX is designed. Most modern language runtimes these days (Rust, Go, Java, Swift) comes with a far richer, less bug-prone & more feature-full set of features in the standard library out-the-gate on all platforms (so you just need to port the 1 standard library) & most libraries build on that standard library so you usually get them for free. Most platform vendors also provide custom APIs to interact more richly with their specific features for performance, battery, usability, security, etc. To take full advantage of a platform, which you're pushed to usually by market forces, POSIX doesn't help you. * EDIT: Also, POSIX is gigantic. The majority of useful existing tools probably use 20% of the entire standard. Porting that smaller API surface isn't challenging.
      - pjmlp 8y ago
        Basically POSIX is the C standard library they didn't want to make part of ISO C. As you well point out hardly significant when using other programming languages, even C++ standard library improvements are making it less relevant for C++ devs.
    - pjmlp 8y ago
      That is a recipe for a mono-culture stuck in the past.
      - vertex-four 8y ago
        Of course it is. It’s not a good thing - it’s essentially killed any possibility of a capability-oriented desktop OS for a number of years, for a start. But it’s the state of things - if someone can’t run at least most of their software on a new system, it’s a massive barrier to adoption. The reason Android and iOS got away with it was that nobody was really running important software on their phones in the first place, so it was a brand new capability that didn’t need to be compatible with anything.
        pjmlp 8y ago
        Symbian and J2ME phones were quite popular in Europe and Asia, lacking POSIX compatibility wasn't an issue. In fact, the PIPS library for Symbian was only introduced after the decision to buy Qt. You also forgot about ChromeOS, which I bet hardly anyone on the US school system cares about POSIX. Or the webOS running on LG televisions nowadays. FreeRTOS, mbed and IncludeOS aren't POSIX as well. Regarding Fuchsia, there is already support for ISO C, ISO C++, Rust, Go, Dart/Flutter, with Java/Android on the horizon. So I doubt Fuchsia will miss POSIX that much.
        vertex-four 8y ago
        My experience of J2ME phones (as someone living in India pre-iPhone) was that the Java functionality went unused, except as far as it was used to implement vendor-written apps. ChromeOS benefits from the fact that it is just a web browser - if you want just a web browser rather than a generic operating system, Fuschia is of course unnecessary. Nobody targets webOS, and I suspect people’s choice of TV has more to do with how it looks on the in-store display or how cheap it is than what apps you can run on it. FreeRTOS, mbed and IncludeOS are not generic operating systems in the first place. There are essentially no “pure” ISO C or ISO C++ programs - everybody uses system-specific libraries at some level. Basically, Fuschia suffers from the fact that a large amount of userland will need to be rewritten for it, depending on what market segment it targets. (“Just” being an alternate Android runtime for phones, as an example, wouldn’t require this.) This is an obstacle - not an insurmountable one, but being non-POSIX is an obstacle for any OS, that the people behind it need a strategy to tackle. Whether or not people will code apps for Fuschia will depend on how excited developers get about it at first, and then how many people use it.
- loeg 8y ago
  As I understand it, Fuschia doesn't aim to comply with POSIX. Your suggestion matches the behavior of "O_BENEATH" in Linux and FreeBSD, or e.g., Capsicum directory descriptors with the CAP_LOOKUP capability. https://lwn.net/Articles/604015/ https://lwn.net/Articles/604015/ https://lwn.net/Articles/482858/ https://lwn.net/Articles/482858/
- kiriakasis 8y ago
  you can define a root filesystem and mount everything you want to access there. the restriction is that your program (or a compatibility layer) should always keep an handle on the root filesystem.
tyingq 8y ago
Rob Pike apparently regrets all the "." file naming. https://plus.google.com/u/0/+RobPikeTheHuman/posts/R58WgWwN9jp https://plus.google.com/u/0/+RobPikeTheHuman/posts/R58WgWwN9...
- nickysielicki 8y ago
  This comment made me realize that there are a lot of really good posts on Google Plus by people like Rob Pike, Greg Kroah-Hartman, Linus Torvalds and others. For whatever reason, systems programmers liked Google Plus. Anyway, I hope someone is archiving them somewhere, because there's a lot of knowledge there that will otherwise be lost in a few months.
  - z3t4 8y ago
    I always wondered about the "dot files" convention. That's a really good story! If I knew there where posts like this one I would join in a heartbeat. Where can I find communities like this today !? I always seem to be too late ...
  - donatj 8y ago
    My immediate thought when they announced they were shutting down was "What about Linus's posts?!"
  - dcbadacd 8y ago
    Google Plus played less with people's emotions, that's why it sucked as a social media but was quite okay to follow content makers and to manage/participate in groups.
- deleted 8y ago
  [deleted]
- deleted 8y ago
  [deleted]
- charonn0 8y ago
  Isn't he talking about something different, though? Specifically the convention that file names starting with "." are hidden?
deleted 8y ago
[deleted]
IridiumFX 8y ago
It’s a thing born in google. It shows. The reasoning behind problems and solutions is so abstract that makes for a beautiful paper or a nightmarish reality. In the ‘80s there was a little known machine, called the Commodore Amiga. Paths were addressed by a volume:folder/file schema. Apps had logical volumes too (progdir:) and the os injected others (env: temp: fonts: ..) guess what? Just use that schema and control what an app can access or not. If you don’t give me a volume for a disk, I cannot make my way to it if it’s not collated into a mountpoint thing
- wmf 8y ago
  This particular idea wasn't born in Google; it traces back through previous secure OSes like EROS and KeyKOS maybe with a dash of Plan 9 thrown in.
  - ratmice 8y ago
    NLTSS as well
- mixmastamyk 8y ago
  I wouldn't say the Amiga was little known, though it did lose the war over the desktop. I like that syntax by the way, kind of like drive letters but much more descriptive and not limited to A-Z. Netware also used it for full path specs.
- ycmbntrthrwaway 8y ago
  What you describe is called "namespaces". Plan 9 and Linux have them already.
  - yiyus 8y ago
    In Plan 9, special measures were taken to get dot dot right, not so much because of private namespaces but because of bind. http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames
koblas 8y ago
It's interesting to think that the goal of '..' was for navigation around the command line shell. Which is a tool that 95% of people developing software never really use, or use as a necessity (type one command and leave). It's really an interesting thought piece to consider that the idea of the '..' directory is as legacy as the headphone jack.
- settler4 8y ago
  Maybe if your experience using a shell is only Window's CMD. I live in the shell. My Linux machines are configured to automatically open a terminal upon login and the first thing I do when I get a new Windows installation is getting bash working on it even if it's not for development.
- mixmastamyk 8y ago
  Developers who don't use a command-line shell, at least some of the time, aren't much of a developer.
  - rjplatte 8y ago
    I think he means that people who use IDEs don't use shell file navigation as often.
    - koblas 8y ago
      Yes, that is exactly what I mean. The majority of people developing software are doing it inside of Visual Studio Code or similar tools, where their only exposure to the command line is when they do "create-app"
      - deleted 8y ago
        [deleted]
      - Meph504 8y ago
        You keep using the unqualified term "majority" with no way to actually back that up. This would probably be less critized if you stated it as your opinion (which it is) and not as a definitive statement.
dooglius 8y ago
This should probably point at the upstream page by the fuchsia project (https://fuchsia.googlesource.com/docs/+/HEAD/the-book/dotdot.md https://fuchsia.googlesource.com/docs/+/HEAD/the-book/dotdot...) rather than a mirror source repo
- rjplatte 8y ago
  > Upstream Ironic
  - malkia 8y ago
    Rusty... trying to picturesque it...
- sctb 8y ago
  Sure thing, we've updated the link from https://github.com/fuchsia-mirror/docs/blob/master/the-book/dotdot.md https://github.com/fuchsia-mirror/docs/blob/master/the-book/....
  - rohan1024 8y ago
    I was wondering who updated it. Thanks.
  - romeisendcoming 8y ago
    Lost me at 'story' 5 times in a paragraph. Get rid of the story and get on with the real relation.
starclaps 8y ago
That’s why tools like gosec complain when user input is used to access files - specifically because ‚..‘ allows one to „escape“. Gosec: https://github.com/securego/gosec https://github.com/securego/gosec
akerl_ 8y ago
Given that they’re still resolving “..”, they’re just resolving it in code client-side before requesting the content, they really haven’t gotten rid of “..”: they’ve implemented a path sanitization library, similarly to how many other frameworks (notably those designed to serve files over the web) behave.
- ivanbakel 8y ago
  That is getting rid of ".." - the client-side version is fundamentally different in behaviour, since it can't be resolved without prior knowledge of the filesystem. ".." isn't being sanitised from user paths, it's being translated, because there is no equivalent concept in the file server.
- itp 8y ago
  No, it's deeper than that. They are describing the capabilities of a process handed a file descriptor, which means it does not have a path at all. There's no way to resolve '..' relative to no path. Compare that to a POSIX system where a directory has an actual child which is a reference to the directory's parent whose name is always '..'. The talk about resolving '..' is merely a demonstration that the behavior of "cd .." can be supported/emulated in a context where you have both an open directory and corresponding path, without requiring that '..' literally exist.
  - loeg 8y ago
    To add to that, as the document writes in the very first section, it can sort of be thought of as a chroot. You can't .. out of a chroot; there is no "..". (Yeah, it's not exactly a chroot; that's well covered in the document.)
delinka 8y ago
>If a handle is provided to a directory, it should imply access to resources within that directory (and additionally, their subdirectories). Maybe within Fuschia, but not in my UNIXy systems. All users on a system need r-x on /Users, but that doesn't give them r-x on every user's home directory located as a subdirectoy of /Users - a handle acquired by bob to /Users does not imply access to /Users/delinka. Further, if your process is treating '..' as a subdirectory, you're doing it wrong. Paths must be normalized (e.g. ~ expanded, . and .. resolved) before requesting a handle via absolute path. Lastly, this document reads as if knowing a full path grants access to that path and its subdirectories. If that's the case ... oy.
- wahern 8y ago
  > Further, if your process is treating '..' as a subdirectory, you're doing it wrong. Paths must be normalized (e.g. ~ expanded, . and .. resolved) before requesting a handle via absolute path. ".." is required for atomic traversal (similar to symlinks), which is important in some situations, such as making sure a file tree you've descended isn't removed out from under you in a way that breaks your state. (The directory itself can be moved while you hold the handle, but the important thing is that being able to rely on ".." permits certain algorithms that are safer and more consistent.) Canonicalizing paths introduces race conditions between canonicalization and actual access, which is why this is performed in the kernel. Canonicalizing paths makes sense if you're accepting paths from untrusted sources and you cannot make use of POSIX openat() + extensions like O_BENEATH. HTTP GET requests are supposed to be idempotent, anyhow, so there shouldn't exist any sort of race condition as a conceptual matter. But in regular software, it's better to just pass paths as-is. The shell performs "~" expansion, but it doesn't resolve "." or "..". And the shell performs other expansions, like file globbing, for which there's rarely any reason to implement in regular software. Supporting "~" expansion but not file globbing is inconsistent; if you're not the shell, don't implement shell-like features as it just creates confusion and unnecessary complexity. Calling ".." a holdout from POSIX is misleading. The semantics exist for legitimate and even security-relevant reasons, albeit not necessarily reasons that an embedded smartphone OS may care about. What POSIX lacks is O_BENEATH (from Capsicum) or a similar flag for tagging a directory descriptor to prevent relative paths or otherwise ascending the tree. Capsicum extensions make POSIX perfectly capable of implementing strict capability semantics, and they do so by extending semantics in a manner consistent with POSIX. POSIX isn't inherently broken in this regard, it's just perhaps too feature rich for some use cases and not feature rich enough for others. There's no end of complaints about POSIX--either it's too complex or too simple. The fact is, no single specification will ever please anybody, so griping about how POSIX is broken is not only pointless but belies a failure to appreciate the underling issues. The alternative to POSIX is basically nothing. Standardizing on Linux is, at best, a lateral move. Adding optional components to specifications has proven the worse of all worlds, and like most other standards POSIX has been slowly removing some optional components altogether while making others mandatory.
monocasa 8y ago
> this is an essential idea encompassing microkernels, and other “capability-based” systems Tiny nit, but microkernels don't imply a capability based security model. For instance Mach, QNX, Redox, etc. aren't capability based. It's a very good idea for your microkernel to be capability based because it cuts a lot of validation out of the critical path for IPC, but it's by no means a requirement.
- comex 8y ago
  Mach is capability based. (Mach ports are capabilities.)
  - monocasa 8y ago
    You're totally right of course. I was falsely under the impression that the Mach port table only had a single global namespace.
saagarjha 8y ago
> at the time of writing, symbolic links do not exist on Fuchsia Is this for technical reasons, or similar philosophical ones because symbolic links also allow for escaping from “jails”?
- loeg 8y ago
  I think "at the time of writing" implies they're still under consideration, rather than outright rejected due to philosophical differences.
- jepler 8y ago
  Their whole "scheme", where ".." can be dealt with as a canonicalizing step BEFORE submitting the path to the OS for actual opening, seems to require refinement when symlinks are involved. Take a perfectly spherical unix: $ mkdir /tmp/hn $ cd /tmp/hn $ ln -s . foo; mkdir bar; touch baz $ ls -l bar/../baz foo/baz foo/../baz ls: cannot access 'foo/../baz': No such file or directory -rw-r--r-- 1 jepler jepler 0 Nov 28 18:21 bar/../baz -rw-r--r-- 1 jepler jepler 0 Nov 28 18:21 foo/baz Fuscsia-with-symlinks would have some extra hoops to jump through to make rewriting act like POSIX in the case that /tmp/hn is not escaped, which surely you'd want if you went ahead and introduced symlinks.
  - Dylan16807 8y ago
    > which surely you'd want Nope. I don't want that at all. There's a reason that by default cd goes out of its way to make .. ignore the parent of a symlink.
- cyphar 8y ago
  It should be noted that this is not without precedent. Plan9 doesn't have symlinks for example, everything is done using binds. You would be surprised how many complications come up when you add symlinks to path resolution code.
yiyus 8y ago
See also: http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames
- theoh 8y ago
  This paper, depending on how you look at it, either justifies the existence of dot dot, or makes it seem like a wart. IMO dot dot is too well-established and essentially sensible (in a minimal UNIX fashion) for it to be removed on a whim. Consider the approach taken by https://man.openbsd.org/unveil.2 https://man.openbsd.org/unveil.2: if you try to access a file you aren't explicitly allowed to, you get EACCESS or ENOENT. I'm not an expert, and the possibility of errors obviously requires applications to tread more carefully than they might have previously, but it seems like a clean solution.
  - kiriakasis 8y ago
    > removed on a whim They changed the whole concept of having one single root. As I understand the plan is to use many fragmented and independent filesystem handles that can optionally mounted together. So the restriction is more about non being able to access a folder if you don't have access to an appropriate handle. Paths like folder1/folder2/../folder2/file are still perfectly fine.
    - theoh 8y ago
      To clarify: that's the Plan 9 situation. I'm talking about the merits of the Fuchsia situation (".. is no longer available"). References to a forbidden parent directory from a chroot can just return ENOENT, because it doesn't exist in that universe. I may not be understanding this fully, but to condemn ".." based on some accusation that it's incompatible with chroot semantics (or "a holdout from POSIX") seems tendentious.
    - AndyMcConachie 8y ago
      Didn't MS-DOS already do this with letters? (e.g., a:, b:, c:) Maybe I don't really understand Fuchsia's approach. But I do really like OpenBSD's approach in unveil.
mattigames 8y ago
For irony sake one may link this file as https://github.com/fuchsia-mirror/docs/blob/master/the-book/../../master/the-book/dotdot.md
- Someone1234 8y ago
  Not as ironic as you'd think considering the article specifically talks about support for .. in userland tooling and uses CWD as an example.
- kiriakasis 8y ago
  well actually the article complains about stuff like https://github.com/fuchsia-mirror/../internal-page/secret-report/ https://github.com/fuchsia-mirror/../internal-page/secret-re... which quite more serious
- Retr0spectrum 8y ago
  Nope, that actually 404's. It may appear to work, because most browsers will apply some equivalent to the `__fdio_cleanpath()` function mentioned in the article, resolving the traversal locally. You can verify this as follows: curl -I --path-as-is 'https://github.com/fuchsia-mirror/docs/blob/master/the-book/../../master/the-book/dotdot.md'
  - mattigames 8y ago
    if most browsers apply it... doesn't that means it works? Maybe not in the resolved-by-the-server-returning-a-304 sense but in at least some sense.
csours 8y ago
I'm very surprised that I agree after reading. As I read initially, I was thinking "what about..." but it's really about separation of concerns.
dsl 8y ago
It is ironic this is coming out of a company with an internal site dedicated to depreciating "Considered Harmful" because it is the epitome of grandstanding. Here is some further reading on why we should downvote these types of things into oblivion: https://meyerweb.com/eric/comment/chech.html https://meyerweb.com/eric/comment/chech.html
- gowld 8y ago
  That's a silly article. The essay says that we should disregard essays if they have a title like the essay's own, so... pradox. When "considered harmful" is considered flaimbait rhetoric, perhaps the problem is with the readers who refuse to engage with any mildly-worded criticism.
  - donatj 8y ago
    It’s surely in how the reader interprets it, but I always read “considered harmful” as quite mild but to the point. Something like “isn’t great”. A refreshing opposite of clickbait as the actual proposition is in the title.
vermilingua 8y ago
This may be off topic, and is hard to ask without doomsaying: but can we trust Fuchsia? Google is a machine for turning user's personal data into their dollars, and has been getting more and more crafty at achieving this goal. I want to believe that this is a good-natured effort at improving the state of modern operating systems; but I feel like I've been burned by my trust in Google too many times.
- landryraccoon 8y ago
  > I feel like I've been burned by my trust in Google too many times I'm genuinely curious about this. In what way do you feel burned? I think I get what you're saying. I use gmail and Google uses an algorithm on gmail to decide what ads to show me when I do a search. The ads make money for Google and I don't get any of that money. All I get is free search results. I think what you're saying is that because Google is using your information to show you ads and make money that somehow you feel cheated? But I don't feel particularly burned by this arrangement. If Google gave an option to start charging me money to avoid ads if I'm honest with myself I'm pretty sure I'll choose to continue seeing ads instead. I guess I'm curious about exactly how you feel burned.
  - eridius 8y ago
    Probably the most obvious example is when Google almost destroyed the entire RSS ecosystem with Google Reader.
    - SquareWheel 8y ago
      I'd argue that competitors flourished when Google Reader went down. I still use RSS today. Besides, it's certainly not their responsibility to maintain a service indefinitely.
      - eridius 8y ago
        It was a close call. For a number of months there it looked like RSS was going to be well and truly dead. The fact that my comment is being downvoted tells me that people here have a shockingly poor memory for this sort of thing. And heck, it's not even the death of Google Reader that was bad for RSS. The life of Google Reader was bad for RSS too. It was a "good enough" product released for free, as in good enough that it wasn't worth trying to compete with a free product so nobody did, but it wasn't actively maintained and just served to cause the entire RSS ecosystem to stagnate for years.
mjfl 8y ago
"considered harmful" considered harmful The origin of the meme was an editor's labeling of Dijkstra's commentary. Titling your own article this makes it seem like you wrote the article, forgot you wrote it, found it again, and are presenting it to others as an interesting perspective.
vinceguidry 8y ago
Coming from Ruby, NodeJS annoys the crap out of me with its lack of decent autoload functionality. Seeing relative paths in modules really annoys me. JS world can't figure out what should be global and what should be local it seems.
- ricardobeat 8y ago
  What do you mean? Global packages are global, local packages are local.
- loeg 8y ago
  Wrong thread?
hnbroseph 8y ago
i really wish the "considered harmful" meme/trope/whatever would take a nap.
DSingularity 8y ago
Any good papers to read on fuchsia?
- rohan1024 8y ago
  This is ironic cd.. Here you go https://fuchsia.googlesource.com/docs/+/HEAD/the-book/ https://fuchsia.googlesource.com/docs/+/HEAD/the-book/
rustcharm 8y ago
In "classic" Unix, isn't ".." really just another file name, created by default as a hard link to the parent directory?
infinity0 8y ago
> What about shell traversal? > [..] For example, if the CWD is “/foo/bar”, and a user calls “cd ..”, then the underlying call may be transformed into “chdir /foo/bar/..”, which can be canonicalized to “/foo”. So fuchsia also won't have CWD, then? Because if it has CWD, then the process can always chdir / rendering this lack-of-.. exercise pointless.
- IshKebab 8y ago
  I don't see what made you think it won't have a CWD.
- rictic 8y ago
  fuchsia is not posix, so they can (and I suspect have) change the semantics of chdir to be more restrictive.
- bun_at_work 8y ago
  I think the idea in Fuchsia is that directories can be passed as resources to children processes, which can then move freely around in that directory structure. If the parent passes a child "/foo/bar/", the child sees "bar/" and it's contents, which might be "bar/baz/bingo/". The child can then move to "/bar/baz/bingo/" do work there, and then recurse up the hierarchy. It simply cannot ".." out of "foo/". Hope that makes sense.
IgorPartola 8y ago
So can a process create a symlink a la ln -s ../../.. root And gain access? Or are symlinks now fucked with too? This seems like a poorly thought out plan, and unnecessary at that.
- otabdeveloper1 8y ago
  > This seems like a poorly thought out plan, and unnecessary at that. Welcome to Google Future (c). Enjoy your stay.

345 points by rohan1024 8y ago

That makes it impossible to properly implement POSIX on top of it and seems like a poor design. The correct design is to have both a "root" and a "current object" associated with every "file descriptor object" and allow ".." to work up to the "root" (and thoughtfully handle cases where the "current object" is moved outside the "root"). You can't do it with paths because that doesn't track directories being renamed, that would cause the descriptor to suddenly become inaccessible in the middle of an operation.

empath75 8y ago
> That makes it impossible to properly implement POSIX on top of it so don't implement posix. I don't know why people get so hung up on new oses being POSIX compliant.
- vertex-four 8y ago
  Mostly because being POSIX compliant makes it far easier to port a lot of command-line and server-side software quickly. Most new operating system projects don’t have the manpower to rewrite the world - I think the last major one was probably Android.
  - empath75 8y ago
    Well. They’re google. Also the redox team knocked out rust versions of most of the standard utilities in a few months.
  - vlovich123 8y ago
    You'd be surprised how generic most code is or how easy it is to port if it's POSIX-like. Out of the major consumer OSes, only OSX is POSIX certified (in 10.5 which is only 11 years ago). Hell, the majority of Android code is written in Java which has no POSIX roots at all so avoiding POSIX isn't a death knell you seem to imply. iOS, while it supports POSIX, most of the developers use iOS-specific APIs unless they're in shared code with another OS & not using something like Boost. Have you actually tried using POSIX networking APIs? They're god awful. POSIX threading primitives are terrible. etc etc. POSIX is sometimes useful for writing cross-platform C/C++ code. It's extremely limiting though, out-of-date, & doesn't actually offer the write-once run anywhere you'd like with POSIX. You can write POSIX code that will fail to build, fail at runtime, or even behave incorrectly when you run it on another "POSIX" system (at least as far as Windows/Linux/OSX/Android are concerned). Certainly a far cry from how a standard is supposed to behave. POSIX also, for the most part, targets the lowest common denominator of platform features which means the POSIX API isn't as rich as makes sense for the majority of applications, doesn't have the same performance/security, and/or isn't as easy-to-use. POSIX also leaves many subtle decisions to implementations' discretion which means that even if everything works in the happy path, it'll break in subtle corner cases. For example, PATH_MAX is defined as 256, _XOPEN_PATH_MAX is 1024, but Linux & OSX both have unbounded limits so a POSIX program can easily fail to be able to open all files on an OS; making this a build-time constant was the stupidest decision in the world & endemic of how POSIX is designed. Most modern language runtimes these days (Rust, Go, Java, Swift) comes with a far richer, less bug-prone & more feature-full set of features in the standard library out-the-gate on all platforms (so you just need to port the 1 standard library) & most libraries build on that standard library so you usually get them for free. Most platform vendors also provide custom APIs to interact more richly with their specific features for performance, battery, usability, security, etc. To take full advantage of a platform, which you're pushed to usually by market forces, POSIX doesn't help you. * EDIT: Also, POSIX is gigantic. The majority of useful existing tools probably use 20% of the entire standard. Porting that smaller API surface isn't challenging.
    - pjmlp 8y ago
      Basically POSIX is the C standard library they didn't want to make part of ISO C. As you well point out hardly significant when using other programming languages, even C++ standard library improvements are making it less relevant for C++ devs.
  - pjmlp 8y ago
    That is a recipe for a mono-culture stuck in the past.
    - vertex-four 8y ago
      Of course it is. It’s not a good thing - it’s essentially killed any possibility of a capability-oriented desktop OS for a number of years, for a start. But it’s the state of things - if someone can’t run at least most of their software on a new system, it’s a massive barrier to adoption. The reason Android and iOS got away with it was that nobody was really running important software on their phones in the first place, so it was a brand new capability that didn’t need to be compatible with anything.
      - pjmlp 8y ago
        Symbian and J2ME phones were quite popular in Europe and Asia, lacking POSIX compatibility wasn't an issue. In fact, the PIPS library for Symbian was only introduced after the decision to buy Qt. You also forgot about ChromeOS, which I bet hardly anyone on the US school system cares about POSIX. Or the webOS running on LG televisions nowadays. FreeRTOS, mbed and IncludeOS aren't POSIX as well. Regarding Fuchsia, there is already support for ISO C, ISO C++, Rust, Go, Dart/Flutter, with Java/Android on the horizon. So I doubt Fuchsia will miss POSIX that much.
        vertex-four 8y ago
        My experience of J2ME phones (as someone living in India pre-iPhone) was that the Java functionality went unused, except as far as it was used to implement vendor-written apps. ChromeOS benefits from the fact that it is just a web browser - if you want just a web browser rather than a generic operating system, Fuschia is of course unnecessary. Nobody targets webOS, and I suspect people’s choice of TV has more to do with how it looks on the in-store display or how cheap it is than what apps you can run on it. FreeRTOS, mbed and IncludeOS are not generic operating systems in the first place. There are essentially no “pure” ISO C or ISO C++ programs - everybody uses system-specific libraries at some level. Basically, Fuschia suffers from the fact that a large amount of userland will need to be rewritten for it, depending on what market segment it targets. (“Just” being an alternate Android runtime for phones, as an example, wouldn’t require this.) This is an obstacle - not an insurmountable one, but being non-POSIX is an obstacle for any OS, that the people behind it need a strategy to tackle. Whether or not people will code apps for Fuschia will depend on how excited developers get about it at first, and then how many people use it.
loeg 8y ago
As I understand it, Fuschia doesn't aim to comply with POSIX. Your suggestion matches the behavior of "O_BENEATH" in Linux and FreeBSD, or e.g., Capsicum directory descriptors with the CAP_LOOKUP capability. https://lwn.net/Articles/604015/ https://lwn.net/Articles/604015/ https://lwn.net/Articles/482858/ https://lwn.net/Articles/482858/
kiriakasis 8y ago
you can define a root filesystem and mount everything you want to access there. the restriction is that your program (or a compatibility layer) should always keep an handle on the root filesystem.

tyingq 8y ago

Rob Pike apparently regrets all the "." file naming. https://plus.google.com/u/0/+RobPikeTheHuman/posts/R58WgWwN9jp https://plus.google.com/u/0/+RobPikeTheHuman/posts/R58WgWwN9...

nickysielicki 8y ago
This comment made me realize that there are a lot of really good posts on Google Plus by people like Rob Pike, Greg Kroah-Hartman, Linus Torvalds and others. For whatever reason, systems programmers liked Google Plus. Anyway, I hope someone is archiving them somewhere, because there's a lot of knowledge there that will otherwise be lost in a few months.
- z3t4 8y ago
  I always wondered about the "dot files" convention. That's a really good story! If I knew there where posts like this one I would join in a heartbeat. Where can I find communities like this today !? I always seem to be too late ...
- donatj 8y ago
  My immediate thought when they announced they were shutting down was "What about Linus's posts?!"
- dcbadacd 8y ago
  Google Plus played less with people's emotions, that's why it sucked as a social media but was quite okay to follow content makers and to manage/participate in groups.
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
charonn0 8y ago
Isn't he talking about something different, though? Specifically the convention that file names starting with "." are hidden?

deleted 8y ago

[deleted]

IridiumFX 8y ago

It’s a thing born in google. It shows. The reasoning behind problems and solutions is so abstract that makes for a beautiful paper or a nightmarish reality. In the ‘80s there was a little known machine, called the Commodore Amiga. Paths were addressed by a volume:folder/file schema. Apps had logical volumes too (progdir:) and the os injected others (env: temp: fonts: ..) guess what? Just use that schema and control what an app can access or not. If you don’t give me a volume for a disk, I cannot make my way to it if it’s not collated into a mountpoint thing

wmf 8y ago
This particular idea wasn't born in Google; it traces back through previous secure OSes like EROS and KeyKOS maybe with a dash of Plan 9 thrown in.
- ratmice 8y ago
  NLTSS as well
mixmastamyk 8y ago
I wouldn't say the Amiga was little known, though it did lose the war over the desktop. I like that syntax by the way, kind of like drive letters but much more descriptive and not limited to A-Z. Netware also used it for full path specs.
ycmbntrthrwaway 8y ago
What you describe is called "namespaces". Plan 9 and Linux have them already.
- yiyus 8y ago
  In Plan 9, special measures were taken to get dot dot right, not so much because of private namespaces but because of bind. http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames

koblas 8y ago

It's interesting to think that the goal of '..' was for navigation around the command line shell. Which is a tool that 95% of people developing software never really use, or use as a necessity (type one command and leave). It's really an interesting thought piece to consider that the idea of the '..' directory is as legacy as the headphone jack.

settler4 8y ago
Maybe if your experience using a shell is only Window's CMD. I live in the shell. My Linux machines are configured to automatically open a terminal upon login and the first thing I do when I get a new Windows installation is getting bash working on it even if it's not for development.
mixmastamyk 8y ago
Developers who don't use a command-line shell, at least some of the time, aren't much of a developer.
- rjplatte 8y ago
  I think he means that people who use IDEs don't use shell file navigation as often.
  - koblas 8y ago
    Yes, that is exactly what I mean. The majority of people developing software are doing it inside of Visual Studio Code or similar tools, where their only exposure to the command line is when they do "create-app"
    - deleted 8y ago
      [deleted]
    - Meph504 8y ago
      You keep using the unqualified term "majority" with no way to actually back that up. This would probably be less critized if you stated it as your opinion (which it is) and not as a definitive statement.

dooglius 8y ago

This should probably point at the upstream page by the fuchsia project (https://fuchsia.googlesource.com/docs/+/HEAD/the-book/dotdot.md https://fuchsia.googlesource.com/docs/+/HEAD/the-book/dotdot...) rather than a mirror source repo

rjplatte 8y ago
> Upstream Ironic
- malkia 8y ago
  Rusty... trying to picturesque it...
sctb 8y ago
Sure thing, we've updated the link from https://github.com/fuchsia-mirror/docs/blob/master/the-book/dotdot.md https://github.com/fuchsia-mirror/docs/blob/master/the-book/....
- rohan1024 8y ago
  I was wondering who updated it. Thanks.
- romeisendcoming 8y ago
  Lost me at 'story' 5 times in a paragraph. Get rid of the story and get on with the real relation.

starclaps 8y ago

That’s why tools like gosec complain when user input is used to access files - specifically because ‚..‘ allows one to „escape“. Gosec: https://github.com/securego/gosec https://github.com/securego/gosec

akerl_ 8y ago

Given that they’re still resolving “..”, they’re just resolving it in code client-side before requesting the content, they really haven’t gotten rid of “..”: they’ve implemented a path sanitization library, similarly to how many other frameworks (notably those designed to serve files over the web) behave.

ivanbakel 8y ago
That is getting rid of ".." - the client-side version is fundamentally different in behaviour, since it can't be resolved without prior knowledge of the filesystem. ".." isn't being sanitised from user paths, it's being translated, because there is no equivalent concept in the file server.
itp 8y ago
No, it's deeper than that. They are describing the capabilities of a process handed a file descriptor, which means it does not have a path at all. There's no way to resolve '..' relative to no path. Compare that to a POSIX system where a directory has an actual child which is a reference to the directory's parent whose name is always '..'. The talk about resolving '..' is merely a demonstration that the behavior of "cd .." can be supported/emulated in a context where you have both an open directory and corresponding path, without requiring that '..' literally exist.
- loeg 8y ago
  To add to that, as the document writes in the very first section, it can sort of be thought of as a chroot. You can't .. out of a chroot; there is no "..". (Yeah, it's not exactly a chroot; that's well covered in the document.)

delinka 8y ago

>If a handle is provided to a directory, it should imply access to resources within that directory (and additionally, their subdirectories). Maybe within Fuschia, but not in my UNIXy systems. All users on a system need r-x on /Users, but that doesn't give them r-x on every user's home directory located as a subdirectoy of /Users - a handle acquired by bob to /Users does not imply access to /Users/delinka. Further, if your process is treating '..' as a subdirectory, you're doing it wrong. Paths must be normalized (e.g. ~ expanded, . and .. resolved) before requesting a handle via absolute path. Lastly, this document reads as if knowing a full path grants access to that path and its subdirectories. If that's the case ... oy.

wahern 8y ago
> Further, if your process is treating '..' as a subdirectory, you're doing it wrong. Paths must be normalized (e.g. ~ expanded, . and .. resolved) before requesting a handle via absolute path. ".." is required for atomic traversal (similar to symlinks), which is important in some situations, such as making sure a file tree you've descended isn't removed out from under you in a way that breaks your state. (The directory itself can be moved while you hold the handle, but the important thing is that being able to rely on ".." permits certain algorithms that are safer and more consistent.) Canonicalizing paths introduces race conditions between canonicalization and actual access, which is why this is performed in the kernel. Canonicalizing paths makes sense if you're accepting paths from untrusted sources and you cannot make use of POSIX openat() + extensions like O_BENEATH. HTTP GET requests are supposed to be idempotent, anyhow, so there shouldn't exist any sort of race condition as a conceptual matter. But in regular software, it's better to just pass paths as-is. The shell performs "~" expansion, but it doesn't resolve "." or "..". And the shell performs other expansions, like file globbing, for which there's rarely any reason to implement in regular software. Supporting "~" expansion but not file globbing is inconsistent; if you're not the shell, don't implement shell-like features as it just creates confusion and unnecessary complexity. Calling ".." a holdout from POSIX is misleading. The semantics exist for legitimate and even security-relevant reasons, albeit not necessarily reasons that an embedded smartphone OS may care about. What POSIX lacks is O_BENEATH (from Capsicum) or a similar flag for tagging a directory descriptor to prevent relative paths or otherwise ascending the tree. Capsicum extensions make POSIX perfectly capable of implementing strict capability semantics, and they do so by extending semantics in a manner consistent with POSIX. POSIX isn't inherently broken in this regard, it's just perhaps too feature rich for some use cases and not feature rich enough for others. There's no end of complaints about POSIX--either it's too complex or too simple. The fact is, no single specification will ever please anybody, so griping about how POSIX is broken is not only pointless but belies a failure to appreciate the underling issues. The alternative to POSIX is basically nothing. Standardizing on Linux is, at best, a lateral move. Adding optional components to specifications has proven the worse of all worlds, and like most other standards POSIX has been slowly removing some optional components altogether while making others mandatory.

monocasa 8y ago

> this is an essential idea encompassing microkernels, and other “capability-based” systems Tiny nit, but microkernels don't imply a capability based security model. For instance Mach, QNX, Redox, etc. aren't capability based. It's a very good idea for your microkernel to be capability based because it cuts a lot of validation out of the critical path for IPC, but it's by no means a requirement.

comex 8y ago
Mach is capability based. (Mach ports are capabilities.)
- monocasa 8y ago
  You're totally right of course. I was falsely under the impression that the Mach port table only had a single global namespace.

saagarjha 8y ago

> at the time of writing, symbolic links do not exist on Fuchsia Is this for technical reasons, or similar philosophical ones because symbolic links also allow for escaping from “jails”?

loeg 8y ago
I think "at the time of writing" implies they're still under consideration, rather than outright rejected due to philosophical differences.
jepler 8y ago
Their whole "scheme", where ".." can be dealt with as a canonicalizing step BEFORE submitting the path to the OS for actual opening, seems to require refinement when symlinks are involved. Take a perfectly spherical unix: $ mkdir /tmp/hn $ cd /tmp/hn $ ln -s . foo; mkdir bar; touch baz $ ls -l bar/../baz foo/baz foo/../baz ls: cannot access 'foo/../baz': No such file or directory -rw-r--r-- 1 jepler jepler 0 Nov 28 18:21 bar/../baz -rw-r--r-- 1 jepler jepler 0 Nov 28 18:21 foo/baz Fuscsia-with-symlinks would have some extra hoops to jump through to make rewriting act like POSIX in the case that /tmp/hn is not escaped, which surely you'd want if you went ahead and introduced symlinks.
- Dylan16807 8y ago
  > which surely you'd want Nope. I don't want that at all. There's a reason that by default cd goes out of its way to make .. ignore the parent of a symlink.
cyphar 8y ago
It should be noted that this is not without precedent. Plan9 doesn't have symlinks for example, everything is done using binds. You would be surprised how many complications come up when you add symlinks to path resolution code.

yiyus 8y ago

See also: http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames

theoh 8y ago
This paper, depending on how you look at it, either justifies the existence of dot dot, or makes it seem like a wart. IMO dot dot is too well-established and essentially sensible (in a minimal UNIX fashion) for it to be removed on a whim. Consider the approach taken by https://man.openbsd.org/unveil.2 https://man.openbsd.org/unveil.2: if you try to access a file you aren't explicitly allowed to, you get EACCESS or ENOENT. I'm not an expert, and the possibility of errors obviously requires applications to tread more carefully than they might have previously, but it seems like a clean solution.
- kiriakasis 8y ago
  > removed on a whim They changed the whole concept of having one single root. As I understand the plan is to use many fragmented and independent filesystem handles that can optionally mounted together. So the restriction is more about non being able to access a folder if you don't have access to an appropriate handle. Paths like folder1/folder2/../folder2/file are still perfectly fine.
  - theoh 8y ago
    To clarify: that's the Plan 9 situation. I'm talking about the merits of the Fuchsia situation (".. is no longer available"). References to a forbidden parent directory from a chroot can just return ENOENT, because it doesn't exist in that universe. I may not be understanding this fully, but to condemn ".." based on some accusation that it's incompatible with chroot semantics (or "a holdout from POSIX") seems tendentious.
  - AndyMcConachie 8y ago
    Didn't MS-DOS already do this with letters? (e.g., a:, b:, c:) Maybe I don't really understand Fuchsia's approach. But I do really like OpenBSD's approach in unveil.

mattigames 8y ago

For irony sake one may link this file as https://github.com/fuchsia-mirror/docs/blob/master/the-book/../../master/the-book/dotdot.md

Someone1234 8y ago
Not as ironic as you'd think considering the article specifically talks about support for .. in userland tooling and uses CWD as an example.
kiriakasis 8y ago
well actually the article complains about stuff like https://github.com/fuchsia-mirror/../internal-page/secret-report/ https://github.com/fuchsia-mirror/../internal-page/secret-re... which quite more serious
Retr0spectrum 8y ago
Nope, that actually 404's. It may appear to work, because most browsers will apply some equivalent to the `__fdio_cleanpath()` function mentioned in the article, resolving the traversal locally. You can verify this as follows: curl -I --path-as-is 'https://github.com/fuchsia-mirror/docs/blob/master/the-book/../../master/the-book/dotdot.md'
- mattigames 8y ago
  if most browsers apply it... doesn't that means it works? Maybe not in the resolved-by-the-server-returning-a-304 sense but in at least some sense.

csours 8y ago

I'm very surprised that I agree after reading. As I read initially, I was thinking "what about..." but it's really about separation of concerns.

dsl 8y ago

It is ironic this is coming out of a company with an internal site dedicated to depreciating "Considered Harmful" because it is the epitome of grandstanding. Here is some further reading on why we should downvote these types of things into oblivion: https://meyerweb.com/eric/comment/chech.html https://meyerweb.com/eric/comment/chech.html

gowld 8y ago
That's a silly article. The essay says that we should disregard essays if they have a title like the essay's own, so... pradox. When "considered harmful" is considered flaimbait rhetoric, perhaps the problem is with the readers who refuse to engage with any mildly-worded criticism.
- donatj 8y ago
  It’s surely in how the reader interprets it, but I always read “considered harmful” as quite mild but to the point. Something like “isn’t great”. A refreshing opposite of clickbait as the actual proposition is in the title.

vermilingua 8y ago

This may be off topic, and is hard to ask without doomsaying: but can we trust Fuchsia? Google is a machine for turning user's personal data into their dollars, and has been getting more and more crafty at achieving this goal. I want to believe that this is a good-natured effort at improving the state of modern operating systems; but I feel like I've been burned by my trust in Google too many times.

landryraccoon 8y ago
> I feel like I've been burned by my trust in Google too many times I'm genuinely curious about this. In what way do you feel burned? I think I get what you're saying. I use gmail and Google uses an algorithm on gmail to decide what ads to show me when I do a search. The ads make money for Google and I don't get any of that money. All I get is free search results. I think what you're saying is that because Google is using your information to show you ads and make money that somehow you feel cheated? But I don't feel particularly burned by this arrangement. If Google gave an option to start charging me money to avoid ads if I'm honest with myself I'm pretty sure I'll choose to continue seeing ads instead. I guess I'm curious about exactly how you feel burned.
- eridius 8y ago
  Probably the most obvious example is when Google almost destroyed the entire RSS ecosystem with Google Reader.
  - SquareWheel 8y ago
    I'd argue that competitors flourished when Google Reader went down. I still use RSS today. Besides, it's certainly not their responsibility to maintain a service indefinitely.
    - eridius 8y ago
      It was a close call. For a number of months there it looked like RSS was going to be well and truly dead. The fact that my comment is being downvoted tells me that people here have a shockingly poor memory for this sort of thing. And heck, it's not even the death of Google Reader that was bad for RSS. The life of Google Reader was bad for RSS too. It was a "good enough" product released for free, as in good enough that it wasn't worth trying to compete with a free product so nobody did, but it wasn't actively maintained and just served to cause the entire RSS ecosystem to stagnate for years.

mjfl 8y ago

"considered harmful" considered harmful The origin of the meme was an editor's labeling of Dijkstra's commentary. Titling your own article this makes it seem like you wrote the article, forgot you wrote it, found it again, and are presenting it to others as an interesting perspective.

vinceguidry 8y ago

Coming from Ruby, NodeJS annoys the crap out of me with its lack of decent autoload functionality. Seeing relative paths in modules really annoys me. JS world can't figure out what should be global and what should be local it seems.

ricardobeat 8y ago
What do you mean? Global packages are global, local packages are local.
loeg 8y ago
Wrong thread?

hnbroseph 8y ago

i really wish the "considered harmful" meme/trope/whatever would take a nap.

DSingularity 8y ago

Any good papers to read on fuchsia?

rohan1024 8y ago
This is ironic cd.. Here you go https://fuchsia.googlesource.com/docs/+/HEAD/the-book/ https://fuchsia.googlesource.com/docs/+/HEAD/the-book/

rustcharm 8y ago

In "classic" Unix, isn't ".." really just another file name, created by default as a hard link to the parent directory?

infinity0 8y ago

> What about shell traversal? > [..] For example, if the CWD is “/foo/bar”, and a user calls “cd ..”, then the underlying call may be transformed into “chdir /foo/bar/..”, which can be canonicalized to “/foo”. So fuchsia also won't have CWD, then? Because if it has CWD, then the process can always chdir / rendering this lack-of-.. exercise pointless.

IshKebab 8y ago
I don't see what made you think it won't have a CWD.
rictic 8y ago
fuchsia is not posix, so they can (and I suspect have) change the semantics of chdir to be more restrictive.
bun_at_work 8y ago
I think the idea in Fuchsia is that directories can be passed as resources to children processes, which can then move freely around in that directory structure. If the parent passes a child "/foo/bar/", the child sees "bar/" and it's contents, which might be "bar/baz/bingo/". The child can then move to "/bar/baz/bingo/" do work there, and then recurse up the hierarchy. It simply cannot ".." out of "foo/". Hope that makes sense.

IgorPartola 8y ago

So can a process create a symlink a la ln -s ../../.. root And gain access? Or are symlinks now fucked with too? This seems like a poorly thought out plan, and unnecessary at that.

otabdeveloper1 8y ago
> This seems like a poorly thought out plan, and unnecessary at that. Welcome to Google Future (c). Enjoy your stay.