5 ms·

For what it's worth, the file:///foo/bar url means: 1. an empty hostname 2. path = 'foo/bar' In the file:// url an empty hostname implies 'localhost'. The fil

by treve 10y ago

For what it's worth, the file:///foo/bar url means: 1. an empty hostname 2. path = 'foo/bar' In the file:// url an empty hostname implies 'localhost'. The file:// is, as far as I know, unique in this regard. When parsing URI's generically, and you encounter a URI like: scheme:foo/bar The meaning of 'foo/bar' is also a path. There's many URI schemes like this, such as mailto: and urn:. So given that, file:foo/bar should probably throw an error, but if not, it should be canonicalized into file:///foo/bar (triple slash), because file://foo/bar refers to the 'bar' file on the 'foo' host, and file:///foo/bar refers to /foo/bar on localhost.

derefr 10y ago
One annoying thing: web browsers parse URLs two different ways! 1. Typing "example.com" into your address bar takes you to "//example.com/" (root path of example.com domain in current/default schema) 2. Clicking on a link with a href="example.com" takes you to "///./example.com" (that is: current domain, current schema, relative path reference). The URL parsers built into most runtimes use behavior #2, and I can see the usefulness of it in the sense that you can sort of treat a URL as an extended version of a filesystem Path object, where any string that forms a valid Path (relative or absolute) also forms a valid URL with equivalent semantics. But most URLs people type, in the wild, are implicitly assuming behaviour #1. If you write an unstructured-text "ingestor" that extracts URLs embedded in plaintext on the web or in print, and tries to dereference said URLs, only approach #1 will get you anywhere. That said, I've never seen a single URL library that exposes any parsing API for type-1 URL fragments. It'd be extremely useful for parsing URLs entered by humans as responses to prompts.
- greglindahl 10y ago
  Google around for a "public suffix list" library for your favorite language, it will aid you in guessing that 'example.com' could be a valid FQDN. As a crawler guy, I see common suffix list features in the kinds of url libraries that I use or write.
  - tokenizerrr 10y ago
    Don't forget to update your library whenever a new gTLD comes out
    - greglindahl 10y ago
      It appears that most libraries download the list from Mozilla, but yes, you do need to ensure that it's downloaded frequently.
- groby_b 10y ago
  That'd be because type-1 fragments can be parsed by prefixing them with http:// http:// - or, more technically, humans treat the scheme as implicit and assume the first component is always the net location. (a.k.a domain name) Type-2 URLs are all considered relative URLs, which is why all URL libraries have a resolver concept for relative URLs. (RFC1808 has the gory details.) The problem with type-1's is that they're not in the correct shape for relative address resolution, and so you're always engaging in a bit of gymnastics figuring out if somebody typed out part of the implicit agreement. And if you were to treat them as relative, there's ambiguity, so they can't be allowed in HTML pages. I.e. href="www.example.com" is not clear in type-1 - it could be a domain, it could be a relative URL for the www.example.com file. In type-2, that's entirely clear - there's no scheme, and no net location, so it must be a path. Conversely, we can't use type-2 urls exclusively, because humans would quickly realize that all useful urls start with a double-slash, and would want to start leaving that off, because "all URLs are like that". Most people oddly lack the ability to keep everything strictly clean according to a beautiful grammar and instead thing eliding common prefixes is fine, even if they span multiple distinct items. (There's a lesson for people who design grammars in there, but it's too late to learn the lesson on the web) And here we are. And don't even get me started on the fact that the address bar, on top of all of that, also needs to try to figure out if it's a URL (and which kind), or a query, or maybe a prefix for a custom search engine. As usual, having users makes things really complicated ;)
JoshTriplett 10y ago
> When parsing URI's generically, and you encounter a URI like: > scheme:foo/bar > The meaning of 'foo/bar' is also a path. You can't count on that; a URI scheme can assign arbitrary semantics to what appears after the ':'. For instance, the "mid" URI scheme refers to a Message-Id (not a path), and the "cid" URI scheme refers to a MIMEd part of a message.
userbinator 10y ago
In the file:// url an empty hostname implies 'localhost'. It is quite obvious how the resource is to be retrieved in that case, but one wonders how the resource is to be retrieved in the case that the hostname is not empty --- this is straightforward to figure out for http:// http://, https:// https://, and ftp://, but not file://; in other words, what is its protocol?

by treve 10y ago

derefr 10y ago

One annoying thing: web browsers parse URLs two different ways! 1. Typing "example.com" into your address bar takes you to "//example.com/" (root path of example.com domain in current/default schema) 2. Clicking on a link with a href="example.com" takes you to "///./example.com" (that is: current domain, current schema, relative path reference). The URL parsers built into most runtimes use behavior #2, and I can see the usefulness of it in the sense that you can sort of treat a URL as an extended version of a filesystem Path object, where any string that forms a valid Path (relative or absolute) also forms a valid URL with equivalent semantics. But most URLs people type, in the wild, are implicitly assuming behaviour #1. If you write an unstructured-text "ingestor" that extracts URLs embedded in plaintext on the web or in print, and tries to dereference said URLs, only approach #1 will get you anywhere. That said, I've never seen a single URL library that exposes any parsing API for type-1 URL fragments. It'd be extremely useful for parsing URLs entered by humans as responses to prompts.

greglindahl 10y ago
Google around for a "public suffix list" library for your favorite language, it will aid you in guessing that 'example.com' could be a valid FQDN. As a crawler guy, I see common suffix list features in the kinds of url libraries that I use or write.
- tokenizerrr 10y ago
  Don't forget to update your library whenever a new gTLD comes out
  - greglindahl 10y ago
    It appears that most libraries download the list from Mozilla, but yes, you do need to ensure that it's downloaded frequently.
groby_b 10y ago
That'd be because type-1 fragments can be parsed by prefixing them with http:// http:// - or, more technically, humans treat the scheme as implicit and assume the first component is always the net location. (a.k.a domain name) Type-2 URLs are all considered relative URLs, which is why all URL libraries have a resolver concept for relative URLs. (RFC1808 has the gory details.) The problem with type-1's is that they're not in the correct shape for relative address resolution, and so you're always engaging in a bit of gymnastics figuring out if somebody typed out part of the implicit agreement. And if you were to treat them as relative, there's ambiguity, so they can't be allowed in HTML pages. I.e. href="www.example.com" is not clear in type-1 - it could be a domain, it could be a relative URL for the www.example.com file. In type-2, that's entirely clear - there's no scheme, and no net location, so it must be a path. Conversely, we can't use type-2 urls exclusively, because humans would quickly realize that all useful urls start with a double-slash, and would want to start leaving that off, because "all URLs are like that". Most people oddly lack the ability to keep everything strictly clean according to a beautiful grammar and instead thing eliding common prefixes is fine, even if they span multiple distinct items. (There's a lesson for people who design grammars in there, but it's too late to learn the lesson on the web) And here we are. And don't even get me started on the fact that the address bar, on top of all of that, also needs to try to figure out if it's a URL (and which kind), or a query, or maybe a prefix for a custom search engine. As usual, having users makes things really complicated ;)

JoshTriplett 10y ago

> When parsing URI's generically, and you encounter a URI like: > scheme:foo/bar > The meaning of 'foo/bar' is also a path. You can't count on that; a URI scheme can assign arbitrary semantics to what appears after the ':'. For instance, the "mid" URI scheme refers to a Message-Id (not a path), and the "cid" URI scheme refers to a MIMEd part of a message.

userbinator 10y ago

In the file:// url an empty hostname implies 'localhost'. It is quite obvious how the resource is to be retrieved in that case, but one wonders how the resource is to be retrieved in the case that the hostname is not empty --- this is straightforward to figure out for http:// http://, https:// https://, and ftp://, but not file://; in other words, what is its protocol?