7 ms·

What you are seeing is someone picking up the baton and running with it. HTTP 1.1 has drawbacks which we've been living with for far too long. Getting people us

What you are seeing is someone picking up the baton and running with it. HTTP 1.1 has drawbacks which we've been living with for far too long. Getting people used to change was step one, that was HTTP/2, now were in a position to fix pain points. I'd also like to see updates to IMAP, SMTP and FTP to name a few.

numbsafari 8y ago
I'm with you on IMAP and SMTP, as they serve a specific purpose. One thing I don't understand, is why anyone is holding on to FTP at this stage over HTTP. Outside of legacy enterprise "file transfer" appliances, which I deal with all day and can appreciate the inertia around, I just don't understand what the benefit is over HTTP.
- drb91 8y ago
  There’s no standard for HTTP file uploads, with all the permissions that come along with it. Eg you can POST a file to a path, but the behavior is undefined, as is authorization, etc. S3 is probably the closest thing to replacing FTP, and I would hazard a guess that there just isn’t sufficient reason to dump ftp and upgrade infrastructure. Ftp is surprisingly common when integrating between parties, and this can be expensive to change for non technical reasons.
  - treve 8y ago
    The standard to replace FTP is/was WebDAV. It's not a very modern standard, but it's a standard.
    - drb91 8y ago
      Good point; I never used that directly. Why do you think that hasn’t supplanted FTP?
      - dragonwriter 8y ago
        Because SFTP is a better fit for simple (but secure) file transfer, WebDAV handles a lot more, but it's overkill for most of the things FTP was used for. OTOH, AFAICT, SFTP has largely replaced FTP for the system-to-system role (though ad hoc HTTP-based protocols are also common.)
        drb91 8y ago
        I agree re: the sftp point; people often refer to it as ftp given the interface mimicry.
      - amaccuish 8y ago
        For me I've always found WebDAV to my owncloud server from Windows and Mac dreadfully slow. The sync client itself that uses WebDAV is just fine, so I'd put my guess on poor integration and client implementations.
    - londons_explore 8y ago
      Microsoft kinda broke WebDAV by adding various incompatible extensions and building a horribly broken client into Windows. When users found out about all the issues, they preferred sticking to the old FTP, having a desktop sync app (eg. Google Drive), or using a WebUI for managing files (eg. Dropbox's website) etc.
    - fulafel 8y ago
      Or the S3 protocol, if de facto standards count.
  - ahje 8y ago
    PUT request with HTTP authentication is quite standardized, isn't it? The problem isn't that there are no standardized way of doing it. The problem is that the existing standardized way of doing it has no adoption, because it's not really optimal for current needs.
    - oconnor663 8y ago
      Could you say more about that? Is it just a question of integrating smoothly with websites and cookies, or is it a performance issue?
      - ahje 8y ago
        Frankly, I don't know why most people have chosen to use POST requests instead of PUT. Browsers and servers usually support both PUT and HTTP authentication, and when used over HTTPS it should be quite safe.
    - ape4 8y ago
      With FTP you login and are then in your home directory. Then you upload. With HTTP authentication is a home directory required? Can you do `pwd` to query your home folder. Some FTP clients allow standboxing where you are in /home/user folder but `pwd` says `/`.
      - tptacek 8y ago
        There are like at least 3 different mainstream ways to "log in" to a web server, and all of them are more straightforward than FTP's insane mainframe-era control-channel/data-channel design. The "sandboxing" you're referring to is a serverside chroot, for what it's worth. And, of course, web servers have been doing that since NCSA httpd.
        drb91 8y ago
        In this case, choice may not be a benefit for coordination. If there’s one way to do it, there’s much less to communicate and debug.
  - dragonwriter 8y ago
    > There’s no standard for HTTP file uploads There is a well-established and widely-support IETF standard mechanism (consisting of several IETF standards) for that; it's called WebDAV. > S3 is probably the closest thing to replacing FTP, SFTP is much closer to replacing FTP, to the point that people often say FTP when they mean SFTP, which underlies most non-HTTPS enterprise integrations I've seen in the last decade. But SFTP isn't FTP (not even FTP-over-TLS, which is FTPS, which has much less use.) > Ftp is surprisingly common when integrating between parties SFTP is very common. FTP (including FTPS) is surprisingly common in the sense that any use of it is surprising given the well-established, battle tested, and superior in every way alternatives that are readily available.
otabdeveloper1 8y ago
> HTTP 1.1 has drawbacks which we've been living with for far too long. No it doesn't. Don't fix what's not broke, and doubly so if the only motivation for the change is "well, Megacorp (c) says it's the bee's knees".
- dagenix 8y ago
  > No it doesn't. Saying that anything has no drawbacks is pretty much always going too far.
- Dylan16807 8y ago
  I find "you can't reliably use pipelining" to be a pretty significant drawback.
- h1d 8y ago
  > Don't fix what's not broke Yeah, the world's communication before the Internet existed wasn't exactly broken either, so let's not invent anything? Poor choice.
tptacek 8y ago
The most reasonable "update" to FTP would be to formally replace it with HTTP, because FTP is an awful protocol --- maybe the worst one in common use --- that deserved to die off decades ago.
- skissane 8y ago
  FTP includes support for record-oriented files (STRU R, MODE B). This mostly isn't supported by FTP clients/servers on Unix-like platforms or Windows, but it is on those platforms which have record-oriented filesystem support (IBM mainframes, IBM i, Unisys mainframes, OpenVMS RMS, etc.) Although one could standardise a mechanism for transferring record-oriented files over HTTP, no such standard has been widely adopted. If someone wants to transfer a record-oriented file from e.g. VMS to z/OS and have the record boundaries kept (and with the necessary ASCII-EBCDIC conversion applied), FTP is the only widely adopted standard that can do that. This is also why these platforms often use FTPS (FTP over TLS) instead of SFTP (SSH-based) – SFTP doesn't include any support for record-oriented files, only the stream-oriented files used on Unix and Windows.
  - kbenson 8y ago
    > Although one could standardise a mechanism for transferring record-oriented files over HTTP Or you could just provide a simple API over HTTP, whether an actual REST system or a single endpoint one or two params with well defined inputs that can be accepted (a CGI, basically). Why bother formalizing some standard when the tools to handle this are so ubiquitous (Apache+$LANG on the server, cURL or wget on the client)? > SFTP doesn't include any support for record-oriented files, only the stream-oriented files used on Unix and Windows. That's because SFTP isn't really FTP (in the protocol sense) at all. It's just a specialized shell started after an SSH session/tunnel is created. That it includes FTP in the name is really just marketing because they wanted to supersede the real FTP. In that respect, it makes sense for them to just cover what 99% of the users of FTP needed and stop.
    - skissane 8y ago
      curl and wget on the client can't easily do this, when the client is another mainframe/minicomputer OS with a record-oriented filesystem. I don't believe they have any platform-specific code to support record-oriented files. You probably can get it to work with external configuration (e.g. on z/OS, using JCL to invoke curl/wget with a DD statement which sets the necessary dataset parameters.) But FTP-over-TLS is already a well-documented and well-understood technology in mainframe environments. What possible advantage could one get by replacing it with something hacked together with Apache/curl/wget?
protomyth 8y ago
I'd also like to see updates to IMAP, SMTP and FTP to name a few. I would really like to see each of these disappear to be replaced by new ways of doing things. The whole concept of e-mail needs a new thought. FTP should have already gone the way of the dodo if for nothing else the firewall issues its has.
- stephenr 8y ago
  Apart from mandating TLS, what's wrong with email? FTP I agree - SFTP is an existing better option.
  - h1d 8y ago
    Email protocol is so dated people can only keep patching with hacks. It may look like it's working from users' point of view but check how encoding is handled, it's a mess and then why is there still no widely deployed end to end encryption for such a core protocol? Also DNS and domain is broken too. There's no point using UDP which only brings security problems and still no decent way to encrypt which domain you're querying through your browser and see how domain is governed by a structure which make crappy rules like who owns new TLD and you get some absurd "base" price for each domains instead of something more transparent and public. I wish if you write a book, you could easily have 'some-title.book' or when releasing a movie, you could get 'some-title.movie' but somehow it went wrong and no one can fix it. Lack of improvements on the security on these points make me wonder if agencies just want to keep them dated so they can tap into people's communication easily instead of making the world a better place where everything is p2p encrypted.
    - tialaramex 8y ago
      For DNS: D-PRIVE protocols including DoH and DoT prevent eavesdroppers seeing your queries or tampering with them. DNSSEC lets your client ensure the answers you see are genuine. For hiding hostnames eSNI is under development. Cloudflare with a recent Firefox nightly lets you see this for yourself.
      - 8y ago

numbsafari 8y ago

I'm with you on IMAP and SMTP, as they serve a specific purpose. One thing I don't understand, is why anyone is holding on to FTP at this stage over HTTP. Outside of legacy enterprise "file transfer" appliances, which I deal with all day and can appreciate the inertia around, I just don't understand what the benefit is over HTTP.

drb91 8y ago
There’s no standard for HTTP file uploads, with all the permissions that come along with it. Eg you can POST a file to a path, but the behavior is undefined, as is authorization, etc. S3 is probably the closest thing to replacing FTP, and I would hazard a guess that there just isn’t sufficient reason to dump ftp and upgrade infrastructure. Ftp is surprisingly common when integrating between parties, and this can be expensive to change for non technical reasons.
- treve 8y ago
  The standard to replace FTP is/was WebDAV. It's not a very modern standard, but it's a standard.
  - drb91 8y ago
    Good point; I never used that directly. Why do you think that hasn’t supplanted FTP?
    - dragonwriter 8y ago
      Because SFTP is a better fit for simple (but secure) file transfer, WebDAV handles a lot more, but it's overkill for most of the things FTP was used for. OTOH, AFAICT, SFTP has largely replaced FTP for the system-to-system role (though ad hoc HTTP-based protocols are also common.)
      - drb91 8y ago
        I agree re: the sftp point; people often refer to it as ftp given the interface mimicry.
    - amaccuish 8y ago
      For me I've always found WebDAV to my owncloud server from Windows and Mac dreadfully slow. The sync client itself that uses WebDAV is just fine, so I'd put my guess on poor integration and client implementations.
  - londons_explore 8y ago
    Microsoft kinda broke WebDAV by adding various incompatible extensions and building a horribly broken client into Windows. When users found out about all the issues, they preferred sticking to the old FTP, having a desktop sync app (eg. Google Drive), or using a WebUI for managing files (eg. Dropbox's website) etc.
  - fulafel 8y ago
    Or the S3 protocol, if de facto standards count.
- ahje 8y ago
  PUT request with HTTP authentication is quite standardized, isn't it? The problem isn't that there are no standardized way of doing it. The problem is that the existing standardized way of doing it has no adoption, because it's not really optimal for current needs.
  - oconnor663 8y ago
    Could you say more about that? Is it just a question of integrating smoothly with websites and cookies, or is it a performance issue?
    - ahje 8y ago
      Frankly, I don't know why most people have chosen to use POST requests instead of PUT. Browsers and servers usually support both PUT and HTTP authentication, and when used over HTTPS it should be quite safe.
  - ape4 8y ago
    With FTP you login and are then in your home directory. Then you upload. With HTTP authentication is a home directory required? Can you do `pwd` to query your home folder. Some FTP clients allow standboxing where you are in /home/user folder but `pwd` says `/`.
    - tptacek 8y ago
      There are like at least 3 different mainstream ways to "log in" to a web server, and all of them are more straightforward than FTP's insane mainframe-era control-channel/data-channel design. The "sandboxing" you're referring to is a serverside chroot, for what it's worth. And, of course, web servers have been doing that since NCSA httpd.
      - drb91 8y ago
        In this case, choice may not be a benefit for coordination. If there’s one way to do it, there’s much less to communicate and debug.
- dragonwriter 8y ago
  > There’s no standard for HTTP file uploads There is a well-established and widely-support IETF standard mechanism (consisting of several IETF standards) for that; it's called WebDAV. > S3 is probably the closest thing to replacing FTP, SFTP is much closer to replacing FTP, to the point that people often say FTP when they mean SFTP, which underlies most non-HTTPS enterprise integrations I've seen in the last decade. But SFTP isn't FTP (not even FTP-over-TLS, which is FTPS, which has much less use.) > Ftp is surprisingly common when integrating between parties SFTP is very common. FTP (including FTPS) is surprisingly common in the sense that any use of it is surprising given the well-established, battle tested, and superior in every way alternatives that are readily available.

otabdeveloper1 8y ago

> HTTP 1.1 has drawbacks which we've been living with for far too long. No it doesn't. Don't fix what's not broke, and doubly so if the only motivation for the change is "well, Megacorp (c) says it's the bee's knees".

dagenix 8y ago
> No it doesn't. Saying that anything has no drawbacks is pretty much always going too far.
Dylan16807 8y ago
I find "you can't reliably use pipelining" to be a pretty significant drawback.
h1d 8y ago
> Don't fix what's not broke Yeah, the world's communication before the Internet existed wasn't exactly broken either, so let's not invent anything? Poor choice.

tptacek 8y ago

The most reasonable "update" to FTP would be to formally replace it with HTTP, because FTP is an awful protocol --- maybe the worst one in common use --- that deserved to die off decades ago.

skissane 8y ago
FTP includes support for record-oriented files (STRU R, MODE B). This mostly isn't supported by FTP clients/servers on Unix-like platforms or Windows, but it is on those platforms which have record-oriented filesystem support (IBM mainframes, IBM i, Unisys mainframes, OpenVMS RMS, etc.) Although one could standardise a mechanism for transferring record-oriented files over HTTP, no such standard has been widely adopted. If someone wants to transfer a record-oriented file from e.g. VMS to z/OS and have the record boundaries kept (and with the necessary ASCII-EBCDIC conversion applied), FTP is the only widely adopted standard that can do that. This is also why these platforms often use FTPS (FTP over TLS) instead of SFTP (SSH-based) – SFTP doesn't include any support for record-oriented files, only the stream-oriented files used on Unix and Windows.
- kbenson 8y ago
  > Although one could standardise a mechanism for transferring record-oriented files over HTTP Or you could just provide a simple API over HTTP, whether an actual REST system or a single endpoint one or two params with well defined inputs that can be accepted (a CGI, basically). Why bother formalizing some standard when the tools to handle this are so ubiquitous (Apache+$LANG on the server, cURL or wget on the client)? > SFTP doesn't include any support for record-oriented files, only the stream-oriented files used on Unix and Windows. That's because SFTP isn't really FTP (in the protocol sense) at all. It's just a specialized shell started after an SSH session/tunnel is created. That it includes FTP in the name is really just marketing because they wanted to supersede the real FTP. In that respect, it makes sense for them to just cover what 99% of the users of FTP needed and stop.
  - skissane 8y ago
    curl and wget on the client can't easily do this, when the client is another mainframe/minicomputer OS with a record-oriented filesystem. I don't believe they have any platform-specific code to support record-oriented files. You probably can get it to work with external configuration (e.g. on z/OS, using JCL to invoke curl/wget with a DD statement which sets the necessary dataset parameters.) But FTP-over-TLS is already a well-documented and well-understood technology in mainframe environments. What possible advantage could one get by replacing it with something hacked together with Apache/curl/wget?

protomyth 8y ago

I'd also like to see updates to IMAP, SMTP and FTP to name a few. I would really like to see each of these disappear to be replaced by new ways of doing things. The whole concept of e-mail needs a new thought. FTP should have already gone the way of the dodo if for nothing else the firewall issues its has.

stephenr 8y ago
Apart from mandating TLS, what's wrong with email? FTP I agree - SFTP is an existing better option.
- h1d 8y ago
  Email protocol is so dated people can only keep patching with hacks. It may look like it's working from users' point of view but check how encoding is handled, it's a mess and then why is there still no widely deployed end to end encryption for such a core protocol? Also DNS and domain is broken too. There's no point using UDP which only brings security problems and still no decent way to encrypt which domain you're querying through your browser and see how domain is governed by a structure which make crappy rules like who owns new TLD and you get some absurd "base" price for each domains instead of something more transparent and public. I wish if you write a book, you could easily have 'some-title.book' or when releasing a movie, you could get 'some-title.movie' but somehow it went wrong and no one can fix it. Lack of improvements on the security on these points make me wonder if agencies just want to keep them dated so they can tap into people's communication easily instead of making the world a better place where everything is p2p encrypted.
  - tialaramex 8y ago
    For DNS: D-PRIVE protocols including DoH and DoT prevent eavesdroppers seeing your queries or tampering with them. DNSSEC lets your client ensure the answers you see are genuine. For hiding hostnames eSNI is under development. Cloudflare with a recent Firefox nightly lets you see this for yourself.
    - 8y ago