11 ms·
Shutting down public FTP services
- LeoPanthera 10y agoPublic FTP servers were where I downloaded most of the software for my computers, back in the 90s. There's nothing really like it anymore - you can't have anonymous sftp. But perhaps we don't care anymore. The web is gradually consuming all that came before it.
- mschuster91 10y ago> The web is gradually consuming all that came before it. It's about cost, too. HTTP can be cached very efficiently, but FTP not at all. If I were the operator in charge and I had the choice between next-to-free caching by nearly anything, be it a Squid proxy, apt-cache or nexus, or no caching and having to maintain expensive servers, I'd choose HTTP.
- agumonkey 10y agoWhat we need if FTP over HTTP.
- astrodust 10y agoWhat about Gopher? Will nobody think of Gopher?
- badsectoracula 10y agoActually i thought about Gopher (i even have my own client - http://runtimeterror.com/tools/gopher/ http://runtimeterror.com/tools/gopher/ - although it only does text) since it basically behaves as FTP++ with abstract names (sadly most modern gopherholes treat it as hypertext lite by abusing the information nodes). Gopher generally avoids most of FTP's pitfalls and it is dead easy to implement.
- kevin_thibedeau 10y agoAt least there's still Jigdo.
- dmoreno 10y agoAs a side note, there was a nice podcast about Gopher a few days ago at Techstuff [1] [1] http://shows.howstuffworks.com/techstuff/what-was-gopher.htm http://shows.howstuffworks.com/techstuff/what-was-gopher.htm
- mschuster91 10y agoWell, there is WebDAV. At least Windows and OS X support it (Windows from Explorer, OS X from Finder), no idea about mainstream Linux/Android/iOS support though. Also, no idea if WebDAV can deal with Unix or Windows permissions, but I did not have that problem when I set up a WebDAV server a year ago. IIRC WebDAV uses GET for retrieval, so the read parts can be cached by an intermediate proxy and the write part be relayed to the server.
- donatj 10y agoAs someone who once tried to write a WebDAV server I cannot with good convince recommend it. It's Bizarre extended HTTP protocol that should not exist.
- mschuster91 10y agoOut of curiosity: why did you try to write your own WebDAV server? Apache ships a pretty much works-OOTB implementation - the only thing I never managed to get working was to assign uploaded files the UID/GID of the user who authenticated via HTTP auth to an LDAP server.
- donatj 10y agoMore specifically a CalDAV server which is a bizarre extension of WebDAV that shouldn't exist. We wanted one to connect to our internal identity server. That project was abandoned.
- carussell 10y agoRelated: "I Hope WebDAV Dies"[1] 1. https://news.ycombinator.com/item?id=10213657 https://news.ycombinator.com/item?id=10213657
- eikenberry 10y agoLinux has great webdav support with the fuse filesystem, davfs. I used it with my box.net account and it worked fine. http://savannah.nongnu.org/projects/davfs2 http://savannah.nongnu.org/projects/davfs2
- astrodust 10y agoExactly. You can offload HTTP to anyone, an HTTP proxy is absolutely trivial to set up, and an HTTP cache is even easier. FTP is not nearly as trivial, plus it's a stupid, broken protocol that deserves to die. The whole thing is a giant bag of hurt.
- mschuster91 10y ago> FTP is not nearly as trivial, plus it's a stupid, broken protocol that deserves to die. I agree with you, but FTP has one very valid use case left: easy file sharing, especially for shared hosting. FTP clients are native to every popular OS, from Android to Windows (only exceptions I know are Win Mobile and iOS), and there's a lot of ecosystem built around FTP. There is SCP and SFTP but they don't really have any kind of widespread usage in the non-professional world.
- astrodust 10y ago> [Has] one very valid use case left: easy file sharing, especially for shared hosting. Nope. Nope. Nope. Not easy. Not secure. Not user friendly. Not anything good. Have an iPhone and need to FTP something? Don't have installation rights on your Windows workstation and need to FTP something? Unpleasant if not confusing as all hell. Dropbox or a Dropbox-like program is significantly easier to get people on board with. Any "ecosystem" built around FTP is rotten to the core. Blow it up and get rid of it as soon as you can. Some vendors insist on using FTP because reasons, but those reasons are always laziness. I can't be the only one that would prefer they use ssh/scp/rsync with actual keys so I can be certain the entity uploading a file is actually them and not some random dude who sniffed the plain-text password off the wire.
- Spooky23 10y agoHuh? Windows explorer has FTP built into the shell. As does every other desktop OS. If you care about file integrity, you want to out of band to verify the signatures on the binary anyway. Dropbox is blocked in most commercial networks and offers no assurance of anything.
- mato 10y ago> It's about cost, too. HTTP can be cached very efficiently, but FTP not at all. It's ironic that you mention cost and caching but lot of services used for software distribution of one kind of another (e.g. Github releases) are following the "HTTPS everywhere" mantra and HTTPS can't be cached anywhere other than at the client.
- calpaterson 10y agoGithub probably trust their CDN with their TLS key. For Debian, packages are secured by PGP in combination with checksums so it's not relevant for them. The debian repos are often HTTP only.
- mschuster91 10y ago> and HTTPS can't be cached anywhere other than at the client. No. Nexus for example can certainly cache apt, as well as Squid can do if you provision it with a certificate that's trusted by the client. Also, Cloudflare supports HTTPS caching if you supply them with the certificate, and if you pay them enough and host some special server that handles the initial crypto handshake you don't even have to hand over your cert/privkey to them (e.g. required by law for banks, healthcare stuff etc)
- mato 10y agoTo clarify; what I meant is that HTTPS can't be cached by third parties. If I want to run a local cache of anything served over HTTP it's as easy as spinning up a Squid instance. With resources served over HTTPS I can't do that.
- dragonwriter 10y ago> Public FTP servers where where I downloaded most of the software for my computers, back in the 90s. There's nothing really like it anymore Modulo UI details, the common download-only public side of public FTP servers is a pretty similar experience to a pretty barebones file download web site. Anonymous file download web sites are, to put it mildly, not rare.
- johannes1234321 10y ago... until you want to offer directory structure a with multiple files
- throwanem 10y agoSo, configure your HTTP daemon to serve directory indexes, and point it at the root of whatever tree you want to serve.
- niftich 10y agoIn the "transition" from FTP to HTTP, the level of abstraction in popular use has shifted out of the protocol and into resources (mime-types) [1], rel types [2], server logic [3][4], and client logic [5]. In the past, I've said that this extensible nature of HTTP+HTML is what made them so successful [6], but once specialized protocols began to falter, tunneling other semantics over HTTP became not just a niceity, but also a necessity (for a diverse set of reasons, like being blocked at a middlebox, being accessible from an the browser where most people spend their time, etc). [1] http://www.iana.org/assignments/media-types/ http://www.iana.org/assignments/media-types/ [2] https://www.iana.org/assignments/link-relations/ https://www.iana.org/assignments/link-relations/ [3] https://wiki.apache.org/httpd/DirectoryListings https://wiki.apache.org/httpd/DirectoryListings [4] http://nginx.org/en/docs/http/ngx_http_autoindex_module.html http://nginx.org/en/docs/http/ngx_http_autoindex_module.html [5] http://stackoverflow.com/a/28380690 http://stackoverflow.com/a/28380690 [6] https://news.ycombinator.com/item?id=12440783 https://news.ycombinator.com/item?id=12440783
- copperx 10y agoApache works better for this than FTP. I use it all the time: just configure it to serve indexes. Apache lets you configure the index to include CSS, fancy icons, custom sorting, and other stuff. All over HTTPS. What's not to like?
- icebraining 10y agoThere's Bittorrent. It's anonymous and not web based.
- mxvzr 10y agoand it does checksum for you. edit: thinking about it not sure I agree with the anonymous part considering the swarm can be monitored. the access log is essentially publicly distributed.
- dTal 10y agoNot quite; the "currently accessing" list is public. While it is of course possible to make an access log from this with continuous monitoring, it's not possible to arbitrarily query historical data.
- crypt1d 10y agoNeither is FTP really, the user's IP is still logged somewhere, you just use a common user (anonymous) with everyone else. The modern name of such a feature would probably be something like 'No registration required'. It goes to show how much the meaning of the word 'anonymous' changed over the last 30 years.
- ryukafalz 10y ago>There's nothing really like it anymore - you can't have anonymous sftp. Strictly speaking there's nothing stopping someone from writing an anonymous sftp server that lets anyone log in as a 'guest' user or similar - it's just that nobody has (as far as I'm aware).
- derefr 10y ago"Unauthenticated SSH" is basically what the git:// protocol is. I wonder if you could use git-daemon(1) to serve things other than git repos? Or you could just convert whatever you want to serve into a git repo, I guess.
- toomuchtodo 10y agoPlease use IPFS instead.
- derefr 10y agoIPFS requires a stateful thick client with a bunch of index data, no? Would it be efficient to, say, build a Debian installer CD that goes out and downloads packages from an IPFS mirror? Because that's the kind of use-case anonymous FTP is for.
- toomuchtodo 10y agoMany many years ago I was on the team that managed the compute cluster for the CMS detector at the LHC (Fermilab Tier-1). When we would perform a rolling reinstall of the entire worker cluster (~5500 1U pizza box servers), we would use a custom installer that would utilize Bittorrent to retrieve the necessary RPMs (Scientific Linux) instead of HTTP; the more workers reinstalling at once, the faster each worker would reinstall (I hand wave away the complexities of job management for this discussion). I'm not super familiar with IPFS (I've only played with it a bit to see if I could use it to backup the Internet Archive in a distributed manner), but I'm fairly confident based on my limited trials that yes, you could build a Debian installer CD to fetch the required packages from an IPFS mirror. No need to even have the file index locally. You simply need a known source of the file index to retrieve, and the ability to retrieve it securely.
- zkms 10y ago> But perhaps we don't care anymore. The web is gradually consuming all that came before it. It is partly the web/HTTP eating everything but also that FTP is legitimately a bad protocol and is less tolerant of horrid shit going on in layers below it (like NAT) than HTTP is.
- zeta0134 10y agoI think my favorite "feature" of the FTP protocol has to be ASCII mangling, wherein the FTP server tries to mess around with line endings and text encoding mid-transfer. It's so bad that vsftpd, one of the better FTP servers for Linux systems, pretends to support it but silently refuses to perform the translation. http://sdocs.readthedocs.io/en/master/sdocs/ftpserver/vsftpd/vsftpd.html http://sdocs.readthedocs.io/en/master/sdocs/ftpserver/vsftpd...
- kalleboo 10y agoI wrote a custom FTP server once (it was database-backed instead of filesystem-backed - e.g. you could do searches by creating a directory in the Search directory) and I added in insulting error messages if a client tried to exercise one of the more antiquated features of the spec (e.g. EBCDIC mode)
- vhodges 10y agolftp (https://lftp.yar.ru/ https://lftp.yar.ru/) works over http (and more) and works just the way you would expect it to ie: lftp http://ftp.debian.org/debian/ http://ftp.debian.org/debian/
- Da_Blitz 10y agoi wrote a simple sftp server awhile back that advertises no auth methods, plain ssh tools log right in without prompting for a password code; https://pypi.python.org/pypi/noauthsftp https://pypi.python.org/pypi/noauthsftp handy for moving files around a network
- throwaway76543 10y ago> There's nothing really like it anymore - you can't have anonymous sftp Nonsense. http is exactly like anonymous ftp and it does a much better job of it. Pretty much every anonymous ftp site started also serving their files via http decades ago -- which is why ftp is no longer needed. Case in point: Debian makes all these files available over http. This isn't going away. ftp://ftp.debian.org/debian/ http://ftp.debian.org/debian/ http://ftp.debian.org/debian/
- jhbadger 10y agoIt really isn't as convenient if you have to download lots of files at one time though. FTP has mget. That's probably why FTP lives on for scientific data (NCBI, ENSEMBL, etc). Yes, you could use some tool like wget or curl to spider through a bunch of http links, but that's more work.
- Symbiote 10y agoIt would make more sense to offer rsync (unauthenticated), to ensure integrity of what's transferred. rsync -r rsync://... ./ will retrieve everything in a directory.
- mgbmtl 10y agoI suppose you can sometimes do `mget x.csv` on FTP (if the client supports it?), but with wget you can do: wget -r -A 'x.csv' https://example.org/ https://example.org/ (where 'x' is an asterisk, but HN's formatting eats it) More work, in the sense that it's more command line options to remember, I agree, but otherwise it's easier to integrate in scripts and much more flexible than mget. (I don't miss FTP for the sysadmin side of maintaining those servers.)
- deleted 10y ago[deleted]
- justinsaccount 10y ago> FTP has mget Not quite, ftp CLIENTS have mget. The ftp protocol has absolutely no awareness of mget. In fact, ftp is terrible at downloading more than one file at a time because it has no concept of pipelining and keepalive, both things that http supports. With a nice multi protocol client like lftp, http directory indexes work just like an ftp server: $ lftp http://http.debian.net/debian/ cd: received redirection to `http://cdn-fastly.deb.debian.org/debian/' cd ok, cwd=/debian lftp cdn-fastly.deb.debian.org:/debian> ls drwxr-xr-x -- / -rw-r--r-- 1.0K 2017-01-14 10:44 README -rw-r--r-- 1.3K 2010-06-26 09:52 README.CD-manufacture -rw-r--r-- 2.5K 2017-01-14 10:44 README.html -rw-r--r-- 291 2017-03-04 20:08 README.mirrors.html -rw-r--r-- 86 2017-03-04 20:08 README.mirrors.txt ..[snip].. lftp cdn-fastly.deb.debian.org:/debian> mget README* 5315 bytes transferred Total 5 files transferred lftp cdn-fastly.deb.debian.org:/debian>
- walrus01 10y ago> there's nothing really like it anymore a public facing httpd that uses the default apache2 directory index can be configured to , of course, allow anonymous access and with a log level that is neither more or less detailed than an anonymous ftpd circa 1999.
- smilbandit 10y agoFtp is an old protocol, it was good for it's time but http is just better now. Even though you can't have anonymous sftp you can have anonymous ftps.
- jwilk 10y agokernel.org is shutting down their FTP servers, too: https://www.kernel.org/shutting-down-ftp-services.html https://www.kernel.org/shutting-down-ftp-services.html
- unethical_ban 10y agoAlmost the exact same wording was used in the deb announcement. And it's all true. I'm a nostalgic kind of guy, but FTP is terrible through firewalls and nowadays, bittorrent or http are just as well.
- suls 10y agoWhat has changed in HTTP over the past 10, 20 years? Was there ever a clear benefit of using FTP instead of HTTP? Just wondering why suddenly everyone is in agreement FTP should go away ..
- pvg 10y agoUniversal adoption significantly less protocol cruft.
- acdha 10y agoFTP was older and had more widespread client support in the 90s, so you were more likely to have at least a basic client preinstalled. That's obviously moot by now since just about everything ships with a web browser and/or something like curl. FTP has a standard way to list directories. Using HTTP that way would require you to either have a well-known index file or parse HTML looking for links which don't return text/html responses. The downside is that FTP still has issues with firewalls – I had to troubleshoot that earlier this month, actually – and is another service to maintain if you are already running an HTTP server. In the case of either single-file downloads or something like a Linux package manager, the URLs are well known so directory listings are irrelevant. HTTP has a number of good options for CDNs and caching, so if you care about performance or reliability that's a turn-key service. In the cases where directory listings were more valuable, people usually wanted a richer UI than just a file listing, too, and there are tons of options for that in the web world.
- orionblastar 10y agoIn the old days we had Windows and wanted to install Netscape or download Linux we used anonymous ftp servers. Now Windows has a web browser installed that we can use to download anything we want. Anonymous ftp servers are outdated now. Like Gopher was when web browsers teplaced them.
- dexwiz 10y agoGopher had a lot going for it over the early web. But its death knell was University of Minnesota wanting to charge a licensing fee for it.
- VLM 10y agoftpmail. Now that is outdated. I vaguely remember around 1990 you'd send an email to some peculiar address and in more than an hour but less than a day, the service would execute the commands you gave it and return ... perhaps uuencoded files via email? Some encoding? Text just appeared as text. I think I got new phrack issues that way. Or some other zine. I also obtained some software this way. Some of it was even FOSS/legal. It was a way of working around various quota or security limitations. You'd get your file just as well as FTP, merely slowly. There were of course limitations on the ftp-mail service, you couldn't fetch an entire OS distro this way. Unbelievably, you can still do this today http://www.nws.noaa.gov/tg/ftpmail_using.php http://www.nws.noaa.gov/tg/ftpmail_using.php The docs on that page do bring back some memories. Obviously real ftpmail service didn't have a "open" line it was more like "open anonymous@simtel20.something.something.something.mil" thats what we did for a good time during the first Bush administration with our 2400 baud modems, LOL.
- deleted 10y ago[deleted]
- gtirloni 10y agoGood riddance to FTP. I've wasted enough time troubleshooting it in my career.
- raamdev 10y agoThe only reason I ever remember preferring FTP over HTTP in the 1990s was because FTP usually meant that I could resume large downloads if my connection dropped mid-download. That was a big deal when it took several hours to download something. That benefit largely disappeared for me as broadband got faster and the connections became more reliable.
- justinsaccount 10y agoAlso disappeared when http 1.1 added range requests around 1999
- hobarrera 10y agoFTP has been obsolete since 1999 thanks to ssh/sftp. It should have died along with RSH, but people insisted on keeping it alive.
- jwilk 10y agoHTTP/1.1 was published in 1997: https://tools.ietf.org/html/rfc2068 https://tools.ietf.org/html/rfc2068 But it probably took a few years to be universally adopted.
- tacticus 10y ago
- sitepodmatt 10y agoEcho the good riddance comment. Psst... Someone tell CPanel Inc and all the millions of horrid cPanel shared hosting providers which still promotes this nonsense as the default way to manage files. It is somewhat sad when you meet a new young fresh developer where deployment = filezilla.
- corey_moncure 10y agoHow do you like to manage your files and your deployments?
- shakna 10y agogit, mercurial, fossil. In short, version management. Usually with a good helping of CI so bad things don't go live.
- deleted 10y ago[deleted]
- throwaway2048 10y agofilezilla supports SFTP just fine (nothing to do with FTP)
- kyriakos 10y agoCpanel supports sftp too. Ftp it's just one option and can be disabled.
- TorKlingberg 10y agoIt also comes with bundled malware, so be careful.
- jrimbault 10y agoNot entirely true. Sourceforge bundles things with their installers. https://news.ycombinator.com/item?id=8849950 https://news.ycombinator.com/item?id=8849950 But you could always get a normal installer even then, and I think (don't quote me on that, I never really used SF) Sourceforge stopped doing that after their acquisition ?
- bradfitz 10y agoThanks, Russ! https://research.swtch.com/glob https://research.swtch.com/glob > but if you have an anonymous FTP server accepting glob patterns, there are two more fundamental questions to ask: Do you really need to run an anonymous FTP server anymore?
- jwilk 10y agoHN discussion: https://news.ycombinator.com/item?id=14184528 https://news.ycombinator.com/item?id=14184528
- z3t4 10y agoThere's really nothing stopping FTP from working on CDN's and proxies. For public distribution Bit-torrent is far superior though. I still think SFTP/FTP/FTPS is the best way to upload files, is there any better free alternatives !?
- eleitl 10y agoBut not offering https:// https:// services, huh?
- ilamont 10y agoThere are a fair number of businesses in old industries that "went digital" 10-20 years ago still have the same tech running, including FTP. I work with a company that requires uploading files via FTP into a co-mingled directory (everyone uses the same password). Company is a major player in my industry, too. I would be curious if some government services were still running FTP, judging by the number of Y2K-era government websites I run into.
- reacweb 10y agoyesterday, I have received a link to download some tutorial from china: http://pan.baidu.com/s/1sl993lV http://pan.baidu.com/s/1sl993lV. I have performed the download using firefox, but the connection is so bad, there are many failures. I have tried with curl that fails also. A real nightmare. All these problems would not exist if it was a simple ftp link: filezilla is perfectly optimized to work around shitty connections. And today I learn that ftp is falling in oblivion. What a sad time.
- inimino 10y agoCurl is perfectly capable of resuming a download over HTTP unless the server is doing something stupid. Blame Baidu if you like, but HTTP is not the problem here and there is no benefit to FTP.
- deavmi 10y agoKill FTP with fire.