3 ms·
What?! I had no idea this was going on. I still use ftp often - my yesterday's blog post includes an ftp URL, see http://www.dalkescientific.com/writings/diary/
by dalke 6y ago
What?! I had no idea this was going on. I still use ftp often - my yesterday's blog post includes an ftp URL, see http://www.dalkescientific.com/writings/diary/archive/2020/09/29/simple_knn_search.html.html http://www.dalkescientific.com/writings/diary/archive/2020/0... .
What's the replacement supposed to be? The article suggests one of "BitTorrent, SFTP, rsync, git, even modern variants of HTTP" - which means also changing the workflow, right?
For example, I download chemical data from PubChem (ftp://ftp.ncbi.nlm.nih.gov/pubchem/Compound/CURRENT-Full/SDF/ ). That location a directory containing about 300 files of the form Compound_084500001_085000000.sdf.gz where each file contains records with that range of ids. The number of files increases over time, the number of records in the file also changed(it used to be 250000 records per file), and the content of each file changes (eg, if a record is withdrawn).
I use ncftp to "mget *" the files as a bulk download. What's the non-ftp way to organize and download this sort of data? That is, to discover what files are available and download all of them?
Of course, ncftp won't disappear, but the PubChem distribution also includes, for example, documentation in ftp://ftp.ncbi.nlm.nih.gov/pubchem/data_spec/ . I use Firefox as graphical front-end to, well, browse the ftp site (ncftp would be a step back), and I expect I can use ftp-URLs to provide links to the relevant documentation.
I also provide links to just one file in the distribution, which I use for training and testing purposes. (As an example, see https://chemfp.readthedocs.io/en/latest/using-tools.html#generate-fingerprint-files-from-pubchem-sd-tags https://chemfp.readthedocs.io/en/latest/using-tools.html#gen... ). I expect people to be able to use the browser to download those files. Now I'll have to provide information about alternative ways to get the data?