5 ms·

The problem is the stable of the pointer. From a paper of mine: "The code and simulation results are available online at https://github.com/mylabname/ https://

by Fomite 8y ago

The problem is the stable of the pointer. From a paper of mine: "The code and simulation results are available online at https://github.com/mylabname/ https://github.com/mylabname/ project." That's in print. In another paper, I expressly cite a GitHub repository as the source of the data used in the analysis. Pointing to data in papers is the way most of the scientists I know use GitHub - because it's relatively stable, not tied to an institutional account, and relatively pain free.

IanCal 8y ago
[disclaimer work for related company - Digital Science] It's really worth getting those things into a system that will give you a doi and participate in an archive so that content should never be lost. Figshare is one, zenodo another (I think but can't find on a phone that the data is archived). A doi, if versioned, can ensure people see the actual version you used. It can be pointed somewhere else if the provider goes down, and should be archived to ensure its not lost. URLs are just not stable over time.
- Fomite 8y ago
  I've been consistently unimpressed with the offerings in that space.
  - IanCal 8y ago
    Curious what's missing, iirc you can set figshare to automatically pull from GitHub on a release. Seems pretty low requirements for free long term archiving and fitting into the common workflows with DOIs
    - Fomite 8y ago
      I can't quite put my finger on why I don't care for figshare - it just never seems to quite grip.
ulrikrasmussen 8y ago
Well, it still has a higher chance of being alive than other common alternatives. From my experience, any url put in a CS paper has a very high risk of being dead just a few years after publication. It seems that once the research grant has run out and the researchers have moved on, noone takes on the responsibility of ensuring that the web page is kept alive. Domains and hosting plans are not renewed; universities reorganize their departments and don't map the old URLs to the new ones; or policies change, and personal department hosted web pages for researchers are closed. The solution has to be some kind of independent library service for researchers, like arxiv for data/source code, which is guaranteed to be kept alive.
- IanCal 8y ago
  [disclaimer, work for Digital Science] There are various services for this. Figshare and zenodo are two major ones where you get dois. Figshare is archived with CLOCKSS so if it goes down all public content can be maintained and dois redirected. I think zenodo may be the same.
randomsearch 8y ago
In practice, the pointers are the authors. In Computer Science, it’s a mess that works fine in practice. I go to GitHub - if it’s not there, then I google the authors or the name of the project. And I don’t know what I do if I don’t find any of them or their projects, because I’ve never encountered that scenario. I’ve never seen a paper more than say 10 years old and thought “god I wish I had their code/data” because CS moves at a rapid speed. Important stuff is preserved because other people build on it. For example, it becomes the basis of an open source project. Whilst from a rigorous and idealistic point of view, we want central long term data storage, in practice right now the real problem is that _many scientists do not make their data and code available in any form at all_ rather than worrying about pointers decaying.
- DoofusOfDeath 8y ago
  I have: papers on alias analysis for compilers. I've had a lot of difficulty filling in the gaps in the old-but-important papers I've read.
dmos62 8y ago
It should not be complicated to move the repository somewhere where a search engine query with "mylabname/project" would find it. As a side note, it would be nice to see scientists stop using git for dataset management. There are alternatives that are considerably better adapted to managing data.
- Fomite 8y ago
  The thing with git (and GitHub by extension) is that it is "good enough" for many use cases, and relatively painless to get up and running.
rjsw 8y ago
What is wrong with an institutional account ? We used university FTP servers for many years to host software created by research projects at that institution. I work on an international standard, we are moving our repository in a couple of stages from being hosted at Sourceforge to being hosted directly at iso.org. Reference data in this standard will also be Maybe the research funding agencies should start providing their own hosting with the base URI derived from the project name.
- Fomite 8y ago
  Because in the past four years I've been at three different institutions, and those accounts go away when I leave.
  - rjsw 8y ago
    I presume that the departments or project groups you worked in are still there.
    - IanCal 8y ago
      These very regularly change.
    - Fomite 8y ago
      Some of the labs are - some of them have changed their names, or the PIs have moved on. But two of those papers were with my group, which has moved.
pdkl95 8y ago
The problem is not any particular hosting service such as GitHub, it's that the pointer in your paper ss relying on a single point of failure. There was a bit of drama several years ago when Megaupload was seized and shut down; various small/free projects lost access to the only copy of some of their files. Like your paper, important documents had evolved in forums, which linked to the file hosting service for files that could not be uploaded to the forum. A few projects were the canonical documentation for something that the original author had abandoned, the first result in Google couldn't be updated creating the same pointer problem as your reference in a paper. At the time, a lot of people talked about finding a "replacement file hosting service" in the same way people currently talk about finding a replacement for GitHub. Moving to a different service is still a single point of failure. Instead, when you want to preserve access to data in the long term, you need to assume any single service might fail and build in redundancy. Instead of saying, "[things] are available online at [URL]", you should include in the paper something like: The code and simulation results are available as an archive named: foo_project-2018-06-16.zip The file has the following checksums: MD5: 1271ed5ef305aadabc605b1609e24c52 SHA1: ab69db8315af7de6e673a6ddf128d415157a7c3f (...more...) The file was originally hosted at: $GITHUB_URL $GITLAB_URL ${OTHER_HOSTING_SERVICE_URLS[@]} $INTERNET_ARCHIVE_URL $AUTHOR_UNIVERSITY_URL $COLLAB_UNIVERSITY_URL With tools like git (or rsync, etc), making multiple copies of a project is very easy. Redundancy protects against some risks, but including checksums (and any other relevant metadata) makes content addressable searching possible. Even if all of the URLs in your paper eventually become defunct, someone reading the paper in the future may be able to find your data by searching for the file's hash. The hosting service isn't the (primary) problem; the paper needs to include a pointer that is more robust than a single reference to a single path on a single server.
- lou1306 8y ago
  Nice idea, but more robust checksums would be better (especially for archives containing code).
- Bedon292 8y ago
  Unfortunately most papers (that I have been involved with at least) have very specific size requirements and limits. Spending that much space on sharing the files is not something that can happen. Anything more than "[things] are available online at [URL]" is just going to take up too much space. (Although, from now on I might try to include two URLs) Besides, if / when GitHub ever shuts down, you know people will end up going through and cloning every repo on the system and archiving them at some similar url. So everyone can just visit it at notgithub.com/user/repo instead. Now, I agree that something like that would be awesome. I don't think its that easy to do. Its just not practical. GitHub / Lab are easy you just toggle a switch to make a repo public after the paper has been published, but other services may not be as easy. Especially trying to set up university urls, and keeping them private until after publication. For tech savvy people it might be fine, but not for all. Maybe it would be better if universities either hosted their own git repos, or even banded together and hosted something as a educational service? Rather than a for profit entity being where it is stored. The contact info for any author involved with the paper is going to be right there at the beginning of the paper. So IF something were to ever happen to the one link, contact is not going to be hard. Yes, there are risks, but it still gets the job done for the time being.
mseebach 8y ago
> That's in print. In olden times there'd be an address you could write to for a copy of the code on tape. It's an old problem without an obvious solution.
kuon 8y ago
This is a bit unrelated, but I think it would be cool to have a decentralized way of making persistent URLs.
- wolfgang42 8y ago
  The one that seems to be gaining the most traction is IPFS[1]. /ipfs/ URLs are content-addressed (like git objects). As long as someone has a copy of the data somewhere on the IPFS network it will continue to be available, and if you manage to get a copy of the content from somewhere else you can try hashing it and make sure it matches. (And if it does you can `ipfs add` it and it will become available again, even if nobody who had an original copy was still connected to the network.) [1]: https://ipfs.io/ https://ipfs.io/
  - kuon 8y ago
    I should have added "human readable short persistent url". Like foo.com/xy/github where xy is an identifier as short as possible to avoid collision. And with ipfs you are dependent on the ipfs own domain. To remove domain dependency it would have to be something not based on current DNS system.
    - diggan 8y ago
      > And with ipfs you are dependent on the ipfs own domain No, you might have gotten something wrong here. We (I work at Protocol Labs, I'm part of the development team working on IPFS) are hosting a public gateway you _can_ use but in no way have to use. You can either use your own local gateway, another gateway via IP or accessing the other peers gateways if that's accessible. Nothing is tied to the "ipfs.io" domain.
      - kuon 8y ago
        Yes I understand how ipfs works. But I mean, now we don't have the ability to print an URL that is not dependent on a domain name ipfs or any other. That's why DNS would require an alternative for true persistent and decentralized identifiers that are human readable.

by Fomite 8y ago

IanCal 8y ago

[disclaimer work for related company - Digital Science] It's really worth getting those things into a system that will give you a doi and participate in an archive so that content should never be lost. Figshare is one, zenodo another (I think but can't find on a phone that the data is archived). A doi, if versioned, can ensure people see the actual version you used. It can be pointed somewhere else if the provider goes down, and should be archived to ensure its not lost. URLs are just not stable over time.

Fomite 8y ago
I've been consistently unimpressed with the offerings in that space.
- IanCal 8y ago
  Curious what's missing, iirc you can set figshare to automatically pull from GitHub on a release. Seems pretty low requirements for free long term archiving and fitting into the common workflows with DOIs
  - Fomite 8y ago
    I can't quite put my finger on why I don't care for figshare - it just never seems to quite grip.

ulrikrasmussen 8y ago

Well, it still has a higher chance of being alive than other common alternatives. From my experience, any url put in a CS paper has a very high risk of being dead just a few years after publication. It seems that once the research grant has run out and the researchers have moved on, noone takes on the responsibility of ensuring that the web page is kept alive. Domains and hosting plans are not renewed; universities reorganize their departments and don't map the old URLs to the new ones; or policies change, and personal department hosted web pages for researchers are closed. The solution has to be some kind of independent library service for researchers, like arxiv for data/source code, which is guaranteed to be kept alive.

IanCal 8y ago
[disclaimer, work for Digital Science] There are various services for this. Figshare and zenodo are two major ones where you get dois. Figshare is archived with CLOCKSS so if it goes down all public content can be maintained and dois redirected. I think zenodo may be the same.

randomsearch 8y ago

In practice, the pointers are the authors. In Computer Science, it’s a mess that works fine in practice. I go to GitHub - if it’s not there, then I google the authors or the name of the project. And I don’t know what I do if I don’t find any of them or their projects, because I’ve never encountered that scenario. I’ve never seen a paper more than say 10 years old and thought “god I wish I had their code/data” because CS moves at a rapid speed. Important stuff is preserved because other people build on it. For example, it becomes the basis of an open source project. Whilst from a rigorous and idealistic point of view, we want central long term data storage, in practice right now the real problem is that _many scientists do not make their data and code available in any form at all_ rather than worrying about pointers decaying.

DoofusOfDeath 8y ago
I have: papers on alias analysis for compilers. I've had a lot of difficulty filling in the gaps in the old-but-important papers I've read.

dmos62 8y ago

It should not be complicated to move the repository somewhere where a search engine query with "mylabname/project" would find it. As a side note, it would be nice to see scientists stop using git for dataset management. There are alternatives that are considerably better adapted to managing data.

Fomite 8y ago
The thing with git (and GitHub by extension) is that it is "good enough" for many use cases, and relatively painless to get up and running.

rjsw 8y ago

What is wrong with an institutional account ? We used university FTP servers for many years to host software created by research projects at that institution. I work on an international standard, we are moving our repository in a couple of stages from being hosted at Sourceforge to being hosted directly at iso.org. Reference data in this standard will also be Maybe the research funding agencies should start providing their own hosting with the base URI derived from the project name.

Fomite 8y ago
Because in the past four years I've been at three different institutions, and those accounts go away when I leave.
- rjsw 8y ago
  I presume that the departments or project groups you worked in are still there.
  - IanCal 8y ago
    These very regularly change.
  - Fomite 8y ago
    Some of the labs are - some of them have changed their names, or the PIs have moved on. But two of those papers were with my group, which has moved.

pdkl95 8y ago

The problem is not any particular hosting service such as GitHub, it's that the pointer in your paper ss relying on a single point of failure. There was a bit of drama several years ago when Megaupload was seized and shut down; various small/free projects lost access to the only copy of some of their files. Like your paper, important documents had evolved in forums, which linked to the file hosting service for files that could not be uploaded to the forum. A few projects were the canonical documentation for something that the original author had abandoned, the first result in Google couldn't be updated creating the same pointer problem as your reference in a paper. At the time, a lot of people talked about finding a "replacement file hosting service" in the same way people currently talk about finding a replacement for GitHub. Moving to a different service is still a single point of failure. Instead, when you want to preserve access to data in the long term, you need to assume any single service might fail and build in redundancy. Instead of saying, "[things] are available online at [URL]", you should include in the paper something like: The code and simulation results are available as an archive named: foo_project-2018-06-16.zip The file has the following checksums: MD5: 1271ed5ef305aadabc605b1609e24c52 SHA1: ab69db8315af7de6e673a6ddf128d415157a7c3f (...more...) The file was originally hosted at: $GITHUB_URL $GITLAB_URL ${OTHER_HOSTING_SERVICE_URLS[@]} $INTERNET_ARCHIVE_URL $AUTHOR_UNIVERSITY_URL $COLLAB_UNIVERSITY_URL With tools like git (or rsync, etc), making multiple copies of a project is very easy. Redundancy protects against some risks, but including checksums (and any other relevant metadata) makes content addressable searching possible. Even if all of the URLs in your paper eventually become defunct, someone reading the paper in the future may be able to find your data by searching for the file's hash. The hosting service isn't the (primary) problem; the paper needs to include a pointer that is more robust than a single reference to a single path on a single server.

lou1306 8y ago
Nice idea, but more robust checksums would be better (especially for archives containing code).
Bedon292 8y ago
Unfortunately most papers (that I have been involved with at least) have very specific size requirements and limits. Spending that much space on sharing the files is not something that can happen. Anything more than "[things] are available online at [URL]" is just going to take up too much space. (Although, from now on I might try to include two URLs) Besides, if / when GitHub ever shuts down, you know people will end up going through and cloning every repo on the system and archiving them at some similar url. So everyone can just visit it at notgithub.com/user/repo instead. Now, I agree that something like that would be awesome. I don't think its that easy to do. Its just not practical. GitHub / Lab are easy you just toggle a switch to make a repo public after the paper has been published, but other services may not be as easy. Especially trying to set up university urls, and keeping them private until after publication. For tech savvy people it might be fine, but not for all. Maybe it would be better if universities either hosted their own git repos, or even banded together and hosted something as a educational service? Rather than a for profit entity being where it is stored. The contact info for any author involved with the paper is going to be right there at the beginning of the paper. So IF something were to ever happen to the one link, contact is not going to be hard. Yes, there are risks, but it still gets the job done for the time being.

mseebach 8y ago

> That's in print. In olden times there'd be an address you could write to for a copy of the code on tape. It's an old problem without an obvious solution.

kuon 8y ago

This is a bit unrelated, but I think it would be cool to have a decentralized way of making persistent URLs.

wolfgang42 8y ago
The one that seems to be gaining the most traction is IPFS[1]. /ipfs/ URLs are content-addressed (like git objects). As long as someone has a copy of the data somewhere on the IPFS network it will continue to be available, and if you manage to get a copy of the content from somewhere else you can try hashing it and make sure it matches. (And if it does you can `ipfs add` it and it will become available again, even if nobody who had an original copy was still connected to the network.) [1]: https://ipfs.io/ https://ipfs.io/
- kuon 8y ago
  I should have added "human readable short persistent url". Like foo.com/xy/github where xy is an identifier as short as possible to avoid collision. And with ipfs you are dependent on the ipfs own domain. To remove domain dependency it would have to be something not based on current DNS system.
  - diggan 8y ago
    > And with ipfs you are dependent on the ipfs own domain No, you might have gotten something wrong here. We (I work at Protocol Labs, I'm part of the development team working on IPFS) are hosting a public gateway you _can_ use but in no way have to use. You can either use your own local gateway, another gateway via IP or accessing the other peers gateways if that's accessible. Nothing is tied to the "ipfs.io" domain.
    - kuon 8y ago
      Yes I understand how ipfs works. But I mean, now we don't have the ability to print an URL that is not dependent on a domain name ipfs or any other. That's why DNS would require an alternative for true persistent and decentralized identifiers that are human readable.