8 ms·

Great to see there's some resistance. What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the curren

Great to see there's some resistance. What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the current government. I'm assuming good intentions on the part of Harvard, but keeping this data online against the express will of the government is gonna cost (political) capital. And from what I can see, the archive is hosted by US entities on US-controlled servers on US soil? This is the same thing that's been bothering me with archive.org lately, by the way. I haven't found a good way to simply (for some reasonable definition definition of "simple") contribute 10 TiB or so of redundant storage on my (european) home server either. That kind of thing might (have to) serve to ensure tamper-resistance for that data, given the current political climate on both sides of the pond. Any pointers welcome.

EnnEmmEss 2y ago
If I remember correctly, Harvard has immunity to eminent domain under the Massachusetts constitution. Maybe it has a similar right which would make it immune to such attacks?
- black_puppydog 2y ago
  That may be so, but given what Trump and Musk have been up to, the situation of the courts, and how they blatantly don't give a f*k about what's constitutional or not, I wouldn't rely on this so-called "immunity".
  - 0xEF 2y ago
    I really hope I am wrong, but I'm planning on seeing some headlines about Musk shutting down Harvard next week over this. For anyone who still thinks the existing laws, constitutions and policies mean anything to this current regime, prepare to get some whiplash. They are proving that none of that matters if they simply ignore it and do what they want to do anyway.
    - shakna 2y ago
      It wouldn't be the first time that Musk has called for Harvard to be defunded. [0] [0] https://www.businessinsider.com/elon-musk-calling-for-harvard-defunding-2023-12 https://www.businessinsider.com/elon-musk-calling-for-harvar...
- lou1306 2y ago
  I beg you, please stop applying rule-of-law mindset against might-makes-right adversaries. It creates blind spots giving the illusion that the attack surface is way smaller than it actually is. Muskolites are taking on the SSN system without any Congressional oversight as we speak. The President is attacking ius soli which is a Constitutional right. If they decide that sending their sleuths to Cambridge MA to physically destroy this data is in their best interest, they will do so and handle the courts later. Just stop pretending they will play by the book.
  - squigz 2y ago
    How do you recommend engaging with the situation then? Throw out the book too?
    - maaaaattttt 2y ago
      Not OP but torrent would be my go to for a short term solution.
      - anonzzzies 2y ago
        Torrent would do it but 16TB is still not that cheap to just have running in your house nas (otherwise I would).
        lou1306 2y ago
        Of course this is more of a job for responsible universities/libraries/research centres around the world, rather than single individuals. I would be surprised if Harvard didn't already go through their contact list to ensure as many copies are being made as possible.
        bilekas 2y ago
        Torrent would definitely be a great approach, 16TB is a commitment, but there's no problem with partial seeding also. I would choose selective chunks which consist of completed files.
bjackman 2y ago
I think a fully distributed storage system must be the way here. There must be some IPFS type system where Harvard could say "we designated a set of data that we can add to as needed but only delete from with a critical mass of storage providers' consent, here are some instructions for you to add your spare capacity to become a storage provider".
- skeeter2020 2y ago
  This sounds an awful lot like Pied Piper!
- BorisMelnik 2y ago
  I was just thinking IPFS or possibly even some blockchain solution for this. Bram Cohen (the creator of bittorrent) also has a new project I think in the DePin sector. Sensitive information just can't be hosted on a centralized server anymore, it has to be distributed for the good of the project.
lisp2240 2y ago
Fourth paragraph
zombot 2y ago
> ...how they intend to secure this "vault" against the current government. Yup, I was about to ask whether Trump could still force them to delete what he doesn't like. Time will tell, I guess.
- LadyCailin 2y ago
  In general, no. By withholding federal funds, and with owning congress and scotus, yes.
  - bilekas 2y ago
    I'll admit I'm not privy to US educational system etc, but isn't Harvard a private university ? Why would they be receiving significant federal funds in the first place ?
    - matwood 2y ago
      Yeah, I'm not sure what Trump could really do here. Harvard gets < 1B in federal funds every year, mainly for research. Harvard also has a ~50B endowment. Short of sending in the military or confiscating Harvard's private funds, it seems like Harvard can wait him out. With that said, the current administration appears to have a war on education in general so who knows what ideas they could cook up.
      - olafalo 2y ago
        Quoting Trump directly, from Agenda 47: > Furthermore, I will direct the Department of Justice to pursue federal civil rights cases against schools that continue to engage in racial discrimination. And schools that persist in explicit unlawful discrimination under the guise of equity will not only have their endowment taxed, but through budget reconciliation, I will advance a measure to have them fined up to the entire amount of their endowment.
    - tristor 2y ago
      Research grants. That said, Harvard has the largest university endowment in the world, they'd be just fine operating at their current size off their endowment proceeds effectively forever.
milesrout 2y ago
>I'm assuming good intentions on the part of Harvard, but keeping this data online against the express will of the government is gonna cost (political) capital. Harvard like other liberal institutions has little to no political capital in a Republican white house in the first place. Why would it cost them any to host data that is in the public domain? This sort of silly overreaction is part of why people voted for Trump: it is genuinely funny to see people overreact to Trump because theyve been told by the legacy media that he is a "threat to democracy" over and over. Last time he was elected his government also removed a bunch of climate data from govt websites which was quickly mirrored by third parties. Nobody was taken away by the Gestapo then and there is no reason to think things will be any different this time.
- Juliate 2y ago
  > Nobody was taken away by the Gestapo Someone forgot the "kids in cages" episodes in 2018 when Trump was only testing the waters... EDIT: https://en.wikipedia.org/wiki/Trump_administration_family_separation_policy https://en.wikipedia.org/wiki/Trump_administration_family_se...
  - Juliate 2y ago
    Someone didn't like to be reminded of things. :)
    - Juliate 2y ago
      Wow, amazing. There _are_ people that do not want history to be documented and reminded... For the record, my previous comments being upvoted, then largely downvoted, for a matter of facts.
  - milesrout 2y ago
    https://www.aclu.org/news/smart-justice/president-obama-wants-continue-imprisoning-immigrant-families https://www.aclu.org/news/smart-justice/president-obama-want... Nothing to do with Trump. Also Trump's policy was completely sensible, and no different to what happens when adults are sent to prison anywhere else: you cant put their kids in prison with them obviously.
rswail 2y ago
From the announcement: "This work is made possible with support from the Filecoin Foundation for the Decentralized Web and the Rockefeller Brothers Fund." https://fil.org/ https://fil.org/
tlb 2y ago
It might not be ironclad, but the ease with which federal workers can fiddle with data they're hosting themselves vs. fiddling with data in Harvard's library is a pretty big difference. And if it ever came to demands for censorship, it wouldn't be Harvard Library's first rodeo.
Thorrez 2y ago
>how they intend to secure this "vault" against the current government Is there any risk of the government ordering them to take it down? That seems unlikely to me. The US has strong free speech protection, stronger than European free speech protection. >keeping this data online against the express will of the government is gonna cost (political) capital. Costing them political capital (aka the government is unhappy) is different from the government ordering them to take it down. Also, when you say "express will", are you saying the government has explicitly publicly stated that they don't like that Harvard is hosting this data?
- dmd 2y ago
  > The US has strong free speech protection The US literally just told (e.g., [0]) all scientists working for it they are not permitted to publish papers or speak at conferences or travel. What the US "has" is irrelevant when laws are being ignored. [0] https://www.science.org/content/blog-post/revised-and-extended-what-s-happening-inside-nih-and-nsf https://www.science.org/content/blog-post/revised-and-extend...
  - marcinzm 2y ago
    That's an employer telling employees what to do? Google can tell the same thing to it's employees and does. I don't see the relevance to the government telling other entities what to do.
    - dmd 2y ago
      The employer is the US federal government, in this case.
      - marcinzm 2y ago
        And employees signed contracts which allow the government to tell them how to do their jobs or fire them. Just like any other employer. Harvard did not.
lloeki 2y ago
> What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the current government. Maybe this? > In addition to the data collection, we are releasing open source software and documentation for replicating our work and creating similar repositories. With these tools, we aim not only to preserve knowledge ourselves but also to empower others to save and access the data that matters to them. https://github.com/harvard-lil/data-vault https://github.com/harvard-lil/data-vault And since the data lives here: https://source.coop/repositories/harvard-lil/gov-data/description https://source.coop/repositories/harvard-lil/gov-data/descri... Combined with this: > To download an individual dataset by name you can construct its URL, such as: > https://source.coop/harvard-lil/gov-data/collections/data_gov/fruit-and-vegetable-prices/v1.zip https://source.coop/harvard-lil/gov-data/collections/data_go... > https://source.coop/harvard-lil/gov-data/metadata/data_gov/fruit-and-vegetable-prices/v1.json https://source.coop/harvard-lil/gov-data/metadata/data_gov/f... > To download large numbers of files, we recommend the aws or rclone command line tools: > aws s3 cp s3://us-west-2.opendata.source.coop/harvard-lil/gov-data/collections/data_gov/<name>/v1.zip --no-sign-request So one could "easily" mirror the whole thing, making it distributed.
- beej71 2y ago
  And 16 TB isn't what it used to be (though it is a big transfer). I'm hopeful that it's already being mirrored in multiple places including overseas.
headcanon 2y ago
> how they intend to secure this "vault" against the current government Definitely a concern, if they want to harass Harvard and the other universities they could, but I don't think they'll bother. They know the data will be backed up, that's not the point. Taking it off of data.gov accomplishes two things: 1) Makes it look like they're doing something, playing to the base. Easy to do 2) Delegitmize any insights the data might have. "Sure you have 'data', is it official data? I don't see it on data.gov. How do we know its not fraudulent?" It makes it harder to use it to justify policy changes. It adds one more tool to the denial crowd.

EnnEmmEss 2y ago

If I remember correctly, Harvard has immunity to eminent domain under the Massachusetts constitution. Maybe it has a similar right which would make it immune to such attacks?

black_puppydog 2y ago
That may be so, but given what Trump and Musk have been up to, the situation of the courts, and how they blatantly don't give a f*k about what's constitutional or not, I wouldn't rely on this so-called "immunity".
- 0xEF 2y ago
  I really hope I am wrong, but I'm planning on seeing some headlines about Musk shutting down Harvard next week over this. For anyone who still thinks the existing laws, constitutions and policies mean anything to this current regime, prepare to get some whiplash. They are proving that none of that matters if they simply ignore it and do what they want to do anyway.
  - shakna 2y ago
    It wouldn't be the first time that Musk has called for Harvard to be defunded. [0] [0] https://www.businessinsider.com/elon-musk-calling-for-harvard-defunding-2023-12 https://www.businessinsider.com/elon-musk-calling-for-harvar...
lou1306 2y ago
I beg you, please stop applying rule-of-law mindset against might-makes-right adversaries. It creates blind spots giving the illusion that the attack surface is way smaller than it actually is. Muskolites are taking on the SSN system without any Congressional oversight as we speak. The President is attacking ius soli which is a Constitutional right. If they decide that sending their sleuths to Cambridge MA to physically destroy this data is in their best interest, they will do so and handle the courts later. Just stop pretending they will play by the book.
- squigz 2y ago
  How do you recommend engaging with the situation then? Throw out the book too?
  - maaaaattttt 2y ago
    Not OP but torrent would be my go to for a short term solution.
    - anonzzzies 2y ago
      Torrent would do it but 16TB is still not that cheap to just have running in your house nas (otherwise I would).
      - lou1306 2y ago
        Of course this is more of a job for responsible universities/libraries/research centres around the world, rather than single individuals. I would be surprised if Harvard didn't already go through their contact list to ensure as many copies are being made as possible.
      - bilekas 2y ago
        Torrent would definitely be a great approach, 16TB is a commitment, but there's no problem with partial seeding also. I would choose selective chunks which consist of completed files.

bjackman 2y ago

I think a fully distributed storage system must be the way here. There must be some IPFS type system where Harvard could say "we designated a set of data that we can add to as needed but only delete from with a critical mass of storage providers' consent, here are some instructions for you to add your spare capacity to become a storage provider".

skeeter2020 2y ago
This sounds an awful lot like Pied Piper!
BorisMelnik 2y ago
I was just thinking IPFS or possibly even some blockchain solution for this. Bram Cohen (the creator of bittorrent) also has a new project I think in the DePin sector. Sensitive information just can't be hosted on a centralized server anymore, it has to be distributed for the good of the project.

lisp2240 2y ago

Fourth paragraph

zombot 2y ago

> ...how they intend to secure this "vault" against the current government. Yup, I was about to ask whether Trump could still force them to delete what he doesn't like. Time will tell, I guess.

LadyCailin 2y ago
In general, no. By withholding federal funds, and with owning congress and scotus, yes.
- bilekas 2y ago
  I'll admit I'm not privy to US educational system etc, but isn't Harvard a private university ? Why would they be receiving significant federal funds in the first place ?
  - matwood 2y ago
    Yeah, I'm not sure what Trump could really do here. Harvard gets < 1B in federal funds every year, mainly for research. Harvard also has a ~50B endowment. Short of sending in the military or confiscating Harvard's private funds, it seems like Harvard can wait him out. With that said, the current administration appears to have a war on education in general so who knows what ideas they could cook up.
    - olafalo 2y ago
      Quoting Trump directly, from Agenda 47: > Furthermore, I will direct the Department of Justice to pursue federal civil rights cases against schools that continue to engage in racial discrimination. And schools that persist in explicit unlawful discrimination under the guise of equity will not only have their endowment taxed, but through budget reconciliation, I will advance a measure to have them fined up to the entire amount of their endowment.
  - tristor 2y ago
    Research grants. That said, Harvard has the largest university endowment in the world, they'd be just fine operating at their current size off their endowment proceeds effectively forever.

milesrout 2y ago

>I'm assuming good intentions on the part of Harvard, but keeping this data online against the express will of the government is gonna cost (political) capital. Harvard like other liberal institutions has little to no political capital in a Republican white house in the first place. Why would it cost them any to host data that is in the public domain? This sort of silly overreaction is part of why people voted for Trump: it is genuinely funny to see people overreact to Trump because theyve been told by the legacy media that he is a "threat to democracy" over and over. Last time he was elected his government also removed a bunch of climate data from govt websites which was quickly mirrored by third parties. Nobody was taken away by the Gestapo then and there is no reason to think things will be any different this time.

Juliate 2y ago
> Nobody was taken away by the Gestapo Someone forgot the "kids in cages" episodes in 2018 when Trump was only testing the waters... EDIT: https://en.wikipedia.org/wiki/Trump_administration_family_separation_policy https://en.wikipedia.org/wiki/Trump_administration_family_se...
- Juliate 2y ago
  Someone didn't like to be reminded of things. :)
  - Juliate 2y ago
    Wow, amazing. There _are_ people that do not want history to be documented and reminded... For the record, my previous comments being upvoted, then largely downvoted, for a matter of facts.
- milesrout 2y ago
  https://www.aclu.org/news/smart-justice/president-obama-wants-continue-imprisoning-immigrant-families https://www.aclu.org/news/smart-justice/president-obama-want... Nothing to do with Trump. Also Trump's policy was completely sensible, and no different to what happens when adults are sent to prison anywhere else: you cant put their kids in prison with them obviously.

rswail 2y ago

From the announcement: "This work is made possible with support from the Filecoin Foundation for the Decentralized Web and the Rockefeller Brothers Fund." https://fil.org/ https://fil.org/

tlb 2y ago

It might not be ironclad, but the ease with which federal workers can fiddle with data they're hosting themselves vs. fiddling with data in Harvard's library is a pretty big difference. And if it ever came to demands for censorship, it wouldn't be Harvard Library's first rodeo.

Thorrez 2y ago

>how they intend to secure this "vault" against the current government Is there any risk of the government ordering them to take it down? That seems unlikely to me. The US has strong free speech protection, stronger than European free speech protection. >keeping this data online against the express will of the government is gonna cost (political) capital. Costing them political capital (aka the government is unhappy) is different from the government ordering them to take it down. Also, when you say "express will", are you saying the government has explicitly publicly stated that they don't like that Harvard is hosting this data?

dmd 2y ago
> The US has strong free speech protection The US literally just told (e.g., [0]) all scientists working for it they are not permitted to publish papers or speak at conferences or travel. What the US "has" is irrelevant when laws are being ignored. [0] https://www.science.org/content/blog-post/revised-and-extended-what-s-happening-inside-nih-and-nsf https://www.science.org/content/blog-post/revised-and-extend...
- marcinzm 2y ago
  That's an employer telling employees what to do? Google can tell the same thing to it's employees and does. I don't see the relevance to the government telling other entities what to do.
  - dmd 2y ago
    The employer is the US federal government, in this case.
    - marcinzm 2y ago
      And employees signed contracts which allow the government to tell them how to do their jobs or fire them. Just like any other employer. Harvard did not.

lloeki 2y ago

> What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the current government. Maybe this? > In addition to the data collection, we are releasing open source software and documentation for replicating our work and creating similar repositories. With these tools, we aim not only to preserve knowledge ourselves but also to empower others to save and access the data that matters to them. https://github.com/harvard-lil/data-vault https://github.com/harvard-lil/data-vault And since the data lives here: https://source.coop/repositories/harvard-lil/gov-data/description https://source.coop/repositories/harvard-lil/gov-data/descri... Combined with this: > To download an individual dataset by name you can construct its URL, such as: > https://source.coop/harvard-lil/gov-data/collections/data_gov/fruit-and-vegetable-prices/v1.zip https://source.coop/harvard-lil/gov-data/collections/data_go... > https://source.coop/harvard-lil/gov-data/metadata/data_gov/fruit-and-vegetable-prices/v1.json https://source.coop/harvard-lil/gov-data/metadata/data_gov/f... > To download large numbers of files, we recommend the aws or rclone command line tools: > aws s3 cp s3://us-west-2.opendata.source.coop/harvard-lil/gov-data/collections/data_gov/<name>/v1.zip --no-sign-request So one could "easily" mirror the whole thing, making it distributed.

beej71 2y ago
And 16 TB isn't what it used to be (though it is a big transfer). I'm hopeful that it's already being mirrored in multiple places including overseas.

headcanon 2y ago

> how they intend to secure this "vault" against the current government Definitely a concern, if they want to harass Harvard and the other universities they could, but I don't think they'll bother. They know the data will be backed up, that's not the point. Taking it off of data.gov accomplishes two things: 1) Makes it look like they're doing something, playing to the base. Easy to do 2) Delegitmize any insights the data might have. "Sure you have 'data', is it official data? I don't see it on data.gov. How do we know its not fraudulent?" It makes it harder to use it to justify policy changes. It adds one more tool to the denial crowd.