3 ms·

Caveat: the last full Kiwix English Wikipedia archive was made in 2018. They could use some help with automating their build process if anyone here has the time

Caveat: the last full Kiwix English Wikipedia archive was made in 2018. They could use some help with automating their build process if anyone here has the time.

voltagex_ 7y ago
From a cursory glance at the site and source code, it's really hard to see who/what is involved with building an archive. There's automated builds set up for the Pi image itself.
- pronoiac 7y ago
  Agreed. I can see that other Wikipedia languages are crawled - https://wiki.kiwix.org/wiki/Content_in_all_languages https://wiki.kiwix.org/wiki/Content_in_all_languages shows dozens of updates this week - but the best leads I have involve poking around the openZIM Github org, https://github.com/openzim https://github.com/openzim . There might be a running "zimfarm" somewhere?
  - bshep 7y ago
    Looks like you are right, you may be able to join the farm to help ( i have not tested as I am away from my computer at the moment ) https://github.com/openzim/zimfarm/blob/master/worker/README.md https://github.com/openzim/zimfarm/blob/master/worker/README...
    - voltagex_ 7y ago
      Someone should ask drone.io or packet.net for an Epyc machine to do this.
  - jedieaston 7y ago
    You can build your own ZIMs from any MediaWiki instance using this tool: https://github.com/openzim/mwoffliner https://github.com/openzim/mwoffliner. Maybe it would be worth putting together another zimfarm that is constantly updating.
- Nemo_bis 7y ago
  The last time I checked, it was more a problem of lacking servers with sufficient resources: https://phabricator.wikimedia.org/T124960 https://phabricator.wikimedia.org/T124960 https://phabricator.wikimedia.org/T219078 https://phabricator.wikimedia.org/T219078 It sure doesn't harm if someone creates their own ZIM files and reports on their results (and/or shares the resulting files).

3 ms·

Caveat: the last full Kiwix English Wikipedia archive was made in 2018. They could use some help with automating their build process if anyone here has the time

by buildbuildbuild 7y ago

Caveat: the last full Kiwix English Wikipedia archive was made in 2018. They could use some help with automating their build process if anyone here has the time.

voltagex_ 7y ago
From a cursory glance at the site and source code, it's really hard to see who/what is involved with building an archive. There's automated builds set up for the Pi image itself.
- pronoiac 7y ago
  Agreed. I can see that other Wikipedia languages are crawled - https://wiki.kiwix.org/wiki/Content_in_all_languages https://wiki.kiwix.org/wiki/Content_in_all_languages shows dozens of updates this week - but the best leads I have involve poking around the openZIM Github org, https://github.com/openzim https://github.com/openzim . There might be a running "zimfarm" somewhere?
  - bshep 7y ago
    Looks like you are right, you may be able to join the farm to help ( i have not tested as I am away from my computer at the moment ) https://github.com/openzim/zimfarm/blob/master/worker/README.md https://github.com/openzim/zimfarm/blob/master/worker/README...
    - voltagex_ 7y ago
      Someone should ask drone.io or packet.net for an Epyc machine to do this.
  - jedieaston 7y ago
    You can build your own ZIMs from any MediaWiki instance using this tool: https://github.com/openzim/mwoffliner https://github.com/openzim/mwoffliner. Maybe it would be worth putting together another zimfarm that is constantly updating.
- Nemo_bis 7y ago
  The last time I checked, it was more a problem of lacking servers with sufficient resources: https://phabricator.wikimedia.org/T124960 https://phabricator.wikimedia.org/T124960 https://phabricator.wikimedia.org/T219078 https://phabricator.wikimedia.org/T219078 It sure doesn't harm if someone creates their own ZIM files and reports on their results (and/or shares the resulting files).