4 ms·

And this is at least the 4th or 5th time within 10 days that either the website is down or performance is badly degraded. So bad for a website that is basicall

And this is at least the 4th or 5th time within 10 days that either the website is down or performance is badly degraded. So bad for a website that is basically the go-to resource where IT professionals get their answers. So bad that the whole platform is closed-source, so we can't even spin multiple instances. The main resource used by developers and sysadmins around the world is locked and centralized, and when that central place goes down all the answers go down with it. We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them. And, most of all, we need alternatives that are open and decentralized, and we need them right now.

amanj41 4y ago
If you use brave browser it usually is able to point you to corresponding web archive pages when SO is down. Nice to have it directly integrated into the browser
Pengtuzi 4y ago
> we'll do the scraping for them Awesome! Let us know when we can start using your new open source service, it must be mirrored and need it right now.
knowhy 4y ago
I think stack exchange provides data dumps via archive.org https://archive.org/download/stackexchange https://archive.org/download/stackexchange
zufallsheld 4y ago
Stackoverflow's data is cc-by-sa 4.0 licensed and can be found here: https://archive.org/details/stackexchange https://archive.org/details/stackexchange So nothing is stopping anyone to spin-up multiple instances.
- mark-r 4y ago
  It's much harder to work with the data if you don't have the code. SO is not open source.
  - distantsounds 4y ago
    https://www.rickcarlino.com/2020/keeping-an-offline-copy-of-stackoverflow-with-kiwix.html https://www.rickcarlino.com/2020/keeping-an-offline-copy-of-... not so sure about that
    - mark-r 4y ago
      That's kind of cool, thanks. How long does it take to do a web scrape versus a DB dump?
    - noSyncCloud 4y ago
      Here's a working link to Kiwix since the link on the page 404s: https://www.kiwix.org/en/download/ https://www.kiwix.org/en/download/
  - MonkeyMalarky 4y ago
    IMO it would be more interesting to not use their code and build some kind of specialized search service. One that learns to rank results from user metrics and feedback?
    - he_is_legend 4y ago
      so like personalised google search, with the terrible results that that inevitably produces for code related content? i am confused though. how do you think SO currently works? Both questions and answers are already ranked via user metrics (individual upvotes and historical upvotes) and feedback (comments)?
- Wowfunhappy 4y ago
  StackExchange is by no means perfect, but the copyleft content license is a key reason I feel comfortable contributing to the platform. (Which I assume is why they do it.)
someguy5344523 4y ago
> We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them. I think that's already the case. Whenever I look up a technical issue, the first result is usually from SO; the next results are usually from websites that copied that first SO page
- rr888 4y ago
  You're lucky, for me recently its the other way around, spam copies first.
NateEag 4y ago
FWIW, the Codidact project has been plugging away at building an OSS replacement for SE: https://www.codidact.org/ https://www.codidact.org/
hddherman 4y ago
A read-only, self-hostable copy of StackOverflow (and its sister sites) is already doable with Kiwix + the content packages they publish. Content packages: https://wiki.kiwix.org/wiki/Content_in_all_languages https://wiki.kiwix.org/wiki/Content_in_all_languages Example of how a site looks like (in this case SuperUser), hosted on my server: https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/index.html https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/inde... And hosting it really isn't all that difficult: https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/ https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/
pugworthy 4y ago
I’m not trying to be funny but it’s interesting how much panic one can read in the parent comment. We all joke about the dependency on SO. It seems to actually be more true than perhaps we’d like to admit.
- 300bps 4y ago
  It’s unjustified panic. It’s freely downloadable and available everywhere.
deleted 4y ago
[deleted]

amanj41 4y ago

If you use brave browser it usually is able to point you to corresponding web archive pages when SO is down. Nice to have it directly integrated into the browser

Pengtuzi 4y ago

> we'll do the scraping for them Awesome! Let us know when we can start using your new open source service, it must be mirrored and need it right now.

knowhy 4y ago

I think stack exchange provides data dumps via archive.org https://archive.org/download/stackexchange https://archive.org/download/stackexchange

zufallsheld 4y ago

Stackoverflow's data is cc-by-sa 4.0 licensed and can be found here: https://archive.org/details/stackexchange https://archive.org/details/stackexchange So nothing is stopping anyone to spin-up multiple instances.

mark-r 4y ago
It's much harder to work with the data if you don't have the code. SO is not open source.
- distantsounds 4y ago
  https://www.rickcarlino.com/2020/keeping-an-offline-copy-of-stackoverflow-with-kiwix.html https://www.rickcarlino.com/2020/keeping-an-offline-copy-of-... not so sure about that
  - mark-r 4y ago
    That's kind of cool, thanks. How long does it take to do a web scrape versus a DB dump?
  - noSyncCloud 4y ago
    Here's a working link to Kiwix since the link on the page 404s: https://www.kiwix.org/en/download/ https://www.kiwix.org/en/download/
- MonkeyMalarky 4y ago
  IMO it would be more interesting to not use their code and build some kind of specialized search service. One that learns to rank results from user metrics and feedback?
  - he_is_legend 4y ago
    so like personalised google search, with the terrible results that that inevitably produces for code related content? i am confused though. how do you think SO currently works? Both questions and answers are already ranked via user metrics (individual upvotes and historical upvotes) and feedback (comments)?
Wowfunhappy 4y ago
StackExchange is by no means perfect, but the copyleft content license is a key reason I feel comfortable contributing to the platform. (Which I assume is why they do it.)

someguy5344523 4y ago

> We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them. I think that's already the case. Whenever I look up a technical issue, the first result is usually from SO; the next results are usually from websites that copied that first SO page

rr888 4y ago
You're lucky, for me recently its the other way around, spam copies first.

NateEag 4y ago

FWIW, the Codidact project has been plugging away at building an OSS replacement for SE: https://www.codidact.org/ https://www.codidact.org/

hddherman 4y ago

A read-only, self-hostable copy of StackOverflow (and its sister sites) is already doable with Kiwix + the content packages they publish. Content packages: https://wiki.kiwix.org/wiki/Content_in_all_languages https://wiki.kiwix.org/wiki/Content_in_all_languages Example of how a site looks like (in this case SuperUser), hosted on my server: https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/index.html https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/inde... And hosting it really isn't all that difficult: https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/ https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/

pugworthy 4y ago

I’m not trying to be funny but it’s interesting how much panic one can read in the parent comment. We all joke about the dependency on SO. It seems to actually be more true than perhaps we’d like to admit.

300bps 4y ago
It’s unjustified panic. It’s freely downloadable and available everywhere.

deleted 4y ago

[deleted]