5 ms·

The US intelligence community actually keeps a clone of Stack Overflow that is updated every 12 hours and copied to servers on the classified version of the Int

The US intelligence community actually keeps a clone of Stack Overflow that is updated every 12 hours and copied to servers on the classified version of the Internet so that people developing classified software can use it without needing to turn around to their unclassified workstations. I'm sure they're not alone in this, so the entirety of the answers there could be restored from mirrors or just used from the mirrors if the main site disappeared. Nearly all of the information there is also available in the public documentation of whatever the question asker is asking about anyway. Forcing developers to use real documentation instead of Stack Overflow would not likely hurt everyday work. It might become harder to find literal worked examples, but even those are mostly duplicated elsewhere and SO is just making it easier on search engines to find it.

netsharc 6y ago
How do you know that info in the first paragraph? On top of that security, it'd also be interesting to monitor what someone (based on their IP and browser fingerprint) was looking up on SO, you could probably figure out what they were working on that way. Although I guess when you work for the spooks, they could figure out how to mask your IP that each query would come from a different IP from the world (basically I'm describing Tor, I wonder if they have their own Tor network).
- langitbiru 6y ago
  > On top of that security, it'd also be interesting to monitor what someone (based on their IP and browser fingerprint) was looking up on SO, you could probably figure out what they were working on that way. When the security incident happened last year on SO, SO team monitored what information the attacker searched in SO. So I guess some people want some privacy in searching information in SO. https://stackoverflow.blog/2021/01/25/a-deeper-dive-into-our-may-2019-security-incident/ https://stackoverflow.blog/2021/01/25/a-deeper-dive-into-our...
- rovr138 6y ago
  The second part, regarding mirrors, https://archive.org/details/stackexchange https://archive.org/details/stackexchange
norswap 6y ago
> Nearly all of the information there is also available in the public documentation of whatever the question asker is asking about anyway. In the absolute that may be true, but even if a low percentage of those are not available elsewhere, they are absolutely crucial. At least, they have been crucial to me. I'm too young to know if it's a recent development, but documentation this day ain't worth much.
- emayljames 6y ago
  Oh, no; it has always been that way. Either too much information (needle in haystack), or undocumented (think Apple's latest Swift). The best documentation I've seen is the PHP site that not only has all function listed, but gives official examples, with user comments that can be voted on.
dataflow 6y ago
> Forcing developers to use real documentation instead of Stack Overflow would not likely hurt everyday work. For beginner-level stuff maybe, but there's so much on StackOverflow about bugs, pitfalls, caveats, breaking changes, rationales, use cases, background, etc. that you can't find easily (if at all) in software documentation, if the documentation even pops up in your searches.
- qntmfred 6y ago
  > there's so much on StackOverflow about bugs, pitfalls, caveats, breaking changes, rationales, use cases, background, etc this question is closed as off-topic.
svara 6y ago
> Nearly all of the information there is also available in the public documentation of whatever the question asker is asking about anyway. That's really not true in any meaningful way. Maybe the information is somewhere, but some Stackoverflow answers are so good that they are the best resource on a specific narrow topic that you will find online. See for example something like [0]. That's the type of answer you would get from a good book or teacher, good luck finding something comparable in the often terse official python docs. Sure, there are a lot of lazy SO Q/As that sound like someone wasn't doing their homework, but that's far from the only type of content there. [0] https://stackoverflow.com/questions/100003/what-are-metaclasses-in-python https://stackoverflow.com/questions/100003/what-are-metaclas... (Edit: another great example is basically anything that concerns advanced usage of matplotlib. That library is so weird and the docs so terrible that working off of SO examples really is the most efficient way to use it.)
- marcosdumay 6y ago
  Yeah, that's the kind of answer that you don't get anymore on SO, mostly because that kind of question isn't allowed anymore. Nowadays questions must be much more specific, what has the consequence that the answers are much less generally useful.
  - ajfjrbfbf 6y ago
    There are less and less general questions because most of them have been answered already. If I see a question like that either I give a link to the docs or mark as duplicate. There's no point in writing such long answers if the information already exists somewhere else.
  - threatofrain 6y ago
    StackOverflow once tried to do an interesting community experiment around replicating documentation on everything but it lacked guidance / self-direction and they cancelled it.
  - yagizdegirmenci 6y ago
    I can't agree this more, once a moderator quoted about this in a moderation room: "I'm starting to become concerned that this room is somehow promoting the idea that all questions need to have an MCVE and/or that debugging questions are the only questions that are on-topic for Stack Overflow. That is not only wrong, it's horribly wrong, dangerously wrong. So I step in and correct this misconception as often as I can."
ArtWomb 6y ago
>>> The US intelligence community actually keeps a clone of Stack Overflow I feel this is a prudent idea. Not necessarily for a full blown start-up, but a community-funded resource. If you stripped out all the extras. And just cloned the questions and top 2 answers. The text based data set itself is very managable. Several gigabytes in scale. Not to mention its applicability for ML/NLP/codegen research ;)
- rovr138 6y ago
  Maybe you can use their dataset, https://www.kaggle.com/stackoverflow/stackoverflow https://www.kaggle.com/stackoverflow/stackoverflow Source is here, https://archive.org/details/stackexchange https://archive.org/details/stackexchange
  - langitbiru 6y ago
    I think what the poster means is more than just unpacking the dataset. So the project should update itself every 12 hours, and has the capability of searching the information (using local database).
    - throwawaybutwhy 6y ago
      > the capability of searching the information Not something SO is known for. Thus, it would be pure improvement over the status quo.
mraza007 6y ago
Hey when you mentioned US intelligence community do you mean the government agencies such as the army,cia or DoD I’m just curious to know

netsharc 6y ago

How do you know that info in the first paragraph? On top of that security, it'd also be interesting to monitor what someone (based on their IP and browser fingerprint) was looking up on SO, you could probably figure out what they were working on that way. Although I guess when you work for the spooks, they could figure out how to mask your IP that each query would come from a different IP from the world (basically I'm describing Tor, I wonder if they have their own Tor network).

langitbiru 6y ago
> On top of that security, it'd also be interesting to monitor what someone (based on their IP and browser fingerprint) was looking up on SO, you could probably figure out what they were working on that way. When the security incident happened last year on SO, SO team monitored what information the attacker searched in SO. So I guess some people want some privacy in searching information in SO. https://stackoverflow.blog/2021/01/25/a-deeper-dive-into-our-may-2019-security-incident/ https://stackoverflow.blog/2021/01/25/a-deeper-dive-into-our...
rovr138 6y ago
The second part, regarding mirrors, https://archive.org/details/stackexchange https://archive.org/details/stackexchange

norswap 6y ago

> Nearly all of the information there is also available in the public documentation of whatever the question asker is asking about anyway. In the absolute that may be true, but even if a low percentage of those are not available elsewhere, they are absolutely crucial. At least, they have been crucial to me. I'm too young to know if it's a recent development, but documentation this day ain't worth much.

emayljames 6y ago
Oh, no; it has always been that way. Either too much information (needle in haystack), or undocumented (think Apple's latest Swift). The best documentation I've seen is the PHP site that not only has all function listed, but gives official examples, with user comments that can be voted on.

dataflow 6y ago

> Forcing developers to use real documentation instead of Stack Overflow would not likely hurt everyday work. For beginner-level stuff maybe, but there's so much on StackOverflow about bugs, pitfalls, caveats, breaking changes, rationales, use cases, background, etc. that you can't find easily (if at all) in software documentation, if the documentation even pops up in your searches.

qntmfred 6y ago
> there's so much on StackOverflow about bugs, pitfalls, caveats, breaking changes, rationales, use cases, background, etc this question is closed as off-topic.

svara 6y ago

> Nearly all of the information there is also available in the public documentation of whatever the question asker is asking about anyway. That's really not true in any meaningful way. Maybe the information is somewhere, but some Stackoverflow answers are so good that they are the best resource on a specific narrow topic that you will find online. See for example something like [0]. That's the type of answer you would get from a good book or teacher, good luck finding something comparable in the often terse official python docs. Sure, there are a lot of lazy SO Q/As that sound like someone wasn't doing their homework, but that's far from the only type of content there. [0] https://stackoverflow.com/questions/100003/what-are-metaclasses-in-python https://stackoverflow.com/questions/100003/what-are-metaclas... (Edit: another great example is basically anything that concerns advanced usage of matplotlib. That library is so weird and the docs so terrible that working off of SO examples really is the most efficient way to use it.)

marcosdumay 6y ago
Yeah, that's the kind of answer that you don't get anymore on SO, mostly because that kind of question isn't allowed anymore. Nowadays questions must be much more specific, what has the consequence that the answers are much less generally useful.
- ajfjrbfbf 6y ago
  There are less and less general questions because most of them have been answered already. If I see a question like that either I give a link to the docs or mark as duplicate. There's no point in writing such long answers if the information already exists somewhere else.
- threatofrain 6y ago
  StackOverflow once tried to do an interesting community experiment around replicating documentation on everything but it lacked guidance / self-direction and they cancelled it.
- yagizdegirmenci 6y ago
  I can't agree this more, once a moderator quoted about this in a moderation room: "I'm starting to become concerned that this room is somehow promoting the idea that all questions need to have an MCVE and/or that debugging questions are the only questions that are on-topic for Stack Overflow. That is not only wrong, it's horribly wrong, dangerously wrong. So I step in and correct this misconception as often as I can."

ArtWomb 6y ago

>>> The US intelligence community actually keeps a clone of Stack Overflow I feel this is a prudent idea. Not necessarily for a full blown start-up, but a community-funded resource. If you stripped out all the extras. And just cloned the questions and top 2 answers. The text based data set itself is very managable. Several gigabytes in scale. Not to mention its applicability for ML/NLP/codegen research ;)

rovr138 6y ago
Maybe you can use their dataset, https://www.kaggle.com/stackoverflow/stackoverflow https://www.kaggle.com/stackoverflow/stackoverflow Source is here, https://archive.org/details/stackexchange https://archive.org/details/stackexchange
- langitbiru 6y ago
  I think what the poster means is more than just unpacking the dataset. So the project should update itself every 12 hours, and has the capability of searching the information (using local database).
  - throwawaybutwhy 6y ago
    > the capability of searching the information Not something SO is known for. Thus, it would be pure improvement over the status quo.

mraza007 6y ago

Hey when you mentioned US intelligence community do you mean the government agencies such as the army,cia or DoD I’m just curious to know