4 ms·

Where were these data sourced from? As far as I know, StackOverflow does not publicize internal analytics with such granularity. If these figures are real, are

Where were these data sourced from? As far as I know, StackOverflow does not publicize internal analytics with such granularity. If these figures are real, are they leaked?

encomiast 3y ago
Users with over 25,000 reputation can see site analytics that include charts that look a lot like these. https://stackoverflow.com/help/privileges/site-analytics https://stackoverflow.com/help/privileges/site-analytics
- MontyCarloHall 3y ago
  That must be it. It would have been nice for the original post to include this information.
- fhoffa 3y ago
  Looking at that source, there's a pretty straightforward explanation of where the page views come from. - Last month, Stack Overflow had ~142,575,642 visits. - Last month, Google gave 127,896,508 visits to Stack Overflow - Last month, Bing gave 7,491,274 visits to Stack Overflow You could say that the Stack Overflow # of pageviews depends mainly on: - How often people are searching Google/Bing for answer. - How often Google/Bing rank Stack Overflow high enough for people to click into it.
- XCSme 3y ago
  I can confirm: https://i.snipboard.io/0wmnSv.jpg https://i.snipboard.io/0wmnSv.jpg
  - XCSme 3y ago
    We can also see that lately there are more questions than answers, which shows that most experts are no longer that active, or that there are more beginners and fewer experts overall.
minimaxir 3y ago
Stack Overflow used to release their data archives quarterly on BigQuery. Looking at the BQ datasets, they were last updated Nov 2022, which doesn't have the latest 2023 info in the submission.
- fhoffa 3y ago
  FWIW, I now analyze the Stack Overflow dumps on Snowflake https://medium.com/snowflake/how-to-load-the-stack-overflow-xml-dumps-into-snowflake-631c90fff57 https://medium.com/snowflake/how-to-load-the-stack-overflow-...
  - dleeftink 3y ago
    Thanks for sharing, good to see alternative options popping up. My wish is that the Stack Exchange dataset could one day be provided as a streaming parquet or arrow table, as underfunded grads and post-grads could then more easily/selectively sample the datasets (similar to how Huggingface provides some of its datasets)[1][2]. The Hugginface repo unfortunately prefilters some of the tables/rows according to some criteria, making it less usable for general analytical queries that the BQ or SEDE datasets enable. If anyone knows of an 'XML-streaming' solution that directly samples from the Internet Archive's data dumps, I am all ears. [1]: https://huggingface.co/docs/datasets-server/rows https://huggingface.co/docs/datasets-server/rows [2]: https://huggingface.co/datasets/HuggingFaceGECLM/StackExchange_Mar2023/tree/main/data https://huggingface.co/datasets/HuggingFaceGECLM/StackExchan...
dleeftink 3y ago
Besides the Archive.org data dump, there is also the Stack Exchange Data Explorer for which there are thousands of user queries[1]. For instance, this user query by Starball tracks network contributions over time[2][3]. [1]: https://data.stackexchange.com/meta.stackexchange/queries https://data.stackexchange.com/meta.stackexchange/queries [2]: https://data.stackexchange.com/meta.stackexchange/query/1759268/network-wide-new-contributions-per-month-since-2018#graph https://data.stackexchange.com/meta.stackexchange/query/1759... [3]: Static image if the query times out: https://i.stack.imgur.com/LYZQm.png https://i.stack.imgur.com/LYZQm.png

Where were these data sourced from? As far as I know, StackOverflow does not publicize internal analytics with such granularity. If these figures are real, are they leaked?

encomiast 3y ago

Users with over 25,000 reputation can see site analytics that include charts that look a lot like these. https://stackoverflow.com/help/privileges/site-analytics https://stackoverflow.com/help/privileges/site-analytics

MontyCarloHall 3y ago
That must be it. It would have been nice for the original post to include this information.
fhoffa 3y ago
Looking at that source, there's a pretty straightforward explanation of where the page views come from. - Last month, Stack Overflow had ~142,575,642 visits. - Last month, Google gave 127,896,508 visits to Stack Overflow - Last month, Bing gave 7,491,274 visits to Stack Overflow You could say that the Stack Overflow # of pageviews depends mainly on: - How often people are searching Google/Bing for answer. - How often Google/Bing rank Stack Overflow high enough for people to click into it.
XCSme 3y ago
I can confirm: https://i.snipboard.io/0wmnSv.jpg https://i.snipboard.io/0wmnSv.jpg
- XCSme 3y ago
  We can also see that lately there are more questions than answers, which shows that most experts are no longer that active, or that there are more beginners and fewer experts overall.

minimaxir 3y ago

Stack Overflow used to release their data archives quarterly on BigQuery. Looking at the BQ datasets, they were last updated Nov 2022, which doesn't have the latest 2023 info in the submission.

fhoffa 3y ago
FWIW, I now analyze the Stack Overflow dumps on Snowflake https://medium.com/snowflake/how-to-load-the-stack-overflow-xml-dumps-into-snowflake-631c90fff57 https://medium.com/snowflake/how-to-load-the-stack-overflow-...
- dleeftink 3y ago
  Thanks for sharing, good to see alternative options popping up. My wish is that the Stack Exchange dataset could one day be provided as a streaming parquet or arrow table, as underfunded grads and post-grads could then more easily/selectively sample the datasets (similar to how Huggingface provides some of its datasets)[1][2]. The Hugginface repo unfortunately prefilters some of the tables/rows according to some criteria, making it less usable for general analytical queries that the BQ or SEDE datasets enable. If anyone knows of an 'XML-streaming' solution that directly samples from the Internet Archive's data dumps, I am all ears. [1]: https://huggingface.co/docs/datasets-server/rows https://huggingface.co/docs/datasets-server/rows [2]: https://huggingface.co/datasets/HuggingFaceGECLM/StackExchange_Mar2023/tree/main/data https://huggingface.co/datasets/HuggingFaceGECLM/StackExchan...

dleeftink 3y ago

Besides the Archive.org data dump, there is also the Stack Exchange Data Explorer for which there are thousands of user queries[1]. For instance, this user query by Starball tracks network contributions over time[2][3]. [1]: https://data.stackexchange.com/meta.stackexchange/queries https://data.stackexchange.com/meta.stackexchange/queries [2]: https://data.stackexchange.com/meta.stackexchange/query/1759268/network-wide-new-contributions-per-month-since-2018#graph https://data.stackexchange.com/meta.stackexchange/query/1759... [3]: Static image if the query times out: https://i.stack.imgur.com/LYZQm.png https://i.stack.imgur.com/LYZQm.png