3 ms·

Making our pageview data easily accessible

29 points by devhxinc 11y ago

danso 11y ago
edit: Added the use of `date` to automatically return last 120 days of data, and crudely label the axes edit: Here's a gist https://gist.github.com/dannguyen/f415b1797f686f995f8e https://gist.github.com/dannguyen/f415b1797f686f995f8e I thought there's a Bash one-liner in there somewhere to read the API and convert it to a Google Static chart...but I haven't had my full cup of coffee this morning. Also, my Bash-fu is weak, so here it is as a partially completed function...I'm sure someone here can turn it into a one-liner: Given a user argument, such as "Star_Wars" [0], the JSON response is parsed with the jq tool [1], then wrangled into a series of comma-delimited values to be passed into Google Static Charts API [2]: Resulting URL which is a timeseries of the past 120 days of daily pageviews for "Star_Wars": https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t:26867,21185,19318,18070,18024,20756,22056,18436,18098,17767,18672,21258,22884,24471,24350,19195,18220,18934,26076,27259,29547,27867,23664,19716,19332,19375,22766,25316,20725,19062,18836,18509,19057,21748,23147,18438,18824,17608,18556,19966,19937,22061,18896,18428,16753,14423,16903,29012,24012,17570,15676,15587,16551,19074,30278,36592,23440,19306,18325,18281,19491,28537,33151,41537,185853,111159,72979,61927,68754,57846,46054,38470,77358,104545,33733,41420,45332,37476,32674,34710,37770,43845,58388,59431,47990,44572,41607,39417,47977,46063,52561,40330,36673,35941,38812,44684,49243,58960,58053,50050,52144,61630,71900,72059,75722,64551,65993,63971,63492,70945,84578,98991,93108,86095,84668,87699,111377,142213,175058,202539&chds=0,202539&chbh=5,0,1&chxt=x,y&chxp=0,0&chxl=0:|2015-08-17|2015-12-15|1:||101269|202539&chtt=Last+120+days+of+pageviews+for+Wikipedia+page+on+Star_Wars https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t... Here's the bash function function chart_wpgviews(){ # first argument is article name # eh, too lazy to look up string comparison, so hardcoding the start and end date vals, # since it seems to be locked at a max of 4 months anyway...and hard-coding en.wikipedia PAGENAME="$1" PROJECT="en.wikipedia" ENDTIME=$(date +%s) STARTTIME=$(date --date="@$(($ENDTIME-120*24*60*60))" +%s) # 120 days ago ENDDATE=$(date --date="@$ENDTIME" +%Y-%m-%d) STARTDATE=$(date --date="@$STARTTIME" +%Y-%m-%d) RESPONSE=$(curl -s https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/$PROJECT/all-access/user/$PAGENAME/daily/$(tr -d '-' <<< $STARTDATE)/$(tr -d '-' <<< $ENDDATE)) VALS=$(jq '.items[] | .views' <<< $RESPONSE | tr '\n' ',') MAXVAL=$(jq '.items[] | .views' <<< $RESPONSE | sort -rn | head -n1) echo "https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t:${VALS%?}&chds=0,$MAXVAL&chbh=5,0,1&chxt=x,y&chxp=0,0&chxl=0:|$STARTDATE|$ENDDATE|1:||$((MAXVAL/2))|$MAXVAL&chtt=Last+120+days+of+pageviews+for+Wikipedia+page+on+$PAGENAME" } [0] https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/user/Star_Wars/daily/20150101/20151215 https://wikimedia.org/api/rest_v1/metrics/pageviews/per-arti... [1] https://stedolan.github.io/jq/ https://stedolan.github.io/jq/ [2] https://developers.google.com/chart/image/docs/data_formats https://developers.google.com/chart/image/docs/data_formats
flashman 11y ago
> http://top.hatnote.com/ http://top.hatnote.com/ Wow, it's like a window into the zeitgeist.

29 points by devhxinc 11y ago

danso 11y ago

edit: Added the use of `date` to automatically return last 120 days of data, and crudely label the axes edit: Here's a gist https://gist.github.com/dannguyen/f415b1797f686f995f8e https://gist.github.com/dannguyen/f415b1797f686f995f8e I thought there's a Bash one-liner in there somewhere to read the API and convert it to a Google Static chart...but I haven't had my full cup of coffee this morning. Also, my Bash-fu is weak, so here it is as a partially completed function...I'm sure someone here can turn it into a one-liner: Given a user argument, such as "Star_Wars" [0], the JSON response is parsed with the jq tool [1], then wrangled into a series of comma-delimited values to be passed into Google Static Charts API [2]: Resulting URL which is a timeseries of the past 120 days of daily pageviews for "Star_Wars": https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t:26867,21185,19318,18070,18024,20756,22056,18436,18098,17767,18672,21258,22884,24471,24350,19195,18220,18934,26076,27259,29547,27867,23664,19716,19332,19375,22766,25316,20725,19062,18836,18509,19057,21748,23147,18438,18824,17608,18556,19966,19937,22061,18896,18428,16753,14423,16903,29012,24012,17570,15676,15587,16551,19074,30278,36592,23440,19306,18325,18281,19491,28537,33151,41537,185853,111159,72979,61927,68754,57846,46054,38470,77358,104545,33733,41420,45332,37476,32674,34710,37770,43845,58388,59431,47990,44572,41607,39417,47977,46063,52561,40330,36673,35941,38812,44684,49243,58960,58053,50050,52144,61630,71900,72059,75722,64551,65993,63971,63492,70945,84578,98991,93108,86095,84668,87699,111377,142213,175058,202539&chds=0,202539&chbh=5,0,1&chxt=x,y&chxp=0,0&chxl=0:|2015-08-17|2015-12-15|1:||101269|202539&chtt=Last+120+days+of+pageviews+for+Wikipedia+page+on+Star_Wars https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t... Here's the bash function function chart_wpgviews(){ # first argument is article name # eh, too lazy to look up string comparison, so hardcoding the start and end date vals, # since it seems to be locked at a max of 4 months anyway...and hard-coding en.wikipedia PAGENAME="$1" PROJECT="en.wikipedia" ENDTIME=$(date +%s) STARTTIME=$(date --date="@$(($ENDTIME-120*24*60*60))" +%s) # 120 days ago ENDDATE=$(date --date="@$ENDTIME" +%Y-%m-%d) STARTDATE=$(date --date="@$STARTTIME" +%Y-%m-%d) RESPONSE=$(curl -s https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/$PROJECT/all-access/user/$PAGENAME/daily/$(tr -d '-' <<< $STARTDATE)/$(tr -d '-' <<< $ENDDATE)) VALS=$(jq '.items[] | .views' <<< $RESPONSE | tr '\n' ',') MAXVAL=$(jq '.items[] | .views' <<< $RESPONSE | sort -rn | head -n1) echo "https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t:${VALS%?}&chds=0,$MAXVAL&chbh=5,0,1&chxt=x,y&chxp=0,0&chxl=0:|$STARTDATE|$ENDDATE|1:||$((MAXVAL/2))|$MAXVAL&chtt=Last+120+days+of+pageviews+for+Wikipedia+page+on+$PAGENAME" } [0] https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/user/Star_Wars/daily/20150101/20151215 https://wikimedia.org/api/rest_v1/metrics/pageviews/per-arti... [1] https://stedolan.github.io/jq/ https://stedolan.github.io/jq/ [2] https://developers.google.com/chart/image/docs/data_formats https://developers.google.com/chart/image/docs/data_formats

flashman 11y ago

> http://top.hatnote.com/ http://top.hatnote.com/ Wow, it's like a window into the zeitgeist.