3 ms·

I wrote a simple bit of code to calculate the size of index pages with includes (js, css, images) and here's what I get: page resource total url (b)

I wrote a simple bit of code to calculate the size of index pages with includes (js, css, images) and here's what I get: page resource total url (b) (b) (Kb) ---------------------------------------------- 33857 143 34.0 news.ycombinator.com 62690 55975 118.7 arstechnica.com 11844 223862 235.7 seldomlogical.com 218837 765943 984.8 newyorktimes.com l592771 545304 1138.1 guardian.co.uk The one thing I notice watching the index page being broken down, is the sheer number of resources requested. Didn't bother with JS/browser interaction which you can check with browser dev tools.

swiley 10y ago
This is why people don't read news articles. It's not that their lazy, it's that the sites hosting them almost feel hostile.
- bootload 10y ago
  "It's not that their lazy, it's that the sites hosting them almost feel hostile." Agree with that. A driver of the page size (JS, images, video) is advertising. Advertising and advertising techniques are distorting the news and ability to read it.
mixmastamyk 10y ago
Would enjoy seeing the source.
- bootload 10y ago
  it's evil and slow, I'll make a few mods and whack it up, got an email or twitter acct & I'll contact u. -- PR implemented in py using - Beautiful Soup 4 - Requests - htmllib Core idea is to grab an index url and determine size of page and resources by looking at urls likely to be pulled by page. JS not implemented from what I can read. Method is something like: * pull index page with requests by url * extract headers * build list of resource urls * look at content type to determine new request * extract images, text, css from urls in page * look at content length to determine size * total sizes
- bootload 10y ago
  read away ~ https://github.com/peterrenshaw/pageglut https://github.com/peterrenshaw/pageglut
  - mixmastamyk 10y ago
    Thanks!
    - bootload 10y ago
      and write-up why ~ http://seldomlogical.com/2016/JAN/27/measure-that-page/ http://seldomlogical.com/2016/JAN/27/measure-that-page/

swiley 10y ago

This is why people don't read news articles. It's not that their lazy, it's that the sites hosting them almost feel hostile.

bootload 10y ago
"It's not that their lazy, it's that the sites hosting them almost feel hostile." Agree with that. A driver of the page size (JS, images, video) is advertising. Advertising and advertising techniques are distorting the news and ability to read it.

mixmastamyk 10y ago

Would enjoy seeing the source.

bootload 10y ago
it's evil and slow, I'll make a few mods and whack it up, got an email or twitter acct & I'll contact u. -- PR implemented in py using - Beautiful Soup 4 - Requests - htmllib Core idea is to grab an index url and determine size of page and resources by looking at urls likely to be pulled by page. JS not implemented from what I can read. Method is something like: * pull index page with requests by url * extract headers * build list of resource urls * look at content type to determine new request * extract images, text, css from urls in page * look at content length to determine size * total sizes
bootload 10y ago
read away ~ https://github.com/peterrenshaw/pageglut https://github.com/peterrenshaw/pageglut
- mixmastamyk 10y ago
  Thanks!
  - bootload 10y ago
    and write-up why ~ http://seldomlogical.com/2016/JAN/27/measure-that-page/ http://seldomlogical.com/2016/JAN/27/measure-that-page/