3 ms·

Rendering on the server has little actual value in 2015 There is life outside of Google, you know. There are other search engines, other crawlers that are not

by romaniv 12y ago

Rendering on the server has little actual value in 2015 There is life outside of Google, you know. There are other search engines, other crawlers that are not search engines, other programs people use on your website that are neither browsers nor crawlers.

Touche 12y ago
Can you be more specific? If I'm going to spend a great deal of money on compute and tailor my library decisions based on this feature, I need to know that it is really important. Alluding that Yandex might not have this technology yet is not good enough for me.
- mutagen 12y ago
  I use http://hn.premii.com/ http://hn.premii.com/ to read HN on mobile devices. A handful of sites every day aren't compatible with whatever technology they're using to scrape page content. If the comments don't indicate that the article is worth visiting or there aren't any comments yet I don't see the content. Now this is just one anecdote about one pair of eyeballs but I'm a old-school PC full browser using pair of eyeballs, I suspect more and more people and machines are consuming web content in new ways, including without a Javascript interpreter mediating the web experience for them.
- simonw 12y ago
  There are thousands of pieces of software that hit a URL and attempt to do something useful with the contents. Just try linking to a page on your own site on Twitter and tail your logfiles to see all the interesting bots that show up. A couple of examples off the top of my head: If you link to a web page on Reddit, they run a simple crawler to try and extract an image they can use as a thumbnail representation: https://github.com/reddit/reddit/blob/af09fa8dee69bef4f65a1662d9ad91c2329946e1/r2/r2/lib/media.py#L470-L523 https://github.com/reddit/reddit/blob/af09fa8dee69bef4f65a16... If you link to a URL in Slack, they do something similar: https://api.slack.com/docs/unfurling https://api.slack.com/docs/unfurling

by romaniv 12y ago

Touche 12y ago

Can you be more specific? If I'm going to spend a great deal of money on compute and tailor my library decisions based on this feature, I need to know that it is really important. Alluding that Yandex might not have this technology yet is not good enough for me.

mutagen 12y ago
I use http://hn.premii.com/ http://hn.premii.com/ to read HN on mobile devices. A handful of sites every day aren't compatible with whatever technology they're using to scrape page content. If the comments don't indicate that the article is worth visiting or there aren't any comments yet I don't see the content. Now this is just one anecdote about one pair of eyeballs but I'm a old-school PC full browser using pair of eyeballs, I suspect more and more people and machines are consuming web content in new ways, including without a Javascript interpreter mediating the web experience for them.
simonw 12y ago
There are thousands of pieces of software that hit a URL and attempt to do something useful with the contents. Just try linking to a page on your own site on Twitter and tail your logfiles to see all the interesting bots that show up. A couple of examples off the top of my head: If you link to a web page on Reddit, they run a simple crawler to try and extract an image they can use as a thumbnail representation: https://github.com/reddit/reddit/blob/af09fa8dee69bef4f65a1662d9ad91c2329946e1/r2/r2/lib/media.py#L470-L523 https://github.com/reddit/reddit/blob/af09fa8dee69bef4f65a16... If you link to a URL in Slack, they do something similar: https://api.slack.com/docs/unfurling https://api.slack.com/docs/unfurling