4 ms·
When working on the Bing web crawler, we tried to take screenshot of webpage to use as thumbnail. What you describe was a huge pain, most page forced us to dow
by skyde 4y ago
When working on the Bing web crawler, we tried to take screenshot of webpage to use as thumbnail.
What you describe was a huge pain, most page forced us to download 15MB of data and load it in memory just to display 2 small images and some text.
A format like PDF would have been much better because we could have read enough byte until we are able to render the visible part of the document.
But instead we had to download and execute 30 javascript files.