3 ms·
This is interesting, but it leads me to ask, what can you expect a JavaScript-enabled web crawler do? Load the page into the DOM, wait a few seconds for the du
by pak 17y ago
This is interesting, but it leads me to ask, what can you expect a JavaScript-enabled web crawler do? Load the page into the DOM, wait a few seconds for the dust to settle, start firing click events on everything and recording what requests are generated? It would seem a crawler could cause some really undesirable behavior on your site if that were the case. Imagine click events that post comments, delete things, downvote them, or generate emails. Since they are hidden behind JavaScript a web developer might expect a crawler not to try them...
- fredoliveira 17y agoThere have been a few discussions about this topic in the past, and in the end it boils down to the developer being smart about the way he does requests. I'm quite sure (wish I could remember where I read it) the crawler only actually performs GETs (which makes total sense because GET was always meant for retrieving and listing data), so a smart developer would use other HTTP methods (PUT, POST, DELETE) for actions that have modify existing application data.