3 ms·
With the norm nowadays using JS frameworks on the front-end I would expect more and more search engines to use things like PhantomJS to "scrape" the page. If yo
by joshstrange 12y ago
With the norm nowadays using JS frameworks on the front-end I would expect more and more search engines to use things like PhantomJS to "scrape" the page. If you decide to reject traffic like this isn't there a good chance your SEO will fall?
- jsoverson 12y agoThings like phantomjs, not phantomjs. PhantomJS is often used to automate sites against the TOS.
- nkozyra 12y agoI doubt any serious search engine is literally using PhantomJS, so I wouldn't see this as a huge problem. Google is in all likelihood using V8 directly and is explicit about who it is. In fact, most legitimate search engine spiders are.
- joshstrange 12y agoI agree, I was speaking more on trying to block crawlers that look like browsers in general, sorry that wasn't more clear.