4 ms·

Advanced Ajax Techniques for Web Scraping

• HackerDalius 7y ago
  Can the techniques be applied for other scrapers ? Because in the article they are discussed in the context of Web Robots scraping platform.
  • webrobots 7y ago
    This is relevant to frameworks that use headless or normal browser and Javascript. Plus of course jQuery to generate $.ajax requests.
• akriukas 7y ago
  I don't get what these "done()" mean. But otherwise, the logic looks good.
  • webrobots 7y ago
    done() is specific to WR system. Essentially we do not know when a step is finished because it may do waiting from DOM element or Ajax requests. So a developer must place done() where he/she is sure that a step has completed.