3 ms·
It will basically go through all the links it finds, that can be millions of links. You can also tell it to ignore certain links using regular expressions or vi
by calufa 13y ago
It will basically go through all the links it finds, that can be millions of links. You can also tell it to ignore certain links using regular expressions or via tales java apis.
- calufa 13y agoHere is a sample code of a 1 depth scrape: https://github.com/calufa/tales-templates/blob/master/core/src/sample/FirstDepthTemplate.java https://github.com/calufa/tales-templates/blob/master/core/s... This is the api call to start the scraper on twitter: http://localhost:8080/start?process=tales.scrapers.LoopScraper http://localhost:8080/start?process=tales.scrapers.LoopScrap... -template tales.templates.FirstDepthTemplate -threads 2 -namespace com_twitter -baseURL twitter.com