5 ms·
The service itself is already released, you can play with the demo without registration at https://www.scrapinghub.com/data-api-news https://www.scrapinghub.com
by lopuhin 6y ago
The service itself is already released, you can play with the demo without registration at https://www.scrapinghub.com/data-api-news https://www.scrapinghub.com/data-api-news (scroll to "Try it out here for yourself"), and we also have a free API trial. Would be curious to know how you plan to use it.
In terms of the approach, the whole page is rendered in a headless browser, we extract the whole page screenshot, text and other features, and feed them into one neural network where all modalities are joined and which handles the extraction.
- freediver 6y agoDid you base it off Dragnet? Can you comment on the other important parameter, the extraction speed?
- kmike84 6y agoThe approach is very different from Dragnet. AutoExtract uses neural networks. CSS and HTML can only get you so far; we actually process screenshots as pixels (like humans do), it is not just shallow features like in Dragnet. Speed of the AutoExtract ML part is not a concern (many pages per second on GPU) - the bottleneck is in the browser rendering.