2 ms·
Each diff is done slightly differently depending on what it is... we use headless Chrome + Puppeteer on Fargate for crawling. We use Puppeteer itself to take sc
by bluepeter 7y ago
Each diff is done slightly differently depending on what it is... we use headless Chrome + Puppeteer on Fargate for crawling. We use Puppeteer itself to take screenshots and output HTML. We then use a separate Lambda function to extract the text.
From that, we feed the results into "diff" Lambda functions to compute image, text, HTML, and network diffs. We treat the text diff as the primary diff type, and so only if we have a text diff do we do the other comparison types.
From these diffs, we then feed the text + some DOM info into ML. There, we use added text, deleted text, and, for each, the shortest unique CSS selector, the immediate parent tag, and some other items that I may be missing (possibly some approximation of where in the main text the change appears... e.g., top 10%, top 20% IIRC).
Hopefully this answers your question?