3 ms·
> 1. Get as much data about each site as possible. Try to find data that you think will separate the non-ecommerce sites from the ecommerce sites. This data sho
by afro88 10y ago
> 1. Get as much data about each site as possible. Try to find data that you think will separate the non-ecommerce sites from the ecommerce sites. This data should be in a table form- think of an Excel spreadsheet, where each row represents a different site.
This will be the hardest step. Feature extraction from html for identifying specific things like products, for any given site, is very hard (in my experience, which I will admit is pretty limited). Would love to be proven wrong though.
- fnbr 10y agoI was thinking you could approach it as a document classification problem, where you extract the text from the HTML and work with that. I had a lot of success finding duplicate bug reports by comparing the text from the bug reports with reference documents in a variety of topics (e.g. security, networking, C++), and getting a sense of how similar the text is to the reference text. That gives you a score of how relevant each subject is to the document. You could do something similar here- download the text from 10 ecommerce sites, run some sort of topic extraction algorithm (like LDA) on it and then compare the text from the sites you're trying to classify with the text from the reference sites.