2 ms·
> How does a crawler discover these? They are just designed to detect all these filters and try all combinations? Presumably each of the facets is a link that
by xigoi 1y ago
> How does a crawler discover these? They are just designed to detect all these filters and try all combinations?
Presumably each of the facets is a link that adds it to the current query, so if you recursively follow links, you will end up with all combinations.
- nonethewiser 1y agoBut these links... where do they exist? In my experience they are produced from user INPUT. For example I click the sort button and sort by low to high and that adds a &sort=asc. Now I have a new url. After the input. But a crawler just visits the page right? Its not selecting from dropdowns, checkboxes, etc. So apparently all combinations are pregenerated somewhere? That seems weird to me and why I was asking about their use case.