3 ms·
How is that any different than wanting to archive a CGI website from the 90s with a URL structure like http://example.com/?query=foo http://example.com/?query=f
by tshaddox 5y ago
How is that any different than wanting to archive a CGI website from the 90s with a URL structure like http://example.com/?query=foo http://example.com/?query=foo? Unless there's an index page with links to all possible query values, or you can work out how to manually iterate all possible query values, there's not much you can do. This doesn't seem to have anything to do with JavaScript data visualizations specifically.
- titusjohnson 5y agoThat URL structure is trivial for a crawler to walk and index. I'm not sure why you'd assume that there wouldn't be an index page, such a site would have all desired links in the DOM, the crawler just sniffs those out and visits in sequence. There's no need to think that the links would somehow be 'hidden' from the user and have to be randomly enumerated... Not only that, but a site of that era probably also has a sitemap.xml file which would enumerate all available public endpoints, specifically to make it easier for crawlers to index everything.
- tshaddox 5y ago> I'm not sure why you'd assume that there wouldn't be an index page I’m not assuming either way. I’m just pointing out that either type of web site could choose to have an index page or choose not to have an index page.