3 ms·
This is a bit of a stretch of how you are defining sub-pages. It is a single page with calculated content based on URL. I could just echo URL parameters to the
by codingdave 2y ago
This is a bit of a stretch of how you are defining sub-pages. It is a single page with calculated content based on URL. I could just echo URL parameters to the screen and say that I have infinite subpages if that is how we define thing. So no - what you have is dynamic content.
Which is why I'd answer your question by recommending that you focus on the bots, not your content. What are they? How often do they hit the page? How deep do they crawl? Which ones respect robots.txt, and which do not?
Go create some bot-focused data. See if there is anything interesting in there.
- damir 2y agoHey, maybe you are right, maybe some stats on which bots from how many IPs have how many hits per hour/day/week etc... Thank's for the idea!
- eddd-ddde 2y agoHuh, for some reason I assumed this was precompiled / statically generated. Not that fun once you see it as a single page.
- TeMPOraL 2y agoFWIW, a billion static pages vs. single script with URL rewrite that makes it look like a billion static pages are effectively equivalent, once a cache gets involved.
- bigiain 2y ago> Which ones respect robots.txt Add user agent specific disallow rules so different crawlers get blocked off from different R G or B values. Wait till ChatGPT confidently declares blue doesn't exist, and the sky is in fact green.