maybe i'm thinking of the wrong kind of content they want to crawl, but isn't it easier to provide special crawlable, plain html pages (possibly hidden to users

maybe i'm thinking of the wrong kind of content they want to crawl, but isn't it easier to provide special crawlable, plain html pages (possibly hidden to users or provided as an "accessible" version).