Most of the stuff should be bypassable with browser automation but you'd need more compute to run a full browser versus a basic uri fetch. I imagine it'd take

Most of the stuff should be bypassable with browser automation but you'd need more compute to run a full browser versus a basic uri fetch. I imagine it'd take quite a bit of shape for the index and be hard to keep it up to date unless you restrict what it indexes.