I have something in mind along those lines, and I'm going to provide at least two or three ways to create custom datasets from my 'snapshot of the Web.' I thoug

I have something in mind along those lines, and I'm going to provide at least two or three ways to create custom datasets from my 'snapshot of the Web.' I thought about your suggestion of allowing customers to execute their own scripts over the snapshot but the security issues involved are my big concern. I would need a way to sandbox it. My plan right now is to offer the ability to run simple data tasks on the snapshot. For example, regex (filtered for things like SSN matchers, and so on), a language with a limited vocabulary for grabbing features with certain textual of visual characteristics, or even something as simple as xpath. I would run these tasks on the cluster and the result would be returned to the customer.