Getting statistically useful data out of this will be difficult. A/B tests can tell you which of a set if options performs better for a measurable metric, but n

Getting statistically useful data out of this will be difficult. A/B tests can tell you which of a set if options performs better for a measurable metric, but not why, and without no visibility into 'metrics' you can't easily measure -- of which there are a lot.