5 ms·

If you have enough volume* you can do simple a/b testing to find the demand curve. *this is probably less than you might think; at low volumes you can run a fo

If you have enough volume* you can do simple a/b testing to find the demand curve. *this is probably less than you might think; at low volumes you can run a for a week then b for another week.

gingerlime 10y ago
I'm a huge fan of A/B testing, and use it regularly. (Even wrote an open source implementation with a backend running on AWS Lambda), but have a bit of a moral / public perception / potential backlash fear of running pricing A/B tests... How do you deal with this? (Or you don't?)
- thatcat 10y ago
  >pricing A/B tests moral dilemma You could call the lower price "on sale".
  - stagbeetle 10y ago
    Adding on to this: A "randomly selected sale participation" note.
  - RickS 10y ago
    That would pollute the test, IMO. As things like JCPenny's failed pricing experiments show, price-consciousness and deal-consciousness are two different things, and can lead to some stupid behavior where customers only buy overpriced goods that are marked as being on sale, rather than buying the same thing at the same price without the sale tag.
- matheweis 10y ago
  I am not sure what you mean by a moral issue. Money and goods exchange hands (or not) in a mutually agreed on transaction. I don't think there is an issue so long as there is no intentionally targeted discrimination. If someone is unhappy with their purchase and lets me know, I nearly always give a refund (where possible; the App Store prevents this). Sometimes I've given my software away for free if someone asks nicely. I have never had any backlash to running pricing A/B tests. About once a year I will run prices at 0.5 and 1.5 the normal price for a week or two to make sure I am still on the right point of the demand curve. Sometimes it moves and I will stick with a new price.
  - gingerlime 10y ago
    > I don't think there is an issue so long as there is no intentionally targeted discrimination. Is random discrimination fine? I'm genuinely asking. I don't know the answer, but somehow it still feels morally shaky to me or unfair. Yes, we do give refunds or discounts retrospectively in many cases, and try to be nice and fair and open. But there's just something a bit awkward with randomly charging people more (or less) for the exact same product, at the exact same time. I think most people will feel manipulated if they discovered it, even if they eventually paid the lower price.
    - matheweis 10y ago
      Given my approach; a single A/B test once a year, one week at 0.5 price and one week at 1.5 price, less than 2% got a half-off discount, and less than 2% paid the higher price (actually, since sales usually drop for the 1.5 price, it's likely closer to 1% of sales pay the higher price).
stagbeetle 10y ago
Have you done a/b testing in one of your startups? It's a fool's errand at low volumes. Whatever you find out isn't statistically significant.
- matheweis 10y ago
  > ... at low volumes whatever you find out isn't statistically significant. Depends on what volume you are measuring. In my case I have an abysmally low conversion rate, so I can in fact have a low volume of sales relative to the number of visitors and still have statistically significant results.

If you have enough volume* you can do simple a/b testing to find the demand curve. *this is probably less than you might think; at low volumes you can run a for a week then b for another week.

gingerlime 10y ago

I'm a huge fan of A/B testing, and use it regularly. (Even wrote an open source implementation with a backend running on AWS Lambda), but have a bit of a moral / public perception / potential backlash fear of running pricing A/B tests... How do you deal with this? (Or you don't?)

thatcat 10y ago
>pricing A/B tests moral dilemma You could call the lower price "on sale".
- stagbeetle 10y ago
  Adding on to this: A "randomly selected sale participation" note.
- RickS 10y ago
  That would pollute the test, IMO. As things like JCPenny's failed pricing experiments show, price-consciousness and deal-consciousness are two different things, and can lead to some stupid behavior where customers only buy overpriced goods that are marked as being on sale, rather than buying the same thing at the same price without the sale tag.
matheweis 10y ago
I am not sure what you mean by a moral issue. Money and goods exchange hands (or not) in a mutually agreed on transaction. I don't think there is an issue so long as there is no intentionally targeted discrimination. If someone is unhappy with their purchase and lets me know, I nearly always give a refund (where possible; the App Store prevents this). Sometimes I've given my software away for free if someone asks nicely. I have never had any backlash to running pricing A/B tests. About once a year I will run prices at 0.5 and 1.5 the normal price for a week or two to make sure I am still on the right point of the demand curve. Sometimes it moves and I will stick with a new price.
- gingerlime 10y ago
  > I don't think there is an issue so long as there is no intentionally targeted discrimination. Is random discrimination fine? I'm genuinely asking. I don't know the answer, but somehow it still feels morally shaky to me or unfair. Yes, we do give refunds or discounts retrospectively in many cases, and try to be nice and fair and open. But there's just something a bit awkward with randomly charging people more (or less) for the exact same product, at the exact same time. I think most people will feel manipulated if they discovered it, even if they eventually paid the lower price.
  - matheweis 10y ago
    Given my approach; a single A/B test once a year, one week at 0.5 price and one week at 1.5 price, less than 2% got a half-off discount, and less than 2% paid the higher price (actually, since sales usually drop for the 1.5 price, it's likely closer to 1% of sales pay the higher price).

stagbeetle 10y ago

Have you done a/b testing in one of your startups? It's a fool's errand at low volumes. Whatever you find out isn't statistically significant.

matheweis 10y ago
> ... at low volumes whatever you find out isn't statistically significant. Depends on what volume you are measuring. In my case I have an abysmally low conversion rate, so I can in fact have a low volume of sales relative to the number of visitors and still have statistically significant results.