4 ms·

I have wanted to perform a rigorous double-blind wine-tasting experiment for a long time, but have always lacked the resources. The true test of the validity o

I have wanted to perform a rigorous double-blind wine-tasting experiment for a long time, but have always lacked the resources. The true test of the validity of wine-tasting isn't whether experts can tell "good" wine from "bad" wine, but whether experts can reliably describe the same wine with the same language with an acceptably small margin of error. My ideal experiment would give blindfolded wine experts a number of wines to taste. They would rate each wine using some appropriate standardized scale that would measure a number of relevant qualities of the wine. The experts would be given multiple samples of each wine, randomized appropriately. The experiment would measure how closely each expert mirrored their own rating of each wine on subsequent tastings. This is basically a slightly more complex version of the famous (and hypothetical) "Lady tasting tea" experiment [1]. I like this setup because the ratings of each expert are compared only to their own previous ratings. I think this minimizes ambiguity and external sources of error. [1] https://en.wikipedia.org/wiki/Lady_tasting_tea https://en.wikipedia.org/wiki/Lady_tasting_tea

JoeAltmaier 12y ago
Doesn't have to cost much. Just choose some supposed wildly different wines, and tear up a sheet to make blindfolds. My father-in-law Thomas is from India. He likes whiskey including Indian brands; his colleague was a Scotch whisky snob. At a party he challenged the snob to name all the whiskeys on the bar. Thomas went into another room with the bottles in a bag; came out carrying a tray with half a dozen shot glasses. The snob tasted each one, make erudite remarks about their qualities and selected a brand for each one. Thomas opened the bag and showed that only the Indian bottle had the seal cracked. Needless to say Thomas doesn't get invited over there any more.
- gabemart 12y ago
  > Doesn't have to cost much. Just choose some supposed wildly different wines, and tear up a sheet to make blindfolds. The hard part is finding qualified experts who are willing to participate. The experiment would be most interesting and uncontroversial if it was done with experts whose credentials were unassailable.
  - JoeAltmaier 12y ago
    You could find experts in your social circle. The results would be just about as satisfying personally.
  - deleted 12y ago
    [deleted]
- jghn 12y ago
  This is the real problem. The extreme majority of the "snobs" as you put it really don't have the pallet or skill to really be able to discern what they think they can, it's mostly brand loyalty, marketing, etc. There often are people who can discern these things but they're much fewer and further between than people realize. Of course, those in the first category who agree with my statements will never have their mind changed, they'll just assume that they're in the second category. The likelihood of that being true has a lot of zeros after the decimal. The problem with articles like these is that many "experts" fall into the first category, not the second. This is not without merit though as often these are the same "experts" that people trust to make buying decisions.
- applecore 12y ago
  Sorry, but that story is pretty dumb. If an expert is tricked into drinking the same shitty Indian whisky six times, they're going to be thrown off.
  - judk 12y ago
    If they don't notice something funny after 5 tastes, their expertise is not useful.
- timtadh 12y ago
  My wife tried this on me with yogurt. I like yogurt a lot and like to try every type of yogurt available in the store. She thought I just like buying the most expensive stuff. So one night she setup a blind taste test for me. I was able to distinguish and name the different yogurts (there were only a couple in the test). But, I do think that Thomas' test was a bit unfair. He arranged as if it was a test of different whiskeys but then gave the same one every time. That isn't a double blind trial and it physiologically manipulates the participant. It makes a good story but I don't think it proves anything. Furthermore, you really need a larger sample size to say whether or not "snobs" can distinguish the real thing from the fake. If you are testing just one "snob" it would help to run multiple randomized trials.
mikeryan 12y ago
What you are describing is a sommelier exam. There's an interesting documentary on some guys taking their master sommelier exam on Netflix called "Somm" who show an impressive array of these skills.
deleted 12y ago
[deleted]
shalmanese 12y ago
One cheap test I like to do when with friends is to order a flight of something, have each person taste it in turns, then have one person shuffle the glasses around while the others aren't watching, recording how they were shuffled (if you want everyone to participate, just have a second person do a second shuffle). It's then everybody's job to taste again and see if they can put the glasses back in order. So far, nobody I've done this with has performed better than chance with vodka tastings and people barely perform better than chance with red wine tastings, even when they're 3 different varietals. The test tends to come out after at least a few drinks though so most people's palates are at least somewhat compromised.

JoeAltmaier 12y ago

Doesn't have to cost much. Just choose some supposed wildly different wines, and tear up a sheet to make blindfolds. My father-in-law Thomas is from India. He likes whiskey including Indian brands; his colleague was a Scotch whisky snob. At a party he challenged the snob to name all the whiskeys on the bar. Thomas went into another room with the bottles in a bag; came out carrying a tray with half a dozen shot glasses. The snob tasted each one, make erudite remarks about their qualities and selected a brand for each one. Thomas opened the bag and showed that only the Indian bottle had the seal cracked. Needless to say Thomas doesn't get invited over there any more.

gabemart 12y ago
> Doesn't have to cost much. Just choose some supposed wildly different wines, and tear up a sheet to make blindfolds. The hard part is finding qualified experts who are willing to participate. The experiment would be most interesting and uncontroversial if it was done with experts whose credentials were unassailable.
- JoeAltmaier 12y ago
  You could find experts in your social circle. The results would be just about as satisfying personally.
- deleted 12y ago
  [deleted]
jghn 12y ago
This is the real problem. The extreme majority of the "snobs" as you put it really don't have the pallet or skill to really be able to discern what they think they can, it's mostly brand loyalty, marketing, etc. There often are people who can discern these things but they're much fewer and further between than people realize. Of course, those in the first category who agree with my statements will never have their mind changed, they'll just assume that they're in the second category. The likelihood of that being true has a lot of zeros after the decimal. The problem with articles like these is that many "experts" fall into the first category, not the second. This is not without merit though as often these are the same "experts" that people trust to make buying decisions.
applecore 12y ago
Sorry, but that story is pretty dumb. If an expert is tricked into drinking the same shitty Indian whisky six times, they're going to be thrown off.
- judk 12y ago
  If they don't notice something funny after 5 tastes, their expertise is not useful.
timtadh 12y ago
My wife tried this on me with yogurt. I like yogurt a lot and like to try every type of yogurt available in the store. She thought I just like buying the most expensive stuff. So one night she setup a blind taste test for me. I was able to distinguish and name the different yogurts (there were only a couple in the test). But, I do think that Thomas' test was a bit unfair. He arranged as if it was a test of different whiskeys but then gave the same one every time. That isn't a double blind trial and it physiologically manipulates the participant. It makes a good story but I don't think it proves anything. Furthermore, you really need a larger sample size to say whether or not "snobs" can distinguish the real thing from the fake. If you are testing just one "snob" it would help to run multiple randomized trials.

mikeryan 12y ago

What you are describing is a sommelier exam. There's an interesting documentary on some guys taking their master sommelier exam on Netflix called "Somm" who show an impressive array of these skills.

deleted 12y ago

[deleted]

shalmanese 12y ago

One cheap test I like to do when with friends is to order a flight of something, have each person taste it in turns, then have one person shuffle the glasses around while the others aren't watching, recording how they were shuffled (if you want everyone to participate, just have a second person do a second shuffle). It's then everybody's job to taste again and see if they can put the glasses back in order. So far, nobody I've done this with has performed better than chance with vodka tastings and people barely perform better than chance with red wine tastings, even when they're 3 different varietals. The test tends to come out after at least a few drinks though so most people's palates are at least somewhat compromised.