Judging season is upon us and I have been busy judging wines, but wine competitions have come under fire lately. LA Times writer Jerry Hirsch helped propogate the bad press by calling attention to a study conducted by retired Humboldt State professor Robert Hodgson http://articles.latimes.com/2009/jan/29/business/fi-wine29 . It’s not new to challenge the process, but this recent four year study claims that only 10% of judges could consistently judge a wine when tasted multiple times. This is not unusual information, most of us know that if you are presented three samples of which two are identical, but you are told that there is a difference among all three that it is almost unheard of to call out that two are the same. Your brain creates distinction amongst the samples. I have not seen the proof created to validate Hodgson’s claims, but I wonder about the parameters, were all the wines identical in temperature? Were all judges tested in this way? Were other competitions evaluated? What time of day were the samples presented? What about order error or the wines placed before the identical samples, this can have a dramatic effect on how that sample might taste. There are many variables to be considered.
Frankly I am not convinced about this study until I can read the brief and fully understand the methodology. I am saddened that at a time where so many in this industry are struggling an attack on judgings would hit. Judgings can really help sell wine. While Professor Hodgson says, “Consumers should have a healthy skepticism about the medals awarded to wines from the various competitions,” I believe there is nothing about a competition that makes it less useful or less accurate than the recommendations of a retailer, magazine or other wine critic. In fact I believe panels of judges can actually be more fair than one individual’s palate no matter how critically acclaimed.
Most critics judge open label while almost all competitions are 100 percent blind, ie the judges may have information as to the variety or the vintage and in rare instances price or origin, but they are not swayed by the label or by the reputation of the producer. Judging panels are usually diverse including winemakers, educators, retail and restaurant buyers, salespeople, writers and sometimes even consumers. The diversity of the panel allows for checks and balances while a writer that tastes for a review gives you only the impression of one palate.
Granted, judgings are varied in the quality of the judges and their prestige, so I generally try to evaluate which judgings are worth doing. Consumers might think to do the same thing, just as Parker may appeal to some while Tanzer appeals to others, perhaps consumers can gauge the competitions that seem to fit with their general palate preferences.
On a personal level I gauge which competitions are most enjoyable and that is usually a reflection of the calibre of judges and the sense of camaraderie amongst them. That said there are judgings that are of course well respected that I have not been invited to. Not only do judgings offer a source of information for consumer buying and give wineries medals to tout and display, but they are a great resource for wine buyers- many use them to find out what wines they like without prejudice. One of the ways I became more versed in wines of the world was wine competitions. Sometimes you might get a chance to try wines you would never choose to open like fruit wines, hybrids or varieties you might not normally gravitate towards. Judging with more experienced tasters is the best way to hone your palate, especially if you have the benefit of trying a type of wine that is their specialty. It is extremely eye-opening to evaluate wines fairly even if they are of a style of which you are not normally fond.
Most competitions treat the wines as a group amongst themselves, for example a gold medal Syrah doesn’t need to be the best Syrah you have ever had, it’s more like a county fair judging of livestock, how does the wine match up compared to the other wines entered. I have never tasted the wines he makes, but Hodgson claims that he conducted this study because wines he entered into competitions sometimes won gold medals and other times won nothing. All I can say is a girl might win a crown at her home town beauty pageant but that doesn’t make her Miss America. It all depends on the competition. 
Dallas Morning News Competition http://www.dallaswinecomp.com/ (February)
When time allows I like to take in some of the local culture. The days I was in Dallas the King Tut exhibition was in town so I was able to visit that with my friend Joel Butler, MW. Over thew weekend we were comfortably housed at the lovely Hotel Adolphus.
Dining experiences included York Street, an amazing but tiny restaurant in an unusual location (6047 Lewis) with my friends Dr. Bob Small (he makes Dr. Bob’s Handcrafted Ice Cream http://www.drbobsicecream.com/ try the Strawberries with Sour Cream & Brown Sugar) and Drew Hendricks of Pappas Bros. Steakhouse. The next night was a walkaround tasting of last year’s winning wines with some of the area’s top restaurants. Particularly notable was the White Seaweed Salad from Tei-An, so I joined Drew and our friends Meghan and Brandan there. We had the Omakase (tasting) menu including the phenomenal pairing of buckwheat tea with a truffled risotto. I also thought the soba course with egg was incredible. Tei-An 1722 Routh Street, Suite 110 Dallas, TX 75201 214 220-2828
National Women’s Wine Competition http://www.nwwc.info/ (March)
In Santa Rosa the National Women’s Wine Competition offered a unique opportunity to interact with some of the most amazing women in the industry. We hit some local dining spots such as Syrah Bistro 205 Fifth Street, Santa Rosa, CA 95401 707 568-4002 www.syrahbistro.com and of course Willi’s Wine Bar 4404 Old Redwood Highway Santa Rosa, CA 95403 707 526-3096 www.williswinebar.net
San Diego International Wine Competition http://www.sdiwc.com/
Part of my love of competitions is visiting fun places! The luxury of staying at the Westgate Hotel in San Diego can’t be beat. Add to that a visit to Old Town San Diego (missed that this time). I did make it to the amazing San Diego Zoo (www.sandiegozoo.org) where I had the chance to see BABY meerkats. That’s right BABY MEERKATS!! (I love meerkats almost as much as wine.) Let me tell you though, the pandas were pretty BORING. I also ate at El Indio a fun Mexican restaurant and tortilla factory near the airport just off N I-5 (exit Washington Street and proceed north) at 3685 India Street (619) 299-0333. It was founded in 1940. Really good chips and taquitos (they claim they first coined the word taquito.) www.el-indio.com. We also enjoyed two meals at the Yard House, one of the better chain restaurants with an unending selection of beers on tap. They carded me too, twice! Made me feel great. The gala dinner was a blast as Robert Whitley was kind enough to allow me to sing two of my songs acapella for the group to raise cash donations for Camp Oliver. Hmm, maybe they were paying me to STOP singing?! One of the highlights of this judging was an amazing little day glow pink wine that I thought tasted EXACTLY like Tootsie Rolls. While fruit wines are a unique and often scoffed at judging category there was also an amazing little strawberry wine (Saint James Winery $8.99), but this “Tootsie Roll” wine was astounding. Let’s just say that I would have been able to suck that stuff down as a Freshman in college. It was from Trout Springs Winery and called Afternoon Delight ($19.99). We sent it to sweepstakes so that everyone could taste it!
Los Angeles International Wine and Spirits Competition (formerly called Los Angeles County Fair Wines of the World Competition) http://www.lawinecomp.com Wine Judging May 27-29, 2009/Spirits June 1-2, 2009
Held in conjunction with the Los Angeles County Fair Association I have to admit this is my favorite competition of all. The number and calibre of judges is amazing. We also all know how to have a really great time. I may hold some bias as I am the Chair of the spirits side of the competition. The ability to judge with such an esteemed group cannot be beat. I learned my judging skills due to the amazing help and guidance from the best on the circuit, Don Galleano, Gary Eberle, Mitch Cosentino, Darrell Corti, Dan Berger and many others. Plus judges are often invited back to attend the Los Angeles County Fair in September where they teach consumers directly about the products they judged. This cannot be beat, plus you get to eat fried twinkies and sno cones and watch pig races and see baby animals. And Dr. Bob runs it so there is always ice cream! And Tequila! YAY!
San Francisco International http://www.sfwinecomp.com/ (June)
The San Francisco International judging is another fun event held annually by Anthony Dias Blue. There is a friendly rivalry between the SF competition and the LA, but Andy still keeps inviting me back, I am honored. He and his crew of judges are some of the funnest (I know that is not a word) in the biz. We have been known to judge hard and party hard! There has been occasional karaoke…enough said.
RebeccaChapa.com 
I couldn’t agree more about the merit of a score/judging from a panel of wine pros versus a single taster. A panel of judges will tend to balance each other out. Without question, the wines are tasted with one eye towards the winemaker (“Is it flawed or technically/varietally correct”) and the other towards the consumser (“Would Joe six pack be happy if he bought this wine”)
Even though she seems to be so young (worth asking for her ID?) Chapa has an incredible (and consistent) palate. I trust her judging results.
This consumer (OK, kinda wine-geeky) respectfully disagrees with the suggestion that wine judgings have more value than evaluations of critics. A wine gets a silver medal from a competition, but what does that tell me? Nothing. I know nothing about the panel, the setting, the rules of the particular competition. How many wines were the judges tasting over what period of time? How familiar are the judges with the varietal or region? All I know is that a random group of people with qualifications unknown to me, using undisclosed (or not readily available) criteria, gave the wine a silver. I have no means of benchmarking. And, as occasionally happens, I’m familiar with the wine, and think it is much better than the other silvers in the category, as well as better or equal to those awarded golds (A “What Were They Thinking??” moment). Doesn’t leave me with the warm fuzzies regarding the competition. And even if it’s a big name competition, what are the odds that the same panel will judge the same varietals in successive vintages? Well, I have no way of knowing. Seems pretty random to me.
Compare this knowledge vacuum to what I get from the four or five critics that I follow (all of whom taste blind, by the way). I have a pretty good idea of their likes and dislikes, and know where we agree and disagree. I can benchmark against their evaluations, and know that I typically agree with critic A on Zin, critic B on Pinot Noir, critic C on Italian reds, and so on (as well as who I disagree with on various varietals). And, importantly, it’s not the score that I look at, it’s the description of the evaluation that supports the score. An 89 from Wine Enthusiast, for example, doesn’t mean much to me until I drill down to the see who the critic was and what their review had to offer. But at least I can get at this information, so in reality, an 89 from WE means a lot more to me than a silver medal from wherever.
I’m curious. Ron says that he trusts your judging results. But how would he know it’s you? Wouldn’t you be on a panel, with aggregated results? Don’t see how that could work…unless you talk about the wines you judged on your blog or elsewhere. Should this be the case, under my criteria you then morph from judge to critic….
Thank you for your thoughtful comment. I totally understand your valid points. All the competitions that I mention have a list of judges and their biographies and credentials as well as information regarding the methodology they use on their websites. While I understand that this process may seem random to you, remember that an individual palate is very random naturally. Does the critic in question disclose whether they ate spicy Thai food the night before the tasting or whether they were having a fight with their spouse? Did Fedex arrive in the middle of their tasting? Did they have a deadline to meet so they had to taste more wines than usual that day? I know that the most reputable critics try to keep their tasting as consistent as possible, but there is no way to be 100% consistent whether in a judging or as an individual critic.
Also please remember that I was not speaking as much about an educated consumer such as yourself who researches wines before they go to a wine store, I was thinking more in terms of a less savvy consumer who is at Safeway and needs to know on the fly what wine to grab, or maybe a buyer for a retailer chain that has less experience and less opportunity to taste the wines themselves. They do not read four or five critics reviews and cross reference them with their own experiences. Obviously the ideal scenario would be for the consumer to determine what they like themselves, but we know that is not generally the norm.
Critics can be helpful but I find that all too often their critiques rely on flowery language rather than on what the structure, balance, alcohol, tannin and acidity in the wine is like. This is my pet peeve. If you were to read a review “blind” I would think you would be hard-pressed to figure out what the wine was. While some critics taste blind the consumer that buys according to critical review does not BUY blind. Which of these three Spectator rated wines would you buy? And how much would you pay?
WINE A”Ripe, plush and deeply concentrated, with tiers of currant, plum, blackberry and spicy cedary oak. Firmly tannic, lively and deep revealing extra flavor nuances on the long engaging finish. Best from 2008 through 2013. 11, 800 cases made. J.M.”
WINE B”Very ripe, but also racy with an exotic array of graphite, tar, sweet dark chocolate, plum, fig and violet. Has great structure and drive through the grip-filled finish. Drink now through 2009. 3,000 cases made, J.M”
WINE C”Firm and well-structured, exhibiting ripe raspberry, wild berry and blackberry flavors that are supported by ripe, integrated tannins. Ends with a pepper and cedar note. Drink now through 2011. 2, 213 cases made. J.L.”
Do these wines sound similar to you? The pricing, varieties and regions are all different (all are New World Wines though.)
Here’s a Decanter review of one of the wines above, same vintage… can you match it to the wine? “Intense spice, dark fruit and liquorice, rich concentrated cherry and damson fruit with vanilla oak and fresh acidity, stylish. 2-7 years (AR)”
Ron judges with me in San Diego. I think he just meant to say that he is aware that I have extensive tasting experience and as such I am a credible judge. If you were to look around the room in San Diego you would see many judges more experienced than I as well a handful that are just starting out. While scores are aggregated the panels are managed by the head judge to ensure that the veteran judges are able to show the newer judges the ropes. In San Diego we were even shown our individual medal count to give us an idea of how overly harsh or overly generous we were (I was on par). American Idol would not work if you had three Paula’s judging and no Simon, similary judgings try to create checks and balances.
Sadly there is no one good way to suggest wines for a consumer. I just did not appreciate the backlash for an activity that, along with critics, attempts to help the consumer. Each consumer will make their own choice as to what matters to them, but they should have as many options open to them as possible. The Los Angeles International Wine Competition has been judging wine for seventy years. They bring wine to the consumer directly through extensive educational tastings at the fair. They help to promote spirits and olive oils in their judgings now too. They give me an opportunity to taste numerous wines that would never have crossed my lips otherwise. They do much more than just dole out bronze, silver and gold medals. Why mess with a good thing?
PS The Decanter wine matched WINE B
Hi Rebecca, Found a link to this article at wine review on line. I have not had the time to read and am runnig out the door, but will. Just wanted to give you a heads up regarding Tim Patterson’s article in the current issue of Wines and Vines on the issue and the study done at Pooch’s California State Fair Competition.
[…] of gauging wine quality. {For an interesting take on the nature of competition wine-judging, check this post at http://www.rebeccachapa.com.} Proliferation of medals—from diverse sources—compounded the […]