We examine the degree of consensus in quality ratings of US among prominent wine publications. Ratings are an important source of information for both wine consumers and wine researchers. For the purpose of wine research, are ratings on the ubiquitous 100 point scale reliable, objective measures of quality? The value of expert judgment has been called into question by a number of studies, especially in the context of wine competitions and tasting events. Our study is part of a much smaller literature focusing on ratings by expert critics. We look at four publications: Wine Spectator (WS) and Wine Enthusiast (WE), which review a broad selection of the wine market, and Wine Advocate (WA) and International Wine Cellar (IWC), which are more selective and focus more on the high-end of the market. We find a similar level of consensus, measured by the correlation coefficient, between some pairs of critics regarding wines from California and Washington as Ashton (2013) does for critics of Bordeaux wine. However, among other pairs the correlation is much lower, suggesting almost no consensus. Consensus is not found to be related to the blinding policies (or lack thereof) of the critical publications. Our findings show that quality ratings have a substantial degree of objectivity to them.
This article was written by Karl Storchmann