MaxDiff
Encyclopedia
Maximum difference scaling (MaxDiff) is a discrete choice
model first described by Jordan Louviere in 1987 while on the faculty at the University of Alberta
. The first working papers and publications occurred in the early 1990s. With MaxDiff, survey respondents are shown a set of the possible items and are asked to indicate the best and worst items (or most and least important, or most and least appealing, etc.). According to Louviere, MaxDiff assumes that respondents evaluate all possible pairs of items within the displayed set and choose the pair that reflects the maximum difference in preference or importance. MaxDiff may be thought of as a variation of the method of Paired Comparisons
. Consider a set in which a respondent evaluates four items: A, B, C and D. If the respondent says that A is best and D is worst, these two responses inform us on five of six possible implied paired comparisons:
The only paired comparison that cannot be inferred is B vs. C. In a choice among five items, MaxDiff questioning informs on seven of ten implied paired comparisons. MaxDiff questionnaires are relatively easy for most respondents to understand. Furthermore, humans are much better at judging items at extremes than in discriminating among items of middling importance or preference . And since the responses involve choices of items rather than expressing strength of preference, there is no opportunity for scale use bias
.
In 1938 Richardson introduced a choice method in which subjects reported the most alike pair of a triad and the most different pair. The component of this method involving the most different pair may be properly called “MaxDiff” in contrast to a “most-least” or “best-worst” method where both the most different pair and the direction of difference are obtained. Ennis, Mullen and Frijters (1988) derived a Thurstonian scaling model for Richardson’s method of triads so that the results could be scaled under normality assumptions about the item percepts. Richardson’s method of triad and most-least methods belong to a class of methods that do not require the estimation of a cognitive parameter as occurs in the analysis of ratings data. Other methods in this class include the 2- and 3-alternative forced choice methods, the triangular method, the duo-trio method and the specified and unspecified methods of tetrads. All of these methods have well-developed Thurstonian scaling models. There are a number of possible processes through which subjects may make a most-least decision, including paired comparisons and ranking, but it is not known how the decision is reached.
Max Diff on the other hand forces respondents to make choices between options, while still delivering rankings showing the relative importance of the items being rated.
analysis, in particular multinomial logit
. Several algorithms could be used in this estimation process, including maximum likelihood
, neural network
s, and the Hierarchical Bayes model
. The Hierarchical Bayes model
is beneficial because it allows for borrowing across the data.
Discrete choice
In economics, discrete choice problems involve choices between two or more discrete alternatives, such as entering or not entering the labor market, or choosing between modes of transport. Such choices contrast with standard consumption models in which the quantity of each good consumed is assumed...
model first described by Jordan Louviere in 1987 while on the faculty at the University of Alberta
University of Alberta
The University of Alberta is a public research university located in Edmonton, Alberta, Canada. Founded in 1908 by Alexander Cameron Rutherford, the first premier of Alberta and Henry Marshall Tory, its first president, it is widely recognized as one of the best universities in Canada...
. The first working papers and publications occurred in the early 1990s. With MaxDiff, survey respondents are shown a set of the possible items and are asked to indicate the best and worst items (or most and least important, or most and least appealing, etc.). According to Louviere, MaxDiff assumes that respondents evaluate all possible pairs of items within the displayed set and choose the pair that reflects the maximum difference in preference or importance. MaxDiff may be thought of as a variation of the method of Paired Comparisons
Pairwise comparison
Pairwise comparison generally refers to any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some quantitative property. The method of pairwise comparison is used in the scientific study of preferences, attitudes, voting systems, social...
. Consider a set in which a respondent evaluates four items: A, B, C and D. If the respondent says that A is best and D is worst, these two responses inform us on five of six possible implied paired comparisons:
- A > B, A > C, A > D, B > D, C > D
The only paired comparison that cannot be inferred is B vs. C. In a choice among five items, MaxDiff questioning informs on seven of ten implied paired comparisons. MaxDiff questionnaires are relatively easy for most respondents to understand. Furthermore, humans are much better at judging items at extremes than in discriminating among items of middling importance or preference . And since the responses involve choices of items rather than expressing strength of preference, there is no opportunity for scale use bias
Response bias
Response bias is a type of cognitive bias which can affect the results of a statistical survey if respondents answer questions in the way they think the questioner wants them to answer rather than according to their true beliefs...
.
In 1938 Richardson introduced a choice method in which subjects reported the most alike pair of a triad and the most different pair. The component of this method involving the most different pair may be properly called “MaxDiff” in contrast to a “most-least” or “best-worst” method where both the most different pair and the direction of difference are obtained. Ennis, Mullen and Frijters (1988) derived a Thurstonian scaling model for Richardson’s method of triads so that the results could be scaled under normality assumptions about the item percepts. Richardson’s method of triad and most-least methods belong to a class of methods that do not require the estimation of a cognitive parameter as occurs in the analysis of ratings data. Other methods in this class include the 2- and 3-alternative forced choice methods, the triangular method, the duo-trio method and the specified and unspecified methods of tetrads. All of these methods have well-developed Thurstonian scaling models. There are a number of possible processes through which subjects may make a most-least decision, including paired comparisons and ranking, but it is not known how the decision is reached.
Process
The basic steps are:- select products to be tested
- show products to potential consumers (textually or visually)
- respondents choose the best and worst from each task
- input the data from a representative sample of potential customers into a statistical software program and choose the MaxDiff analysis procedure. The software will produce utility functions for each of the features. In addition to utility scores, you can also request raw counts which will simply sum the total number of times a product was selected as best and worst. These utility functions indicate the perceived value of the product on an individual level and how sensitive consumer perceptions and preferences are to changes in product features.
Why use maximum difference scaling?
Max Diff is an antidote to standard rating scales or importance scales. Respondents find these ratings scales very easy but they do tend to deliver results which indicate that everything is "quite important", making the data not especially actionable.Max Diff on the other hand forces respondents to make choices between options, while still delivering rankings showing the relative importance of the items being rated.
Analysis
Estimation of the utility function is performed using multinomial discrete choiceDiscrete choice
In economics, discrete choice problems involve choices between two or more discrete alternatives, such as entering or not entering the labor market, or choosing between modes of transport. Such choices contrast with standard consumption models in which the quantity of each good consumed is assumed...
analysis, in particular multinomial logit
Multinomial logit
In statistics, economics, and genetics, a multinomial logit model, also known as multinomial logistic regression, is a regression model which generalizes logistic regression by allowing more than two discrete outcomes...
. Several algorithms could be used in this estimation process, including maximum likelihood
Maximum likelihood
In statistics, maximum-likelihood estimation is a method of estimating the parameters of a statistical model. When applied to a data set and given a statistical model, maximum-likelihood estimation provides estimates for the model's parameters....
, neural network
Neural network
The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
s, and the Hierarchical Bayes model
Hierarchical Bayes model
The hierarchical Bayes model is a method in modern Bayesian statistical inference. It is a framework for describing statistical models that can capture dependencies more realistically than non-hierarchical models....
. The Hierarchical Bayes model
Hierarchical Bayes model
The hierarchical Bayes model is a method in modern Bayesian statistical inference. It is a framework for describing statistical models that can capture dependencies more realistically than non-hierarchical models....
is beneficial because it allows for borrowing across the data.
See also
- Conjoint analysis (in marketing)Conjoint analysis (in marketing)Conjoint analysis is a statistical technique used in market research to determine how people value different features that make up an individual product or service....
- marketing researchMarketing researchMarketing research is "the function that links the consumer, customer, and public to the marketer through information — information used to identify and define marketing opportunities and problems; generate, refine, and evaluate marketing actions; monitor marketing performance; and improve...
- quantitative marketing researchQuantitative marketing researchQuantitative marketing research is the application of quantitative research techniques to the field of marketing. It has roots in both the positivist view of the world, and the modern marketing viewpoint that marketing is an interactive process in which both the buyer and seller reach a satisfying...
External sources
- Cohen, Steve and Paul Markowitz (2002), “Renewing Market Segmentation: Some New Tools to Correct Old Problems,” ESOMAR 2002 Congress Proceedings, 595-612, ESOMAR: Amsterdam, The Netherlands.
- Cohen, Steve (2003), “Maximum Difference Scaling: Improved Measures of Importance and Preference for Segmentation,” 2003 Sawtooth Software Conference Proceedings, Sequim, WA.
- Ennis, Daniel M, Kenneth Mullen, and Jan E.R. Frijters. (1988). "Variants of the method of triads: Unidimensional Thurstonian models," British Journal of Mathematical and Statistical Psychology, 41, 25-36.
- Richardson, M.W. (1938). "Multidimensional psychophysics," Psychological Bulletin, 35, 659-660.
- Louviere, J. J. (1991), “Best-Worst Scaling: A Model for the Largest Difference Judgments,” Working Paper, University of Alberta.
- Thurstone, L. L. (1927), “A Law of Comparative Judgment,” Psychological Review, 4, 273-286.