Market Research: Foundations to Practice · Advanced Quantitative Techniques
The Problem with Simple Rating Questions
Asking respondents to rate a long list of features on importance produces data that looks precise but is largely useless for prioritization. This chapter explains why, setting up the techniques the rest of this module addresses.
A common but flawed approach to feature prioritization asks respondents to rate each item on a list independently, such as rating the importance of ten different product features on a 1-to-5 scale, which frequently produces the 'everything is important' phenomenon, where most items receive similarly high ratings because rating items independently never forces a respondent to make a genuine trade-off between them [21].
Key Takeaways
- Rating a long list of items independently often produces the 'everything is important' phenomenon, since respondents face no real trade-off in rating multiple items highly.
- Independent ratings fail to discriminate between genuine priorities, since there's no cost to a respondent for rating many items similarly high.
- MaxDiff, conjoint analysis, and TURF share a common design principle: forcing genuine trade-offs rather than independent ratings.
- Simple independent ratings remain adequate for short lists or absolute satisfaction questions; trade-off techniques are justified for longer priority lists.