Editor’s note: Today we publish the final post in our series on the seven sins of consumer psychology from the presidential address of professor Michel Tuan Pham at the recent conference of the Society for Consumer Psychology. Read the introduction here.

The final sin is certainly not a recent one, but it is still a major one. More than 35 years ago, in a JCR editorial titled “Research by Convenience,” Robert Ferber (1977) already complained about the over-reliance on student samples in studies purported to be about consumers in general. Ferber questioned whether students were really the right respondents for certain topics such as financial decision making or family purchases. He also questioned the degree to which, independent of the topic, results obtained from college students could be generalized to the broader population of consumers that our samples are meant to represent (see also Sears 1986).

A variant of the “Research by Convenience” criticism includes complaints that too much of our research is North-American-centric (Gorn 1997)—a criticism that has also been made about psychology in general (Arnet, 2008). Another variant includes the criticism that too much of our theorizing is based on the upper end of the knowledge-expertise continuum, whereas large segments of the consumer population are bound to be less educated and less “intelligent” than the student population that we typically sample in our studies (Alba 2000).

On the surface, it would appear that new sources of inexpensive experimental respondents such as Mechanical Turks, which has become very popular in our field, should help address this research-by-convenience problem. Indeed, from a demographic point of view, MTurk participants seem to be a little more like “real consumers” than the typical college undergrad (Berinsky, Huber, and Lenz, 2012). There is also some evidence that some well-known judgment biases can be replicated on MTurk participants (Goodman). However, before we declare the “sin of research by convenience” partially absolved by MTurks, we need to temper our optimism in three respects. First, regardless of what has been shown or claimed to date, it is not clear to me that a particular sample of individuals who self-selected to participate in this peculiar marketplace—that is, individuals who are willing to perform computer-mediated mindless tasks for a couple of dollars an hour—are necessarily that more representatives of “real-world” consumers than are typical college undergrads. Second, there is disturbing evidence of increased MTurk sophistication in seeing through and “gaming” our studies (Chandler, Mueller, and Paolacci ACR 2012). Finally, and most seriously, I see a real danger that the low data collection costs of Mechanical Turk is gradually shifting our research agendas toward studies than can be done on MTurks—i.e. short online, survey-type experiments—as opposed to studies that should be conducted to advance our field. This last point taps into another meaning to the phrase “Research by Convenience”—one that Ferber did not discuss, but is, in my opinion, perhaps even more serious.

Finally, I should be noted that the sin of research by convenience is not limited to the convenience of the sample of respondents that we study. It extends to the convenience of the instruments that we use to study them. Instead of studying actual consumption behavior, much of our research is based on vignette-like studies, in which respondents are asked to imagine a certain consumption situation and report how they would respond in such situations. The real question is whether the observed responses in these studies are good representations of the actual responses that we would observe had actual consumption behavior been analyzed.

Our colleagues in economics often criticize such studies because vignette-based responses entail no costs and no rewards. “Without some incentive compatibility,” they would say, “this is just cheap talk.” I am not sure that this is the main problem, however. My concerns are a bit different. First, scenario-based studies tend make the focal aspect of the treatment very prominent (e.g., “imagine buying insurance two years from now vs. next month”), thereby potentially exaggerating the strength of the effects. Second, I suspect that participants who are asked to project themselves into a certain consumption situation tend to adopt an overly analytical mindset that is not representative of how consumers would actually respond to the situation in real-life (see, e.g., Dunn & Ashton-James, 2008; Snell, Gibbs, & Varey, 1995, for relevant findings). Finally, I believe that scenarios are poorly-suited for the studying of the effects of “hot” variables such as emotional responses and motivational states (Pham, 2004), whose influence on our behavior is difficult to imagine with a genuine experience .

Conclusions: Increasing our Relevance and Impact

General

1. Expand our research focus to non-purchase dimension of consumer behavior, especially need and want activation, nonpurchase modes of acquisition (sharing, borrowing, stealing), and every aspect of actual consumption.
2. Embrace broader theoretical perspectives on consumer behavior beyond information processing and BTD, especially motivation, social aspects, and deep cultural aspects (as opposed to cross-cultural aspects). Less emphasis on unique and micro-level explanations.
3. Expand our epistemology to encourage (a) further phenomenon-based research (provided that phenomenon is robust and really grounded in CB), (b) more descriptive research, and (c) tests of popular industry theories
4. Greater attention to content aspects of CB with corresponding increase in domain specificity (and decrease in presumed generality). Key opportunity in area of motivational content.
5. Lower tolerance of theories of studies.
6. Greater emphasis on replication, robustness, and sensitivity testing.
7. Greater reliance on studies with real consumers, as opposed to students or Mturks. Encouragement of field studies. Decreased reliance on scenarios, especially when studying hot processes of CB.

Practical

1. CB syllabi need to be revamped (especially those structured in terms of information processing and JDM) to reflect broader theoretical perspectives
2. Greater substantive grounding in how we teach CB to our graduate students
3. Encourage PhD students to take or TA MBA-level course in CB and in basic marketing
4. Encourage to a limited extent (rather than strongly discourage) activities that strengthen our grounding in and understanding of business issues (executive teaching, consulting, book writing).
5. Pay more attention to citations and impact as opposed to mere number counting in promotions. (Simple new metric proposed: average citation percentile rank in given journal in given year)

Editor’s comments: We’d like to thank Michel Tuan Pham for kindly letting us publish his presidential address, and would welcome comments from readers. What do you think about these sins?

Editor’s note: Today we continue our series on the seven sins of consumer psychology from the presidential address of professor Michel Tuan Pham at the recent conference of the Society for Consumer Psychology. Read the introduction here.

The next sin that I want to point out is a sin of overgeneralization. This is a sin that we commit both as authors and as reviewers and readers of the literature.

As authors, we tend to overgeneralize from our limited data. We all know that getting an experiment to work takes a lot of effort. We have to think very carefully about the consumption context we use, the product category, the precise stimuli, the exact procedure, the measures that are most likely to pick up the effect, etc. We often run pretest after pretest, and still may need to try different versions of the experiment to eventually get it to work. We all do that. And that’s fine. However, once a study eventually works as we intended, we quickly develop a supreme confidence in our own results and interpretation, forgetting how much effort it took us to get the effect in the first place. As a result, we tend to believe that our findings are more general and robust than they actually are. The psychology is akin to the fundamental attribution error: we quickly attribute significant data patterns to some trait-like pet theory, forgetting to account for the multitude of contextual factors that may have contributed to this data pattern.

A related issue, of course, is the issue of how transparent is our reporting of the fragility of the results that we produce. This in itself would require an entire address; so I prefer to leave it aside for now.

We also have similar tendency when we read the literature and, to a lesser extent, when we review papers.

Once an effect has been reported in a published paper (especially if it is by a famous author in a prestigious journal), we tend to treat it as gospel, again forgetting that this effect may be more context-specific than a quick reading of the paper may reveal. Moreover, we often generalize the result well beyond the researchers’ original interpretation. As a result, we walk around with oversimplified theories of the world that we use indiscriminately. This impedes scientific progress because, all too often, research ideas are rejected and findings are simply dismissed because we have an unwarranted feeling that “we already know that.”

A good example is the work on the “too-much-choice effect” by Sheena Iyengar: the famous-jam-in- supermarket study. This is a great study and the results are very important. However, if you read the paper carefully, you will see that the effect is very specific and that the authors were very particular in the way they conducted the study. For example, all the jams were from a single brand and only unfamiliar flavors were selected. Yet after the paper was published and received a lot of attention, the fine prints of the study—which the authors carefully disclosed in the paper—were quickly forgotten, and the field began to take it for granted that “consumers do not like when they have too much choice.” It turns out, however, that the effect is very fickle. In a meta-analysis by Scheibehenne and colleagues (2010), the basic effect was found in only a small number of studies; the reverse effect was found in an equal number of studies; and the majority of studies showed no significant effect.

The same issue arises when we review papers. Very often, we dismiss a particular finding based on a loose impression that “we already know that” and “this has already been shown,” without necessarily appreciating that there may be nontrivial distinctions between the new study and the ones that it reminds us of. We also exhibit the same fallacy when authors replicate their findings in a separate study and we ask them to drop the replication study because “we already know that from the other study.”

The answer to this pervasive problem of overgeneralization is very simple. We need more replications and more nuance. As authors, we need to be more willing to replicate our own results: across different samples of respondents, across different stimuli, across different operationalizations of the manipulations, etc. Ideally, these conceptual replications should be done in a way that alters only one variable at the time. When too many variables are changed at the same time across studies (which we often do), we defeat the primary purpose of replication which is to assess the robustness of a result to small differences in method that are theoretically-meaningless. As authors, we should also be more willing to increase the sample size of our studies. All too often, researchers seem to be unwilling to run more subjects, apparently out of fear that the effect might go away. My philosophy is that if we really believe in our effects, we should not be afraid of increasing our sample size.

We also need to be more careful and nuanced in our writing and discussion of our results.

As readers of the literature, we need to be more mindful of what studies actually show and how they were actually conducted. We also need to be more appreciative of studies that, on the surface, merely seem to replicate conceptually what previous studies had shown.

And as reviewers and editors, we need to be much more supportive of close replications within papers, and conceptual replications across papers—these are not wasted journal space.

Ready for the final sin?

Editor’s note: Today we continue our series on the seven sins of consumer psychology from the presidential address of professor Michel Tuan Pham at the recent conference of the Society for Consumer Psychology. Read the introduction here.

A serious problem in our field is a pervasive confusion between what I call “theories of studies” and “studies of theories.” What I mean by this is that what passes off as an empirical study testing theoretical propositions about genuine consumer behavior phenomena is often no more than a conceptualization of a very narrow phenomenon that is created by the study itself.

The studies that I am referring to go like this:

“Past research has shown that when consumers are hungry, they tend to purchase more when they go to the supermarket. However, suppose that we prime them with either the concept of indulgence or the concept of self-control. We predict that hungry consumers primed with the concept of indulgence will purchase more than consumers who are not hungry. However, hungry consumers primed with the concept of self-control will not purchase more than consumers who are not hungry. This because … “

[You can fill in the blank]. That study might be followed-up with another study going like this: “Now suppose that half the participants are asked to remember a 7-digit number and the other half are asked to remember a 2-digit number. Now we predict a 3-way interaction showing that among participants asked to remember a 2-digit number, the results of the previous study would be replicated; in contrast, among participants asked to remember a 7-digit number, hungry participants would purchase more, regardless of whether they are primed with indulgence or self-control.”

I don’t think that this study has been done yet – and I hope that did not unintentionally describe one of our own studies here. However, you will recognize that this pair of studies has some of the qualities of what often gets published in our main journals today: (a) it leverages previous established findings; (b) it combines them in a way that has not been done before, and (c) it yields predictions that are very plausible. If the researchers are reasonably competent, (d) the studies will be conducted in a way that is free from major methodological confounds; and (e) as a result, the reviewers will have a hard time coming up with a better explanation than the one that authors propose. This is a type of paper that is not easy to reject on methodological grounds. From my own observations, it often does not get rejected on “theoretical” ground either, because the conceptualization seems logical and the data pattern is not open to obvious rival interpretations. The only way that such a paper would get rejected is if some of the reviewers are blunt enough to state that “the research is not interesting,” which most of us are generally reluctant to do, at least openly.

However, let’s have a closer look about what such a pair of studies would actually show. I don’t think that it would tell us anything meaningful and general about genuine consumer behavior. At best, what it would only show is a sound and logical explanation of a very narrow phenomenon that can only happen under the very contrived conditions that the researchers created in the lab. Any conceptual claim for such studies is just a mini-theory (with a lower case t) of the authors’ particular studies. They are the behavioral equivalent of analytical marketing models that are based on very implausible assumptions: the math may be correct, but the results tell us very little beyond the model itself.

As authors and reviewers, we need to pay more attention to this distinction and recognize when a paper consists of genuine studies of consumer relevant theories as opposed to mere “theories of studies.”

Move on to the penultimate sin…

Editor’s note: Today we continue our series on the seven sins of consumer psychology from the presidential address of professor Michel Tuan Pham at the recent conference of the Society for Consumer Psychology. Read the introduction here.

Somewhat related to the preceding issue, a strongly limiting aspect of our theorizing is our almost exclusive emphasis on uncovering psychological processes, as opposed to understanding the mental contents on which these processes operate. For example, we know that consumers are more likely to learn empirical correlations among product attributes if the data are consistent with the consumers’ prior beliefs (e.g., price and quality are positively correlated) than if the data are inconsistent with their prior beliefs (e.g., price and quality are negatively correlated) (Broniarczyk & Alba, 1994). However, we do not typically ask ourselves what are consumers’ prior beliefs about correlations among product attributes. Similarly, research on affect generally tells us about when and how affective feelings are likely to be used in judgments and decisions (Pham, 2004). However, this research is largely silent about the actual content of consumers’ feelings. Likewise, recent research on consumer motivation focuses primarily on the mechanics of how consumers pursue their goals, as opposed to the content of the goals that consumers are pursuing.

In other words, in our conceptualization of consumer behavior—whether it is information processing, judgment and decision making, affect, or motivation—what seems to matter mostly to us is the process. This is why we typically use the phrase “explanation” and “process explanation” interchangeably. The actual content of consumers’ thoughts, feelings, and motives doesn’t really seem to matter.

For our field to grow in terms of relevance, it is important that we pay much more attention to issues of content. Consumer behavior is a substantive field. Attempting to explain consumption behavior without reference to the content of consumers’ motives, feelings, actions, and thoughts is extremely impoverishing. When companies or policy makers want to understand consumers, they really want to know what consumers do, what consumer think, what they want, and what they feel—that is, content. They don’t really want to know how each of these things happen, which is what we mostly focus on as a field.

While many branches of psychology also have a process orientation (e.g., cognitive psychology, social psychology, experimental psychology), we should not forget that some of the most influential theories and contributions in psychology were largely about content rather than process per se. Think of Maslow’s pyramid of human needs, Rokeach’s typology of values, Milgram’s work on obedience, Fiske and Norman’s and “Big Five” dimensions of personality, and Hofstede’s dimensions of cultural difference.

My most cited paper is a paper with Raj Raghunathan on the motivational effects of anxiety versus sadness on decision making (Raghunathan & Pham, 1999). I didn’t realize it back then, but in retrospect, the reason why this paper has some impact is because it made salient the differences between anxiety and sadness in terms of motivational content. It was Raj’s idea, not mine, to position the paper that way. Had Raj listened to my original idea, which was to position the paper as an affect-as-information paper, a more process-oriented positioning, the paper would not have had the impact that it had.

Raj also showed good taste for important content issues in his work with Rebecca Naylor and Wayne Hoyer on the “unhealthy equal tasty” intuition (Raghunathan, Naylor, and Hoyer, 2006). This paper is one of the most cited papers in JM that year. What is driving the influence of this paper is not something special about the way this particular intuition operates, it is the fact that this intuition exists in the marketplace and shapes consumption behavior substantially.

Similarly, Goldstein, Cialdini, and Griskevicius (2008) recently conducted some field experiments to test how different forms of requests place on bathroom signs affected the likelihood that hotel guests would reuse their towels. They found that requests using social norms, such as “The majority of guests reuse their towels,” were substantially more effective than the requests frequently used by hotels. Conceptually, this paper was very simple: it was about finding out which content in terms of appeal worked better in the marketplace. Yet, the paper went on to receive JCR’s best article of award and is by far the most cited article in that journal that year.

One of the most fascinating readings about consumer psychology is an old book by Ernest Dichter (1964) called the Handbook of Consumer Motivations. Dichter, as you may know, is considered the “father of motivation research.” Yet Dichter’s approach to consumer motivation was totally different from how we study motivation today: it was all about content. Dichter was a Freudian. He believed that behind every consumption object, whether asparagus, wooden floors, perfume, or toothpaste, lies a symbolic meaning that is deeply rooted in our unconscious motives. For example, he suggested that large kitchen appliances, such as the fridge and the stove, are really surrogates for men to express their deeply rooted desire to be seen as providers for their family. This could explain why there is a strong preference for fridges and stoves that are masculine-looking (large, square-shouldered, and stainless steel). In his book, Dichter offered similar analyses for hundreds of product categories. Although one may quarrel with some of his analyses, I personally found many of them insightful and rather convincing, even though they were written some 50 years ago.  My major takeaway from Dichter’s work is that if you really want generate useful insights about consumer behavior, you really need to pay more attention to matters of content.

Therefore, I would urge us to pay more attention to the content of consumers’ thoughts, feelings, motives, and actions. This is what our external constituents really need, and this is something that is likely to enhance our collective scholarly impact. Doing so requires that we suspend our search of psychological universals, and be more willing to ground our theorizing into certain consumption contexts. This is essentially what Raghunathan, Naylor, and Hoyer (2006) did in the food domain, and what Goldstein, Cialdini, and Griskevicius (2008) did in the environment-conservation-in-hotels domain. We may not necessarily want to be as granular and specific as Dichter was, but we cannot afford to be as generic as we have been. Another example is in cross-cultural consumer research, if we really want to say something meaningful about Asian or Middle-eastern consumers, we need to go beyond our broad distinction such as individualism/collectivism and capture some of the rich market and cultural content differences between say Korean consumers and Indonesian consumers or Dubai vs. Saudi Arabia.

Ready for sin #5?

Editor’s note: Today we continue our series on the seven sins of consumer psychology from the presidential address of professor Michel Tuan Pham at the recent conference of the Society for Consumer Psychology. Read the introduction here.

Our field is also too narrow in the way it defines what consumer knowledge is and how it should be generated. Most of the knowledge that we produce is of the relational type, whereby we link theoretical constructs to one another, say, expertise and depth of search, and then use these theoretical relations to explain some substantive phenomenon of interest. Within this dominant epistemic view, the primary path, by far, has been the hypothetico-deductive (or “theory-driven”) path, whereby constructs are first related to one another to generate some theoretical hypotheses, which are then tested with empirical data that are meant to capture a phenomenon of interest (Lynch et al. 2012). As summarized by Lynch et al. (2012), one of the major criticisms of this style of research is that because the primary emphasis is on testing construct-to-construct relations, the connection between the tested theoretical relations and genuine consumer behavior phenomena is often tenuous. Such papers are often criticized for being pure psychology rather than about consumer behavior. Still many of these papers continue to appear in our journals because this style of research tends to be evaluated more based on the rigor of the test than based on the interestingness or importance of the phenomenon to be explained.

Partly as a reaction to this criticism, there have been recent calls to pursue an alternative path to knowledge generation that is more phenomenon-driven (or effect-based) (Janiszewski 2009; Deighton et al. 2010; Lynch 2011, Park 2012). This alternative path is also relational, but it starts with empirical observations that seem to generalize, which the researchers then try to explain theoretically through a process of induction. Good examples would be research on the attraction effect (Huber, Payne, and Puto, 1982), the mere measurement effect (Morwitz, Johnson, and Schmittlein, 1993), and recent work on various embodiment effects (Hong & Sun, 2012; Lee & Schwarz, 2010). I personally find this expansion of our epistemology healthy, provided that the phenomena are robust and meaningfully related to consumer behavior, which are non-trivial caveats.

Still, our epistemology should be expanded yet further. Whether we choose the traditional hypothetico-deductive path or the more phenomenon-driven path, most of our research is still about documenting some relation between phenomena and theoretical constructs and between constructs. Whenever such theoretical relations are missing, we usually look down at the empirical observations, disparaging them as mere “descriptions” (Alba 2012). Yet, one could argue that some of the most useful insights about consumer behavior came from studies that were essentially descriptive. Classic examples include the Hoyer (1984) observation that consumers actually do very little search when in the aisle of a supermarket, the Dickson and Sawyer (1990) finding that grocery shoppers have very poor knowledge of the products they bought, and Hauser’s findings that typical consideration sets contain between 3 and 6 product alternatives and account for most of the uncertainty in product choice (Hauser 1978; Hauser & Wernerfelt, 1990) (see Lynch, 2011 and Lynch et al. 2012). Interesting descriptive observations have also been advanced by applied consumer researchers such as Paco Underhill (2008). For example, Underhill describes the notion of a “decompression zone” when typical shoppers enter a retail store. The decompression zone is the area (both in time and space) that the consumer needs to switch mentally between the outside of the store and being ready to start shopping. According to Underhill, information that is presented within the decompression zone, is effectively tuned out by the consumer. Although I am not aware of any “scientific” test of this notion, field observations by myself and by my MBA students suggest that the phenomenon is real. This is an example finding that has enormous managerial relevance and yet is treated as having low status within our field. We need to have greater tolerance for descriptive research if it tells us something that (a) we didn’t really know about consumers, (b) is general, and (c) is useful from a substantive standpoint.

Another way in which I recommend that we expand our epistemology is the production of our theoretical hypotheses. When we use the hypothetico-deductive path, we usually generate the hypotheses ourselves based on our understanding of prior academic literature combined with our own substantive observations and intuitions. My impression is that we are pretty good at generating hypotheses that are logically sound and consistent with the literature. However, we are not equally good at generating hypotheses that are important from a substantive standpoint. On the other hand, there are other people who are pretty good at generating hypotheses that are important from a substantive standpoint, but not good at testing their hypotheses. The people that I am referring to are industry consultants. The marketing industry is full of consultants who generate their own “theories” about consumer behavior—theories that are tailored to be seen as relevant by corporate executives and policy makers. Yet, most of these so-called “theories” are not real theories in the sense of being supported by rigorous empirical evidence or prior scholarly literature: they are just speculations that businesses find interesting and believable. I think that these speculations are a treasure trove of interesting hypotheses that we should be willing to test as a field. Rigorous testing is something that we are very good at, much better than the industry. It is our responsibility, as a field, to verify or disprove some of these industry-generated “theories” about consumer behavior.

The new epistemic path that I propose, therefore, is one in which we let industry professionals generate their own relevant “theories”—in other words, we “outsource” that part—but then seize on the more popular “theories” to transform them into inspired testable hypotheses that we next submit to empirical validation. Even though we may not generate the original theory, we would still perform a very important function as consumer scientists: that of providing empirical validity to or debunking widely used “theories.” A recent but rare example of research that followed this inspired validation path is a paper by Brett Martin (2012) in JCR, in which the author tested a phenomenon first identified by Paco Underhill called the “butt-brush effect.” In his book, Underhill (1998) reported noticing that whenever a female consumer is accidentally touched from the back by another consumer while shopping, she tends to interrupt her shopping and often leaves the area. In a carefully conducted field experiment, Martin (2012) verified that it was indeed the case. I was not involved in the review process for this paper, but I am pleased that it was published because such papers need to be encouraged rather than discouraged simply on the ground that the original theory emerged from the industry.

Ready for for sin #4?