Leadership Institute
Date of this Version
2006
Abstract
In many ways, Campbell and Fiske’s (1959) article on multitrait-multimethod (MTMM) approaches to construct validity has stood like a Platonic ideal for personality psychologists since its publication. In the ideal study, and scientific world, our constructs should converge in a robust and coherent fashion across diverse methods. Moreover, we should all aspire to use multiple methods in both validating our measures and in investigating our ideas. Interestingly, that Platonic ideal is not realized as often as expected. If one looks closely at the empirical literature in personality psychology, the expectation that abstract constructs should converge across methods is seldom met at the level implied in the original article. This is not to argue that the Platonic ideal is not appropriate. Rather, one of the major points we would like to make in this chapter is that the ideal of the MTMM approach is often taken too literally and is sometimes misused or misinterpreted. Why speak such apostasies? In large part, because we are motivated to reiterate points made, ironically, by Fiske himself (Fiske, 1971).
What are these points? The first is that different methods, or modes as Fiske (1971) described them, are seldom innocuous. Thus, the literal assumption drawn from Campbell and Fiske (1959) that measures of similar constructs drawn from different methods should converge quite robustly is not met as often as we would like. This can lead to erroneous and nihilistic conclusions, such as the construct of interest, like depression, does not exist (e.g., Lewis, 1999). The second point is the assumption that monomethod studies are problematic, inadequate, and should be avoided at all costs. Or, conversely, we should all be doing multimethod studies. This directive fails to consider the empirical I fact just mentioned, which is that measures of the same construct seldom correlate highly enough across methods to warrant averaging across methods (Fiske, 1971). What are needed, rather than mandates to perform multimethod studies, are theoretical models that successfully incorporate and explain both the overlap and lack thereof of identical constructs across methods. In our following review, we will attempt to highlight the few theories and empirical examples that have done so.
Our third point is that the focus on multiple methods has inadvertently led to a misguided boondoggle to search for the methodological holy grail—the one method that deserves our ultimate attention. Campbell and Fiske (1959) should not be saddled with full responsibility for this phenomenon beyond the fact that they made it clear that we should be pursuing multiple methods. Leave it to human nature that psychologists would take that idea and try to one up the multimethod approach by finding the ultimate method. Thus, we have had hyperbolic statements made for and against particular methods made since the 1960s. People have argued that self-reports are fundamentally flawed and indistinguishable from response styles (Hogan & Nicholson, 1988; Rorer, 1965), that observer ratings are the seat of personality psychology (Hofstee, 1994), that projective tests do not work (Dawes, 1994), and that we should prioritize online measures over all other techniques (Kahneman, 1999). As will be seen in the following reviews, none of these positions is defensible.
As the methods used are often tied inextricably to the ideas in a field, we will first provide a working definition of the field of personality psychology that will serve as an organizing heuristic for the subsequent review. As will be seen, this is a true case of form following function, as the content categories within the field of personality are each dominated by specific methods. Then, we review recent multimethod studies within and across the content domains of personality psychology. We will end with some thoughts about particulars of multimethod approaches in personality psychology.
Comments
Published (as Chapter 22) in Michael Eid and Ed Diener, eds., Handbook of Multimethod Measurement in Psychology (Washington, D.C.: American Psychological Association, 2006), pp. 321–335. Copyright © 2006 American Psychological Association. Used by permission. “This article may not exactly replicate the final version published in the APA edition. It is not the copy of record.”