Construct validation is an ongoing process Franzen, To this end, Campbell and Fiske introduced the Multitrait-Multimethod MTMM Matrix, an evaluation method of the acceptability of questionnaires as measures of constructs or latent variables Price, This method involves a correlation matrix customized to enhance the evaluation of construct validity 1 in terms of the discriminant and convergent validity Brown, The method that Campbell and Fiske operationalize is based on the idea that theoretically related scores of the same or similar traits should also show high positive correlations with each other whereas theoretically unrelated scores of different constructs should also show low or insignificant correlations.
More specifically, two or more traits are measured with two or more methods.
Traits could be hypothetical constructs about stable characteristics, like cognitive abilities. On the other hand, methods can encompass different occasions, or means of data collection e. Note that, trait interpretation suggests that a latent attribute or construct explains the consist patterns observed in the scores Price,; Kline, The MTMM method is carried out during the structural and external stages of the validation process Price, , see Table 1 and Figure 1 for an overview.
For example, during the development of a new self-report scale on social anxiety, the researcher could also measure three constructs e.
- Physical Oceanography of the Dying Aral Sea!
- Browse more videos;
- Handbook of Affective Sciences?
The MTMM correlation matrix see Table 2 consists of four sets of correlations: 1 the monotrait-monomethod block correlations; 2 the monotrait-heteromethod block correlations; 3 the heterotrait-monomethod block correlations; and 4 the heterotrait-heteromethod block correlations Koch et al.
The correlations are divided into the two different method blocks of the table: 1 The monomethod blocks; and 2 the heteromethod blocks Trochim, ; Brown, This example layout involves three different traits, each measured by three methods, generating nine separate variables. The interpretation of the coefficients contained in the body of the table, is included in the table note. The monomethod blocks emerge from correlations of measurements by the same method.
Their number equals the methods used. In order to account for measurement error Koch et al.
Table 1. Summary of the big picture of the place of MTMM in the construct validation process. Benson adapted by Price page Table 2. Notes: Numbers in body of table are correlation coefficients except those in parentheses. Data is artificial.
- Search Tips?
- Applied Psychometrics: The Application of CFA to Multitrait-Multimethod Matrices (CFA-MTMM).
- Introduction to the Functional Renormalization Group!
- House of Windows!
- Handbook of Multimethod Measurement in Psychology / Edition 1.
- Handbook of multimethod measurement in psychology?
Reliability coefficients for each test are in parentheses Reliability Diagonal. Monomethod blocks are marked with a dotted line. Heteromethod blocks are marked by a thick line. The Table format was adopted by Price p. Figure 1. MTMM framework overview. Source: Content adapted by Furr p.
For example, in Table 2 the reliability coefficients range from 0.
The heteromethod blocks emerge from correlations of measurements by different methods. The correlations in their diagonals represent the convergent validity of the same trait measured by different methods. Thus, they should be strong and positive Price, ; Brown, They present correlations evidencing discriminant validity because different traits are measured by different methods.
They also should be the lowest of all Price, Brown explains that discriminant validity is demonstrated when weaker correlations emerge between different traits measured by different methods i. Finally, Brown explains that method effects may be evaluated by comparing the differential magnitude of correlations in the off-diagonal elements of the monomethod blocks. Campbell and Fiske pp. The correlations in the monotrait-heteromethod block have to be statistically significant and strong.
The correlations in the heterotrait-heteromethod block should be weak or should equal the correlations in the heterotrait-monomethod block. The correlations in the heterotrait-monomethod block should be smaller than the correlations in the monotrait-heteromethod block.
- Matthias R. Mehl - Google Scholar Citations?
- On the Road: The Original Scroll.
- The Effect of Long Term Thermal Exposure on Plastics and Elastomers.
- Neuropsychological Assessment in Clinical Practice: A Guide to Test Interpretation and Integration!
- Read Handbook of Multimethod Measurement in Psychology Ebook Free!
- Policy Discourses on Irregular Migration in Germany and the United Kingdom.
The pattern of the trait intercorrelations in all heterotrait triangles should be similar for the mono- as well as the heteromethod blocks. This property also refers to the discriminant validity of the measures summarized by Koch et al. For details refer also to Eid Finally, Trochim in line with Campbell and Fiske also suggests the following guidelines for the interpretation of an MTMM matrix: 1 Reliability coefficients must be the highest values in the matrix because the correlations between a trait and itself are expected to be the strongest than with any other construct also Brown, Recall that all off-diagonal elements must be the smallest to evidence discriminant validity.
If this is not true, then a method effect may be present. Despite the fact that the Campbell and Fiske technique was a significant progress for examining construct validity, the MTMM approach was not widely accepted the years immediately following its inception Brown, Additionally, several limitations were noted first by Jackson This argument is based on the idea that the correlation pattern may be affected by the way variance is distributed when measuring the target traits.
Specifically, the correlation matrix is orthogonalized and then submitted to Principal Components Analysis, with a varimax rotation. The extracted factors equal the number of matrix traits. Additionally, the influence of different measurement methods cannot be taken into account when the monomethod matrix is used Franzen, Later, Franzen continues Cole, Howard, and Maxwell compared the MTMM results with constructs mono-operationalization suggesting that as a rule, the validity coefficients were misleadingly deflated.
Additionally, Cole argued that the cost of the MTMM application in clinical settings deems the method inapplicable Franzen, Crucially, Cole noted that the MTMM is prone to correlated errors due to factors like the time of day, or the instrument administrator.
Books - Psychology - Library Research Guides at Lipscomb University
Cole suggested the use of CFA instead to examine discriminant and convergent validity Franzen, At the same time, non-factorial oriented attempts to find an appropriate method of statistical analysis for the data in the matrix were numerous Eid et al. These methods were mainly based on analysis of variance models or nonparametric alternatives and the reason for the lack of success was attributed to conflicting data in the diagonals in relation to the data in the triangles Sawilowsky, For example, Sawilowsky , proposed a statistical test for analysis MTMM data where the null hypothesis is that the coefficients in the matrix are unordered from high to low.
This is tested against the alternative hypothesis of an increasing trend from the lowest level heterotrait heteromethod to the highest level i. See Sawilowsky , for more details and an applied example. Specifically, CFA can be carried out in MTMM matrices, like in any other covariance matrix, to study the latent dimensions of traits and methods factors Brown, Besides that, many different MTMM approaches have been suggested, such as variance component models or multiplicative correlation models Koch et al.
Each indicator in this specification is a function of a trait, method, and unique factors, Brown explains. The specified MTMM model becomes a baseline model, compared against a set of nested models in which parameters are either removed or added. Based on this comparison, evidence of convergent and discriminant validity emerges, both at the matrix level and at the individual parameter level Byrne, Figure 2 is a path diagram for an MTMM matrix of 3 traits and 3 methods to describe the steps of the correlated methods.
Testing for evidence of convergent and discriminant validity involves comparisons between the model in Figure 2 and three alternative nested MTMM models in Figures Byrne, ; Brown, Byrne describes the steps for comparing the nested models as follows. Usually, a negative variance is associated, either with a residual or a factor and this problem is generally attributed to overparameterization of the MTMM models, due to their inherent complexity Byrne, ; quoting also Wothke, Figure 2.
Model 1 of 4 models that are specified Baseline Model. Figure 3. Model 2 of 4 Nested models that are specified Nested model to test Convergent Validity. Figure 4. Model 3 of 4 Nested models that are specified Nested model to test Discriminant Validity. Its specification is identical to that of the CTCM model, except that there are no trait factors, as shown in Figure 3 Widaman, ; Byrne,.
Figure 5. Model 4 of 4 Nested models that are specified Nested model to test Discriminant Validity.
Introduction: The need for multimethod measurement in psychology
Because these two models are nested, a comparison of this model to the CTCM provides a statistical evaluation of whether effects associated with the different traits emerge Brown, The path diagram of this model is presented in Figure 4. As with the hypothesized CTCM model Figure 2 , each observed variable loads on a trait and a method factor. Nevertheless, in this MTMM model, the traits are perfectly correlated i. The goodness-of-fit of this model is examined and it is compared to the fit of the previous models in Figure 2 and Figure 3 respectively. It is different from Model 1 only in the absence of correlations in the method factors.
The goodness of-fit for this model is evaluated see also Byrne, This information at the matrix level derives through the comparison of particular pairs of models. Note that typically, a summary of goodness-of-fit statistics of all four MTMM models is presented in a table, and a summary of model comparisons in a second table, see an example of both in Table 3 adapted by Byrne Evidence of discriminant validity among traits emerges from the comparison of the CTCM Model in which traits are freely correlated see Model in Figure 2 with the PCTCM model, specified in step 3 in which traits are perfectly correlated see Model in Figure 4.
Multitrait-multimethod change modelling
The procedure is repeated to show evidence of discriminant validity regarding method effects. For an applied example see Byrne For a more detailed description and an applied example see Byrne Table 3. Notes: Based on a table used by Byrne p. Table 4. First, all loadings are examined to see if they are statistically significant.
Then all factor loadings across traits and factor loadings across methods are compared, to see whether method variance or trait variance is higher over all the available ratings Byrne, Generally, correlations among traits should theoretically be small to show evidence of discriminant validity, but such findings are rare in psychology. For an applied example of the correlated method refer to Byrne and Brown Some simpler alternatives to CTCM models have been proposed, and the correlated-uniqueness CU model described next is one of them Kline, The CU model identification must abide by the following rules as reported by Brown There must be at least two traits t and three methods m.
Rules regarding the CU trait model are the same as in the correlated methods approach. Specifically: 1 Each indicator must load on one trait factor with all other cross-loadings fixed to zero; and 2 The trait factor correlations are freely estimated. Consequently, the major difference between the CU method in comparison to the correlated methods is the estimation of method effects.
In the CU model, method effects are assessed by adding correlated uniquenesses errors among the indicators of the same assessment method instead of specifying method factors Brown, After the model specification, the model is tested and the fit of the solution to the data is evaluated. Except fit the inspection of standardized residuals, the modification indices must be also evaluated as proposed by Brown, The parameter estimates Brown continues are then examined.
That is, when loadings of traits factors are substantial and statistically significant, this is an evidence supporting convergent validity. Conversely, when correlations among the trait factors are substantial, this is indicative of low discriminant validity. Method effects are assessed when correlated uniquenesses among indicators of the same method are moderate or large in magnitude Brown, For example, Brown explains, when trait factor loadings are large this suggests that the relation of the indicators to their purported latent constructs is strong thus, convergent validity can be supported while adjusting for method effects.
When modest correlations emerge among trait factors this is indicative of adequate discriminant validity. The significance of method effects is evaluated by the size of the correlated uniquenesses among indicators. For an applied example of the CU, method sees Byrne Figure 6.
quepretukra.tk Saris and Aalberts also examined different interpretations of CU models. Nonetheless, Byrne comments that the general CFA model is as a rule preferable. Specifically, an advantage of the CT-CM model is that it permits fragmenting observed variance into a trait, method, and error variance in comparison to the CU model that is more restrictive Eid et al.