Table 3 Modified criteria of psychometric quality rating based on Terwee, Bot [15] and Cordier, Chen [26]
Psychometric property | Scorea | Quality Criteriab |
|---|---|---|
Content validity | + | A clear description is provided of the measurement aim, the target population, the concepts that are being measured, and the item selection AND target population and (investigators OR experts) were involved in item selection |
? | A clear description of above-mentioned aspects is lacking OR only target population involved OR doubtful design or method | |
- | No target population involvement | |
± | Conflicting results | |
NR | No information found on target population involvement | |
NE | Not evaluated due to “poor” methodological quality | |
Structural validity | + | Factor analysis performed with adequate sample size. Factors should explain at least 50% of the variance |
? | No factor analysis performed and explained variance not mentioned | |
- | Factors explain <50% of the variance | |
± | Conflicting results | |
NR | No information found on structural validity | |
NE | Not evaluated due to “poor” methodological quality | |
Hypothesis testing | + | Specific hypotheses were formulated AND at least 75% of the results are in accordance with these hypotheses; Convergent validity: correlation between similar assessments is at a statistically significant level (p < 0.05) and strength of relationship is ≥0.5 which is consistent with the hypothesis; Discriminant validity: uses appropriate statistical analysis (e.g., t-test p < 0.05 or Cohen’s d effect size ≥0.5) |
? | Doubtful design or method (e.g., no hypotheses) | |
- | Less than 75% of hypotheses were confirmed, despite adequate design and methods; Convergent validity: correlation between similar assessments is not at a statistically significant level (p ≥ 0.05) and strength of relationship is <0.5 which is inconsistent with hypothesis | |
± | Conflicting results between studies within the instrument | |
NR | No information found on hypotheses testing | |
NE | Not evaluated due to “poor” methodological quality | |
Internal consistency | + | Factor analyses performed on adequate sample size (7 * # items and 100) AND Cronbach’s alpha(s) calculated per dimension AND Cronbach’s alpha(s) between 0.70 and 0.95 |
? | No factor analysis OR doubtful design or method | |
- | Cronbach’s alpha(s) <0.70 or >0.95, despite adequate design and method | |
± | Conflicting results | |
NR | No information found on internal consistency | |
NE | Not evaluated due to “poor” methodological quality | |
Reliability | + | ICC or weighted Kappa 0.70 |
? | Doubtful design or method (e.g., time interval not mentioned) | |
- | ICC or weighted Kappa <0.70, despite adequate design and method | |
± | Conflicting results | |
NR | No information found on reliability | |
NE | Not evaluated due to “poor” methodological quality | |
Measurement errorc | + | MIC < SDC OR MIC outside the LOA OR convincing arguments that agreement is acceptable |
? | Doubtful design or method OR (MIC not defined AND no convincing arguments that agreement is acceptable) | |
- | MIC SDC OR MIC equals or inside LOA, despite adequate design and method | |
± | Conflicting results | |
NR | No information found on measurement error | |
NE | Not evaluated due to “poor” methodological quality |