How to Check a Personality Report’s Recommendation Against Its Evidence

What must connect a score to a recommendation?

A report should make a chain of reasoning visible. Start with the instrument and scale: what did the questions aim to measure? Next, ask how the score is interpreted. Is it a raw total, a standardized score, or a comparison with a norm group? A norm group is the reference sample used to place a score in context. Then ask what supports that interpretation for people and conditions like the ones described. Finally, look for evidence connecting the interpretation to the suggested action. Each step makes a different claim.

Validity means evidence and reasoning support a particular interpretation or use of scores. It is not a permanent quality mark that makes every conclusion drawn from a test acceptable. Gregory Cizek’s review distinguishes support for what a score means from justification for how it is used. That distinction matters when a report moves from “your answers suggest a preference for independent decision-making” to “avoid collaborative work.” The first is a description of a tendency; the second recommends an action and assumes that preference predicts difficulty across collaborative settings. The latter claim needs its own support. [Cizek’s review](https://pubmed.ncbi.nlm.nih.gov/22268761/) explains why score interpretation and test use raise distinct validity questions.

The International Test Commission’s Guidelines on Test Use say evidence for score inferences should be accessible for scrutiny, and that evidence of reliability and validity should fit the test’s intended purpose. Reliability, or measurement consistency, matters because an unstable score puts a limit on what can reasonably be inferred. But consistency alone does not show that a scale measures the intended tendency or that a recommendation follows from it. The guidelines also tell competent test users to consider the comparison group and score limitations. A reader need not reproduce a technical review; it is reasonable to ask the provider where the relevant evidence is documented. [The ITC guidance](https://www.intestcom.org/files/guideline_test_use.pdf) supports those checks.

Use four checks: Which score is involved? What does the report say it means? What supports that interpretation for the relevant population? What connects it to the action? A gap does not show the assessment is useless; it identifies what the recommendation has not yet established.

Sources: Defining and distinguishing validity: interpretations of score meaning and justifications of test use; International Test Commission Guidelines on Test Use

When is the recommendation stronger than the score evidence?

Compare the reach of the recommendation with the reach of the evidence. “Use this result to notice which kinds of decisions feel comfortable” is a modest reflection prompt. “This score identifies the career where you will succeed” is a broad prediction. The second statement reaches beyond a tendency to a future outcome, and may affect a consequential choice. It calls for evidence about that outcome, the relevant population and setting, and the specific instrument and score, rather than a general claim that personality matters at work.

For any recommendation, look for the assessment’s exact name and version, the scale behind it, the population studied, the comparison group if one is used, and the stated purpose. Then match the evidence to the claim. A study of one trait and one work outcome cannot automatically support a different trait, population, outcome, or action. If the report does not provide its technical basis, record that you could not verify the connection. Missing documentation in the report is not proof that no evidence exists; it is a reason to ask the publisher or a qualified interpreter for the source and its limits.

Employment research is a useful counterexample to blanket skepticism, but its findings are narrower than career advice. In a 1991 meta-analysis, Tett, Jackson, and Rothstein reviewed 494 studies and retained usable findings from 97 independent samples (13,521 people). They reported corrected mean validities of .29 for confirmatory studies, .12 for exploratory studies, and .38 when job analysis guided measure selection. These are aggregate findings in employment research, not general estimates for every personality test or guarantees for one person. The authors also noted weak reporting of validation-study characteristics, limiting readers’ ability to judge the contexts represented. Their results support selecting measures in relation to a defined job analysis, not general career prediction. [The meta-analysis abstract](https://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.1991.tb00696.x) reports its search base, samples, estimates, and reporting limitation.

A companion 1991 meta-analysis by Barrick and Mount examined five occupational groups—professionals, police, managers, sales, and skilled or semi-skilled workers—and three job-performance criteria: job proficiency, training proficiency, and personnel data. Conscientiousness showed relationships across all groups and criteria. Other trait relationships varied: the abstract reports extraversion relationships with managerial success and training proficiency among managers and sales workers, and openness relationships with training proficiency across the occupational groups. Some estimated true-score correlations were small (ρ < .10). These aggregated findings concern Big Five dimensions and work criteria; they do not establish individual outcomes, validate every personality instrument, or show which job a person should choose. Both analyses date from 1991, so they are not measure-specific evidence for a current consumer report. [Barrick and Mount’s abstract](https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1744-6570.1991.tb00688.x) shows how results varied by trait, occupation, and criterion.

The U.S. Office of Personnel Management makes the same boundary practical: when a personality test is meant to forecast success in a particular job, evidence may be needed that its scores relate to later performance on that job. The meta-analyses show why neither blanket dismissal nor blanket endorsement fits; relevance depends on the measure, sample, criterion, and use. [OPM’s guidance](https://www.opm.gov/policy-data-oversight/assessment-and-selection/assessment-strategy/) describes the job-specific evidence requirement.

A report can support a cautious description while leaving its proposed action uncertain. Its confidence should not exceed what it discloses about the score, sample, outcome, and intended use.

Sources: International Test Commission Guidelines on Test Use; Personality Measures as Predictors of Job Performance: A Meta-Analytic Review; The Big Five Personality Dimensions and Job Performance: A Meta-Analysis; Designing an Assessment Strategy

How can I check the chain in a report I have?

Choose one recommendation and write it in a sentence. Note the score behind it and the report’s explanation. Check its comparison basis and any stated precision or limitations. Then look for evidence about the proposed use, not only the scale’s general interpretation.

Consider an illustrative, hypothetical report that links a preference for making decisions independently to advice to avoid collaborative roles. First check whether the score really supports the stated preference, and what population or norm the comparison represents. Then inspect the behavioral bridge: is there evidence that this measure predicts collaboration outcomes, or has the report moved from preference to prescription without testing the link? Finally, check the recommendation against observable examples. Have clear responsibilities, time, and room to contribute made collaboration workable? Have unclear roles or rushed decisions created friction? Those examples can challenge a sweeping recommendation without proving a different universal rule. Work behavior can reflect team structure and demands as well as a person’s tendencies.

Use the result proportionately. For reflection, it can help you form a question and compare it with repeated experiences, feedback, and practical constraints. For a consequential employment decision, a general report is not enough to establish job fit or justify a hiring or promotion outcome. If you want another structured prompt for self-reflection, the publication’s Work Pattern Report asks about decision-making, planning, ambiguity, feedback, conflict, collaboration, ownership, change, and learning. It is a non-validated self-report with no norms or selection score; it can help organize observations, not validate the original recommendation.

When the chain is incomplete, keep the useful question, limit the conclusion, and ask the provider: “Which score supports this suggestion, what evidence connects it to this use, and what observation would change your interpretation?”

Sources: Defining and distinguishing validity: interpretations of score meaning and justifications of test use; International Test Commission Guidelines on Test Use; Designing an Assessment Strategy

Questions readers ask