How SIOP Principles Guide Personality Evidence Across Job Families

What do SIOP's Principles ask a selection report to establish?

When a report says that a personality score is useful across job families, the reader needs to know what “useful” means and for which decision. The claim should connect scores from a named procedure, interpreted in a stated way, to a defined work behavior or outcome in the intended selection use. A trait label or test name alone cannot establish that connection. The fifth edition of *Principles for the Validation and Use of Personnel Selection Procedures* offers professional guidance for examining this reasoning. It describes validation as an argument about the proposed interpretation and use of scores, drawing on evidence suited to that claim. The document says its Principles guide professional judgment; they are not mandatory or exhaustive, and do not certify a provider or instrument. That distinction matters because a selection report and a self-reflection report make different promises. A selection report may inform a consequential decision about applicants, so its evidence must support the stated work-related inference. A self-reflection report can instead offer tentative language for noticing one’s own tendencies; that does not establish selection validity. To check a cross-family claim, follow the chain from procedure and score interpretation to work outcome, then to the target work. Start by asking what decision the report is meant to support. The evidence needed depends on that answer.

Sources: Principles for the Validation and Use of Personnel Selection Procedures

What does the report's score claim actually mean?

Before judging whether a personality report supports use across job families, translate its headline into a precise score claim. Record the instrument and version, the scale or facet, scoring procedure, and statement being made. Does the report say responses reflect a tendency, that the score can be compared with a reference group, or that it predicts a defined work outcome? Those are different inferences. A scale name alone does not tell you which one the evidence supports. In the fifth edition of Principles for the Validation and Use of Personnel Selection Procedures, SIOP describes validity as a unitary concept: evidence from different sources contributes to judging an interpretation for a particular use. The guide identifies evidence based on relationships with other measures or outcomes, test content, internal structure, response processes, and consequences. These are not separate badges from which a report can choose one and declare the procedure valid. Each source matters to the interpretation it can inform. A raw score is the result before transformation. A standardized score expresses it on a defined scale; a percentile locates it relative to a specified comparison group; a band places it in a category under stated rules. None means “probability of success” unless separate evidence supports that claim. Ask what reference group and scoring rules produced the number or label. Construct means the characteristic a measure is intended to represent. Evidence that items relate in a way consistent with the proposed construct can support that score interpretation. SIOP says relationships with measures of the same or different constructs can contribute convergent or discriminant evidence. These findings help establish what a score represents; they do not, by themselves, establish its relevance to job performance. For a job-related prediction, the guide calls for an additional link between selection scores and work-relevant behavior or outcomes. Reliability concerns score consistency under specified conditions. An unstable score limits how finely it can be interpreted, but consistency alone cannot show that an interpretation is accurate or useful for a work decision. A report documenting internal consistency or coherent structure has answered a question about the measure, not every question about its use. A reader can recognize that evidence while withholding the conclusion that the score establishes likely success across jobs. That asks for the missing link; it does not declare the measure valueless.

Sources: Principles for the Validation and Use of Personnel Selection Procedures

What outcome is the personality score supposed to predict?

A usable report names a work-relevant behavior or outcome and explains why that criterion fits the decision. “Performance” or “fit” without a defined behavior, measure, timeframe, or rationale leaves too much hidden. In SIOP’s Principles for the Validation and Use of Personnel Selection Procedures, fifth edition, a criterion is the measured outcome against which a predictor is evaluated. Examples include productivity, training results, turnover, and ratings of job-relevant behavior. These outcomes are not interchangeable: a score related to training has not thereby been shown to predict retention or overall performance. The report should say what was measured and why it matters. If a report says that a personality score predicts “success,” ask: success at which tasks or behaviors, assessed by whom or by what records, and over what period? A supervisor rating of collaboration is evidence about rated collaboration under the study’s conditions, not automatically output or every part of a role. SIOP says criteria should reflect outcomes critical to the focal work. A narrow measure cannot support a broader claim unless omitted parts of performance are acknowledged. The outcome should represent the work and decision named in the report. The U.S. Office of Personnel Management’s “Designing an Assessment Strategy” illustrates the link between prediction and criterion: if a personality test is intended to forecast later job success for customer-service applicants, evidence would connect scores with subsequent job performance. This illustrates the evidence question, not proof about a particular test. OPM’s “Personality Tests” guidance adds that personality measures may be more useful where work involves substantial interpersonal interaction or teamwork, and less useful in highly scripted jobs where tendencies have little room to affect behavior. If the named behavior rarely involves employee choice, the score-to-outcome connection needs explanation. A criterion can be noisy or incomplete. SIOP recommends attention to relevance, reliability, and contamination, or influences unrelated to the intended outcome. Examples include unequal territories, shift or location differences, and raters knowing predictor scores. Deficiency is the reverse: a measure leaves out relevant work while the report makes a broad claim. Ask how the outcome was gathered, whether it covers the behavior claimed, and what limits are acknowledged. Study timing adds context. In a predictive design, scores are collected around selection and outcomes later, after a specified period or when performance stabilizes. In a concurrent design, scores and outcomes are generally collected from current employees at roughly the same time. SIOP describes both; neither label alone shows that the sample or work setting matches another target. The report should make its design visible so readers can tell what was observed and what is inferred. A useful paraphrase has four parts: this score is evidence about this defined behavior or outcome, measured this way, under these work conditions. Then state the boundary: it does not establish unmeasured outcomes or explain how the relationship applies to other roles. If the report cannot complete that sentence, its prediction claim remains too broad for a consequential decision.

Sources: Principles for the Validation and Use of Personnel Selection Procedures; Designing an Assessment Strategy; Personality Tests

Why does job analysis matter more than the family label?

A job-family label tells you how an organization groups roles; it does not show that the work creates comparable conditions for a personality score to relate to the same outcome. Job analysis describes the behaviors, requirements, and context behind the comparison. SIOP’s *Principles for the Validation and Use of Personnel Selection Procedures* uses “analysis of work” for information about work, workers, organizations, and environments, including complexity, activities, and required competencies. The Principles do not require one fixed level of detail. The needed detail depends on intended use and available information. A report may rely on prior research to extend evidence across job families, but a less detailed analysis should still establish that jobs are similar in work behavior or required characteristics, or fall within a group where validity can be generalized. A family label is a hypothesis about similarity, not evidence for it. Similar titles can conceal differences that matter. SIOP identifies complexity, context, environment, behavior, and requirements as possible sources of variation, including differences by shift, location, and how work is performed. Two roles called “customer support” might differ in discretion, unscripted problem solving, or customer contact. If one role permits a behavior that another tightly scripts, the shared title does not show that a personality score has the same relationship to that behavior in both. A technical report should let a reader compare studied and target work: what people do, under what conditions, which requirements matter, and which outcome the score is meant to predict. It should explain the basis for those descriptions. SIOP cautions that existing job descriptions may or may not fit the research purpose; their relevance needs evaluation. A current document can still be too general. Jobs in different organizations can be comparable if the report explains relevant similarities. The Office of Personnel Management’s *Designing an Assessment Strategy* likewise ties assessment choices to job analysis and work requirements. Together, these sources suggest a practical check: does the report connect its family grouping to the behavior or requirements that make the prediction plausible? If not, ask for the job-analysis basis and comparison used to support transfer. Until that link is documented, the cross-family rationale remains unresolved. This does not prove the assessment useless, nor show that a score predicts performance in every role sharing a label.

Sources: Principles for the Validation and Use of Personnel Selection Procedures; Designing an Assessment Strategy

How should a reader weigh the evidence types in a report?

Evidence must match the claim it supports. A report may present kinds answering different questions about score interpretation and use. The fifth edition of Principles for the Validation and Use of Personnel Selection Procedures treats validity as an argument informed by different sources, not a badge attached to a test. Follow each result to the link it supports: score meaning, work relevance, or prediction of a defined outcome. A favorable result at one link does not complete the others.

| Evidence type | Question it can help answer | What it cannot establish alone | | --- | --- | --- | | Reliability and measurement error | How consistent and precise are scores? | Whether scores predict work outcomes | | Internal structure | Do item relationships fit the interpretation? | Relevance to a job family | | Content | Does procedure content represent work? | A score-to-outcome relationship | | Response process | Do answers fit the interpretation? | Work relevance or prediction | | Criterion relationship | Do scores relate to an outcome in this setting? | Transfer to another family | | Consequences | What follows from use? | The score inference itself | This is a reading aid, not a requirement for identical studies. SIOP describes sources that may contribute to validity; relevance depends on the inference and context. Ask what claim a finding addresses and what remains uncertain.

Reliability means consistency of scores under specified conditions; measurement error is uncertainty around an observed score. Reliability matters because unstable scores limit precision. But a consistent score can still measure something other than the claimed attribute, and consistency alone cannot show that different scores correspond to different later outcomes. The Principles caution that internal structure or relationships among constructs are not, by themselves, proof of job-relatedness.

Content evidence asks whether procedure content represents a defined work domain. A questionnaire about typical preferences does not become a sample of job tasks because its construct is called work-relevant. Score-to-outcome evidence asks whether scores relate to a defined criterion in the studied setting. The U.S. Office of Personnel Management's Designing an Assessment Strategy connects assessment strategy to the intended prediction and suitable evidence. These types answer different questions.

Response-process evidence matters when interpretation depends on how respondents understand or answer items; it may test assumptions about what answers represent. It does not replace work-relevance or outcome evidence. Evidence about consequences helps examine what follows from use, but favorable consequences cannot establish the score inference.

When findings disagree, ask what differs: instrument, sample, criterion, setting, or intended use. A report should describe these features, results, uncertainty, and limitations. The APA interview “Can a personality test determine if you’re a good fit for a job?” advises seeking support for reliability, validity, and fairness claims rather than relying on persuasive assertions. It prompts inspection; it does not establish that a particular report is sound.

Map each result to a link in the inference chain. If evidence supports consistent scoring but leaves work relevance open, record that gap. If scores relate to an outcome in one setting, ask whether target roles are comparable. This avoids overreading favorable results or dismissing a measure because one evidence type is absent. Ask whether evidence supports the particular claim, not merely whether the report says “validated.”

Sources: Principles for the Validation and Use of Personnel Selection Procedures; Designing an Assessment Strategy; Can a personality test determine if you’re a good fit for a job?

How should study design and sample shape the conclusion?

A report should identify the sample, how scores and work outcomes were measured, and what the design can establish. A predictive study measures scores before later outcomes; a concurrent study measures scores and outcomes at roughly the same time. Neither label settles whether evidence fits the intended applicant group, criterion, or job family. SIOP’s Principles for the Validation and Use of Personnel Selection Procedures treat design as one part of a validity argument, not a shortcut to a conclusion.

Read the sample description first. Ask how many people took part, what roles and settings they represented, which assessment version were used, and whether participants resemble those in the proposed use. Incumbents may differ from applicants in ways relevant to the inference; that difference calls for examination, not automatic acceptance or rejection. Look also for disclosure of missing outcome data or restricted score ranges, uncertainty, and counterevidence. Without these details, a result may describe its study, but its reach is harder to judge.

A meta-analysis abstract, such as the South African Big Five review, can broaden the evidence base: it reports 33 studies and 6,782 people. That pooled scope does not erase study variation or establish that an unnamed instrument works in a target role. A credible finding for its sample still needs a relevance argument before supporting another job family.

Sources: Principles for the Validation and Use of Personnel Selection Procedures; The validity of the Big Five personality traits for job performance: Meta-analyses of South African studies

When can evidence travel across job families?

Evidence may travel across job families when a documented method supports applying findings from one work setting to another. The Principles call this transportability: using a specific selection procedure in a new situation on the basis of validation research conducted elsewhere. For transportability, look for a technically sound original study and a clear account of its relevance to the proposed setting. The Principles identify comparability in job content or requirements as central, with similarity of job context and candidate group also potentially relevant. A job-family label cannot establish those similarities by itself. The report should connect studied work to target roles: what people do, which requirements matter, and what conditions could alter the behavior or outcome being predicted. Transportability is one route among several. A local criterion study examines the relationship between scores and a defined work outcome in the organization or setting of interest. Its proximity can help, but a new local study is not automatically stronger: a small or technically weak study may provide little dependable information. Transportability asks whether sound evidence elsewhere is relevant to the target. Synthetic or job-component validity builds an argument from documented relationships between scores and specific work components, then asks whether those components occur in the target roles. Meta-analytic validity generalization combines findings across studies to estimate relationships for the work domains and settings represented in that evidence base. The fifth edition of SIOP’s Principles treats transportability, synthetic or job-component validity, and meta-analysis as distinct approaches to generalizing validity evidence. The useful comparison is not “local evidence good, general evidence bad,” or the reverse. Ask what was measured, how the evidence was produced, which jobs and contexts it covers, and how the report connects those conditions to the target family. Broader evidence may be more informative than a weak local result, while strong local evidence may apply only to the roles and outcome studied. A 2007 statement hosted by the U.S. Equal Employment Opportunity Commission discusses these methods while referring to the 2003 edition of the Principles. It is historical context, not current SIOP guidance or an agency finding about a particular assessment. A report supports transfer across families only when it shows the route from study to target: sound evidence, a defined outcome, relevant work comparisons, and a reason the method applies. If a link is missing, mark transfer as unestablished and request the technical basis. This avoids both requiring a separate local study for every family and accepting generalized claims without examining their bridge.

Sources: Principles for the Validation and Use of Personnel Selection Procedures; Willner statement on transportability

What can broad personality research establish—and what cannot it establish?

A meta-analysis can show whether relationships recur across a defined body of studies. It cannot certify that a personality report predicts performance in every job family. A reader must still decide whether the exact score, outcome, and target work connect well enough for the proposed use. “The validity of the Big Five personality traits for job performance: Meta-analyses of South African studies” illustrates both reach and boundary. This systematic review combined 33 South African studies covering 6,782 individuals to estimate relationships between Big Five traits and several job-performance criteria. Its abstract reports conscientiousness as the strongest predictor across those criteria, while other traits showed validity for particular criteria or subsamples. This is a group-level pattern in the assembled South African research, not a result for every trait, measure, or occupation. It concerns Big Five measures and studied criteria, not an unnamed report or a specific employer’s score interpretation. The authors report similarity to findings in other cultural contexts, suggesting that some personality–performance relationships may recur across settings. But recurrence does not erase differences among instruments, samples, criteria, or jobs. The meta-analysis provides context for a validity argument; it does not complete the argument for a target family that its abstract does not establish. Pooled and local evidence play different roles. A local study may connect one procedure and outcome to an organization’s work, but its reach depends on sample, measures, criterion, and quality. A meta-analysis combines studies to examine recurrence, but relevance to a target role depends on what those studies included and how results were combined. Neither “local” nor “large” is a quality verdict. The question is whether evidence and target share features that matter to the intended inference. For a cross-family claim, broad research is a supporting layer. It can make a relationship plausible and show which outcomes have been studied; the report still needs to identify its precise score and explain why evidence bears on the target work. Meta-analytic averages describe group patterns. They do not tell an applicant what their performance will be, diagnose a person, or determine whether a job is right for them. Treat research as a reason to examine the bridge carefully, not as a substitute for it.

Sources: The validity of the Big Five personality traits for job performance: Meta-analyses of South African studies

How does a cross-family claim change with the report's intended use?

The evidence chain has to end at the decision someone will actually make. A score description that is useful for private reflection does not, by itself, justify screening applicants across job families. The fifth edition of SIOP's *Principles for the Validation and Use of Personnel Selection Procedures* treats validity as evidence for a proposed interpretation and use of scores. In employment, that means asking whether the score supports a defined prediction about later work behavior or outcomes, for the people and roles covered by the decision. Consider an illustrative employer proposing to use one personality report to screen applicants for several job families. Before treating a score as a selection signal, the employer would need to state what work outcome it is meant to predict, how that outcome was defined and measured, and why evidence from the studied roles applies to each target family. If the supporting evidence concerns a different group or a narrow set of roles, the report needs a defensible account of why those differences do not defeat the inference. The question is not whether the groups or roles are identical, but whether the differences change the score-to-outcome argument. That explanation should be explicit enough for a reader to inspect carefully, not hidden behind the family name. The APA interview on personality tests also advises readers to look for evidence behind reliability, validity, and fairness claims. Now consider a person using a report privately to prepare for a career conversation or notice a recurring collaboration pattern. A tentative description can serve as a prompt: in which situations does this tendency seem relevant, what observable behavior follows, and what other conditions could explain it? The distinction is about the claim, not whether personality is important. Tendencies may be expressed differently across tasks and contexts; they do not dictate an individual's conduct or worth. For selection, ask for evidence tied to the consequential prediction and the target families. For reflection, use the report as a provisional question to examine against observed experience. Do not carry a low-stakes interpretation into an employment conclusion without the separate evidence that conclusion requires.

Sources: Principles for the Validation and Use of Personnel Selection Procedures; Can a personality test determine if you’re a good fit for a job?

What should I do when the report leaves a gap?

Write the claim in one sentence: this score, interpreted this way, is meant to indicate this work behavior or outcome for these roles. Mark missing links unknown: instrument and version, score interpretation, outcome, study design and sample, and why evidence from those jobs applies to the target family.

For selection, ask the provider or responsible organization for the technical report. Try: “Please identify the measure and version; explain this score’s interpretation; define the outcome and target roles; summarize the supporting studies and results; and explain why those findings apply across these job families.” SIOP’s *Principles for the Validation and Use of Personnel Selection Procedures* frame validity as evidence for a proposed interpretation and use. The report should connect those parts, not just say the test is validated. Until it does, leave the cross-family conclusion unresolved.

For private reflection, turn a broad label into questions about observable situations: when do you prefer a plan, seek input, or change course, and what conditions affect that pattern? The Work Pattern Report can structure low-stakes reflection across ten continuums. It has no norms, cutoff, selection validation, or job recommendation, so it cannot fill an employer’s evidence gap. The conclusion changes when evidence supports the specific score claim and target work.

Sources: Principles for the Validation and Use of Personnel Selection Procedures

Questions readers ask