Why Hiring Validity Does Not Transfer to Every Job

The hiring decision is narrower than the headline

Imagine an employer presents a report saying that one personality scale is associated with job performance. The employer then uses the same scale to screen applicants for a warehouse role, a research role, and a client-facing role. The headline may sound consistent, but the decisions are not the same. The jobs demand different behaviors, the meaning of good performance may differ, and the consequences of an error may not match.

Validity means support for an intended interpretation and use of scores. It does not mean that a questionnaire is simply ‘valid’ in every context. A report can show that its items relate to a trait, while offering much weaker evidence that the resulting score predicts performance in a particular job. It can also show a relationship with one criterion, such as supervisor ratings, without showing that the score predicts training success, safety behavior, retention, or customer outcomes.

That is why the first sentence to inspect in a technical report is the purpose. Is the evidence for self-reflection, coaching, development, research, or employee selection? A measure designed to describe tendencies for reflection should not be treated as a hiring screen merely because its labels sound work-related. Hiring is a consequential use. It needs evidence for the role and the decision rule, not only an attractive profile.

Separate the trait from the performance criterion

A personality assessment measures responses that are interpreted as indicators of one or more tendencies. Job performance is an outcome, and it is not one single thing. Researchers may study task performance, training performance, contextual performance such as helping or cooperation, attendance, sales, safety, or a supervisor's overall rating. These criteria overlap, but they are not interchangeable.

A meta-analysis combines results from multiple studies. It can estimate an average relationship, but its average does not erase differences among the studies. In an early meta-analysis of the Big Five, conscientiousness showed consistent relationships across the occupational groups and performance criteria examined. Other traits varied by occupation and criterion. Extraversion, for example, was useful in the studied manager and sales groups, while openness and extraversion related to training proficiency across occupations. The finding is informative, but it is not a warrant for treating every trait as equally relevant to every job.

The same caution applies when a report uses a broad phrase such as ‘success at work.’ Ask what success meant in the evidence. A score related to careful task completion may not answer whether someone will build trust with clients. A score related to training performance may not answer whether someone will perform safely after months of routine work. Before comparing a candidate with a cutoff, the employer needs a defensible link between the measured construct and the job outcome.

Job analysis supplies the missing bridge

Job analysis is a systematic description of the important work behaviors, tasks, conditions, and outcomes in a role. It turns a vague requirement such as ‘good with people’ into observable demands: handling an upset customer, explaining a complex choice, documenting a decision, or coordinating a handoff under time pressure. The point is not to find a personality label that sounds like the job. It is to establish which behaviors matter and why.

The O*NET content model illustrates this distinction by separating worker characteristics, skills, knowledge, work activities, and work context. Two jobs can share a title while differing in pace, autonomy, social demands, or consequences of error. Conversely, two different titles can share important work behaviors. A job title alone is therefore weak evidence that a validation result transfers.

The Uniform Guidelines on Employee Selection Procedures make the same logic explicit. A validity study should be based on information about the job, including important work behaviors and their relative importance. For construct validity, the measured construct should be tied to critical or important work behaviors in the job or group of jobs being studied. If several jobs are grouped, the shared behaviors should be comparable in level and complexity. A report that skips this bridge leaves the reader to supply the most important part of the argument.

A broad result can be real and still be too broad

It is tempting to turn a robust average into a universal rule. That shortcut fails for two reasons. First, an average relationship can conceal variation around the average. Second, a study may have examined a narrower set of occupations, criteria, or participants than the proposed use.

Research on conscientiousness shows why the boundary matters. A meta-analytic study found that conscientiousness was a stronger predictor in more highly routinized jobs and a weaker predictor in jobs with higher cognitive-ability requirements. This does not mean the trait stops mattering in complex work. It means the work structure changes the strength of the observed relationship. The relevant question is not ‘Is conscientiousness valid?’ but ‘Under what job conditions, for which performance outcome, and with what degree of uncertainty?’

A later meta-analytic review of self-report conscientiousness found relationships with job performance that did not differ significantly across the applicant and incumbent samples or across concurrent and predictive designs it compared. Yet the authors also reported that only a small minority of the reviewed studies used predictive applicant designs and cautioned that the evidence could not reliably answer how well self-report measures retain predictive value in applied settings where applicants may shape their responses. A broad literature can therefore be useful without settling the exact hiring decision in front of an employer.

Transfer requires comparable jobs and comparable use

Transportability is the question of whether validity evidence from one setting can support use in another. The answer depends on more than the name of the construct. The jobs should share important work behaviors. Their complexity and performance demands should be comparable. The applicant population and criterion should be sufficiently relevant. The assessment should be administered, scored, and interpreted in a way consistent with the original evidence.

The Uniform Guidelines describe these conditions in practical terms. When evidence comes from another study, the user should show that the important behaviors in the user's job are the same as those in the original study, that the original criteria are relevant, that important sample characteristics are comparable, and that the proposed use is consistent with the original findings. For a construct claim extended to jobs outside the studied group, the Guidelines call for additional empirical research for the additional jobs or groups.

This is why ‘validated for hiring’ is incomplete wording. A useful technical manual should identify the jobs, participants, criteria, score use, and limitations. Evidence for a hiring cutoff does not automatically support ranking. Evidence for a development report does not automatically support rejecting applicants. A change in decision rule can change the claim being made.

The sample and the score use can change the result

A validity coefficient is affected by how the study was designed and who took part. Current employees are not the same as applicants. Employees have already passed earlier screens, learned the job, and remained in the organization. Their scores may cover a narrower range than the applicant pool. This range restriction can make a relationship look different from the one an employer would observe before selection.

Recent work on personnel-selection meta-analysis has also stressed that researchers should examine variability across settings, not only a mean estimate. The authors describe problems that can arise when corrections for range restriction are applied uniformly, and they emphasize that operational validity estimates can change depending on the evidence and assumptions used. This does not make meta-analysis useless. It means the average should be read with its design, spread, and correction choices visible.

Self-report creates another boundary in selection. Applicants may answer in a way they believe will help them, whether or not the response reflects their usual tendency. That does not prove that every applicant is distorting a result, and it does not make self-report automatically worthless. It does mean that evidence from low-stakes research participation should not be treated as identical to evidence from a high-stakes hiring screen. A report should say how its evidence matches the intended use and what remains uncertain.

A worked comparison: same scale, different inference

Consider a report that measures a tendency toward organized, dependable work. In a routine records role, the job analysis might identify repeated checking, accurate documentation, and consistent completion of defined procedures as important behaviors. If the assessment's validation evidence connects that construct to comparable criteria in similar work, it may be a reasonable candidate for further review, subject to fairness and local evidence.

Now consider a research role in which the central demands are ambiguous problem definition, changing methods, and learning unfamiliar material. Organized work may still help, but the same score no longer answers the whole question. The job may require different constructs and a different criterion. A client-facing role adds another possibility: the employer may need evidence about communication or interaction in the specific context, not a generic assumption that one personality scale represents those behaviors.

These examples do not produce a hiring recommendation or an invented score. They show how the inference changes when the job behavior changes. The assessment may measure the same construct consistently in both settings. That is a reliability question. Whether the construct supports a prediction about each role is a validity question. Consistent measurement is useful, but it cannot create job relevance where the job analysis and criterion evidence are missing.

The same reasoning applies to an assessment delivered through a video interview or another automated format. A Society for Industrial and Organizational Psychology research brief described initial, mixed evidence for some personality judgments across interview questions, while noting that the study did not examine work-relevant outcomes and relied primarily on student samples in mock interviews. Generalizing construct evidence from an interview task to job performance would therefore be an additional step, not a free consequence of the first result.

Read the report as a decision document

When a personality assessment is proposed for hiring, read the technical documentation as a chain of claims. Start with the construct: what does the instrument measure, and how is it defined? Then inspect the criterion: what outcome was predicted, who rated it, and how close is it to the work that matters here? Next inspect the sample: applicants or incumbents, which jobs, which locations, and which language or population? Finally inspect the use: screening, cutoff, ranking, or one part of a wider process.

A practical checklist is:

1. Identify the exact job and the important work behaviors. 2. Ask which personality construct is linked to those behaviors and why. 3. Check whether the validation sample and performance criterion resemble the proposed use. 4. Look for evidence across comparable jobs, not just a broad average. 5. Check reliability and measurement error, while remembering that reliability is not validity. 6. Ask whether the report documents subgroup evidence, adverse impact monitoring, accommodations, and data governance. 7. Treat an unexplained cutoff or rank order as a decision requiring evidence, not as a natural meaning of the score. 8. Use additional job-relevant evidence rather than letting one personality score stand in for the person or the whole selection process.

If the report cannot answer these questions, the responsible conclusion is not that the assessment has no value. It is that the evidence does not yet support this hiring use with the confidence being claimed. For personal reading or coaching, the same assessment may still offer a prompt for reflection, but that is a different purpose and should be described as such.