Why Job Analysis Comes Before Hiring Tests

The hiring question comes first

A personality report may describe a tendency such as orderliness, sociability, or emotional steadiness. Hiring requires a narrower decision. The employer needs to know whether information from that report adds useful evidence about performance in this role, under these working conditions, for this intended use.

Job analysis makes the question concrete. It examines what people actually do, what outputs or behaviors matter, how difficult the work is, and which competencies support it. The Office of Personnel Management describes it as a systematic examination of job tasks and required competencies. That is why a result that sounds attractive in the abstract is not enough. A preference for frequent interaction, for example, is not the same as effective customer problem-solving.

The practical sequence is: describe the job, identify important behaviors, choose a construct or skill that plausibly relates to those behaviors, then examine evidence for the assessment and its use. Reversing that order invites a report to dictate the job rather than illuminate it.

What a useful job analysis records

A useful analysis is more specific than a polished job advertisement. It can record recurring tasks, the context in which they occur, the work products produced, and the level of importance or difficulty of each behavior. It can also distinguish behaviors that are essential from those that are merely familiar or convenient.

For example, a support role might involve clarifying an unclear request, documenting the resolution, switching between several cases, and explaining a limitation without escalating the exchange. Those are observable activities. They are more useful starting points than a vague requirement such as ‘positive attitude’ or ‘team player.’ The analysis may show that careful follow-through matters more than constant sociability, or that calm communication matters in one part of the role while technical knowledge matters in another.

People with current, direct knowledge of the work, often called subject matter experts, can describe and rate these requirements. OPM notes that supervisors and experienced incumbents can serve in this role when they understand the job thoroughly. Their judgments still need a documented method. A list assembled from one manager’s intuition is not the same as a systematic analysis.

A worked example: from task to assessment claim

Consider a hiring team choosing an assessment for a customer-support position. The team begins with a task: resolve a customer’s request while recording accurate case notes. It then identifies the behaviors that make the task successful: ask focused questions, remain respectful when the request is frustrating, follow the procedure, and leave a usable record for the next employee.

The team might hypothesize that a tendency related to dependable follow-through is relevant. That is only a hypothesis. The personality report’s description must be checked against the instrument’s definition and items. The team must then ask whether research supports a relationship between the resulting score and a defined criterion, such as supervisor-rated case accuracy or another meaningful performance measure. A general claim that the test finds ‘great service people’ skips all three links.

The analysis can also reveal what personality cannot cover. If the position requires knowledge of a product database, a work sample or knowledge measure may be more direct. If it requires explaining a policy, a structured interview or simulation may show the behavior itself. The personality assessment, if retained, should have a clear role in the wider assessment strategy.

Job analysis is not proof of validity

Validity is evidence supporting a particular interpretation and use of assessment scores. It is not a permanent stamp attached to a test. OPM distinguishes content evidence, which asks whether assessment content matches important job requirements, from criterion-related evidence, which asks whether scores predict or relate to an important outcome such as job performance.

Job analysis is especially important for content arguments because it identifies the work behaviors or competencies that the assessment content should represent. But a self-report personality item is usually not a direct sample of a work task. If the employer wants to forecast later job performance, it may need predictive evidence showing that applicant scores are related to subsequent performance. A published coefficient from another role or population may inform the question, but it does not automatically settle the local decision.

Reliability is a separate question. It concerns the consistency of scores under specified conditions. A consistent score can still measure the wrong construct for the job, or support a claim that the evidence does not justify. The report reader should therefore ask two questions: is the score measured consistently enough for this decision, and is this interpretation useful for this job?

Translate the report without overreading it

A report may present a raw score, a standardized score, a percentile, or a band such as low, average, or high. None of these labels identifies a hiring outcome by itself. A percentile describes position within a stated comparison group; it does not mean that a person will perform at that percentage or outperform a given share of colleagues.

The useful translation is conditional. Instead of ‘high conscientiousness means this applicant will be reliable,’ a careful interpretation might be: ‘the respondent’s score is higher than the comparison group on the instrument’s measure of this tendency; if the instrument’s evidence supports this job-related use, the result may add limited information about behaviors involving follow-through.’ Even that sentence needs the report’s norm group, scoring rules, validity evidence, and intended use.

A hiring panel should also avoid treating a narrative paragraph as a new measurement. The story written around a score can be broader, warmer, or more certain than the evidence. Separate what the instrument measured from the interpretation offered by the report, then connect only the relevant part to documented job requirements.

Use the job context to set boundaries

The same tendency can matter differently in different settings. A role involving frequent negotiation may create more opportunities for interpersonal behavior to affect outcomes than a highly scripted role. OPM specifically notes that personality measures are often used where interaction or teamwork is important and may be less useful where there is little room for personality to affect performance.

Context includes more than the job title. Work pace, supervision, autonomy, customer contact, safety requirements, training, team structure, and available tools can change which behaviors matter. An assessment selected for one customer-service environment should not be carried into a different environment merely because the titles match.

This is also where uncertainty becomes practical. If the analysis finds several equally important competencies, one personality score cannot stand in for all of them. If the work is strongly shaped by training or procedure, a personality interpretation may deserve less weight. A report should narrow the question, not make the decision feel effortless.

Check the evidence behind the intended use

Ask what the assessment was designed to do. A report created for self-reflection may help someone discuss working preferences, but that does not establish its suitability for screening applicants. A tool designed for selection should document its construct, scoring, norming, reliability, validity evidence, population, administration, and interpretation rules in a technical manual or comparable source.

Then match the evidence to the employer’s claim. If the proposed use is to rank applicants, evidence should address selection and the relevant performance criteria, not only whether scores correlate with another questionnaire. If the tool is added to an existing interview or work sample, ask what incremental validity it contributes: does it improve prediction beyond the information already collected enough to justify its cost and risks? OPM defines incremental validity in these terms.

A vendor’s assertion that a test is ‘validated’ is not a substitute for evidence about how it will be used. The EEOC’s interpretive questions state that an unsupported assertion is insufficient where validation evidence is required. The wording matters: evidence is tied to a procedure, a job-related claim, and a population, not just to a product name.

Fairness belongs in the same review

A job-related interpretation is not the whole assessment review. The employer should apply the process consistently, examine subgroup differences and prediction errors where appropriate, and consider whether the assessment creates barriers unrelated to the work. The Uniform Guidelines on Employee Selection Procedures describe validation as a demonstration of job relatedness and recognize content, criterion-related, and construct strategies.

The EEOC guidance also rejects the idea that a merely plausible relationship is enough when a selection procedure has adverse impact. In plain language, ‘this seems related to the job’ is weaker than documented evidence connecting the procedure with important work performance. Local evidence may be relevant, while evidence from elsewhere requires careful attention to similarity of job, context, and applicant population.

Applicant experience matters too. OPM recommends explaining what the assessment measures and why the competencies matter, offering a realistic job preview, and allowing questions. Transparency cannot cure weak validity evidence, but unexplained and invasive questions can make a defensible process harder to understand and use responsibly.

A practical review before anyone acts

Before using a personality assessment in hiring, review the chain in order. First, can the employer show a current analysis of the role, including important observable behaviors and their relative importance? Second, does the report state what construct it measures, how scores are produced, and which norm or comparison group applies? Third, is there evidence for the proposed interpretation and job-related outcome?

Next, check the alternatives. Would a work sample, structured interview, knowledge test, or reference to documented work output address the requirement more directly? If the personality measure is combined with other tools, what information does it add? What uncertainty remains around each score, and how will the panel prevent one result from becoming an all-purpose label?

Finally, review administration and communication. Are the same instructions and decision rules used for everyone? Can applicants understand why the measure is included? Are privacy, accommodation, subgroup, and adverse-impact questions addressed by the responsible professionals? These are not decorative additions to a report. They determine whether its interpretation is proportionate to the decision. Do not choose a trait merely because it sounds desirable, apply a cutoff without a defined comparison and purpose, or let an informal impression turn a score into a character judgment. A general personality assessment also should not be used to diagnose a psychiatric condition.

What the report can responsibly tell you

Job analysis gives a personality assessment a defined question, not a magical hiring power. It can help an employer decide whether a measured tendency is plausibly relevant to specific work behaviors and whether the assessment belongs alongside other evidence. It can also expose a mismatch: a report may be interesting while offering little useful information for this role.

For a reader reviewing a hiring report, the decision point is simple. Proceed only when the job requirements, the construct, the score interpretation, and the evidence line up, and when the remaining uncertainty is acceptable for the stakes. Otherwise, ask for the missing explanation or use a more direct source of evidence.

Use this short checklist: identify the job behavior; name what the score measures; verify the norm group and score meaning; inspect reliability and validity evidence for the intended use; compare more direct assessment options; and record what the score cannot establish. Then take the next question to the live topics library for further report-reading guidance.

Questions readers ask