What to Ask When a Personality Report Imputes Missing Answers

A finished report can still contain an estimate

A personality report can look complete even when its scoring used an estimate. Ask what was missing, what value or score was estimated, and what evidence supports that procedure for this instrument and its intended use. Imputation means replacing or estimating missing data so a calculation can proceed; it does not recover the answer you would have given. A displayed score cannot, by itself, show that every response was observed. That distinction matters without making the result automatically unusable. In “Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques,” researchers compared methods for multi-item scales. They reported that regression and within-person mean substitution were promising in the tested conditions and often outperformed listwise deletion, which removes incomplete cases. This shows that retaining information can be preferable in some settings, not that an undisclosed rule suits your report. The ETS article “Measuring the Uncertainty of Imputed Scores” treats uncertainty around imputed scores as a distinct measurement issue. Until you know the rule and how its uncertainty bears on the result, regard dependent conclusions as provisional. Trace the score’s data history before deciding how much weight its personality interpretation can carry.

Sources: Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques; Measuring the Uncertainty of Imputed Scores

What kind of missing answer are we talking about?

Start by locating where the blank occurred. Was one item left unanswered in an otherwise completed report? Was it never presented because the session ended early? Did a technical interruption prevent an answer from being saved? Was the test deliberately designed so different people received different subsets of items? Or was an item unavailable for everyone because it had been removed? They do not describe the same event, and a scoring rule evaluated for one pattern does not automatically fit another. For an individual omission, the respondent saw an item but no answer appears in the record. It does not explain why it was skipped or what answer might have been given. A not-reached item is different: the session may have stopped before the item appeared. With planned incomplete designs, some missing responses are expected because the instrument intentionally distributes item sets; the provider should describe that design and its scoring basis. An item-wide absence is different again. In *Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items*, an ETS research report published in 2023, researchers compared three methods using real educational-test data when responses to an item were missing for all test takers. They reported negligible error in scaled-score estimates for the methods they studied. That concerns a shared item-level gap in educational assessment, not one person’s skipped personality item, and cannot validate a consumer personality report. A technical disruption calls for a more specific account of the administration record. The *Guidelines for Technology-Based Assessment*, published by the International Test Commission and Association of Test Publishers in 2022, address scoring when technology disruption leaves an assessment incomplete. Their scope is technology-based testing; they are not a general endorsement of filling skipped personality answers. If a screen froze, a connection dropped, or a session ended unexpectedly, ask whether the item was displayed, whether an answer was recorded, what event the system logged, and how the provider handled the incomplete session. The blank itself does not establish the cause. Ask the provider to identify the affected item or scale output, whether the item was presented and saved, and which category of missingness the scoring policy covers. Without that classification, you cannot judge whether the cited method fits; keep the interpretation provisional until the data history is clear.

Sources: Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items; Guidelines for Technology-Based Assessment

What exactly did the system estimate?

Ask the provider whether it estimated a response to a particular item, calculated a scale score from the answers you completed, estimated a broader score from a statistical model, or withheld the result. These are different operations. “Imputed” can describe more than one of them, so ask which level was affected: an item, a facet, a domain, or the report’s overall score. Also ask how many outputs relied on incomplete data and where the scoring rule appears in the manual or policy. That turns a vague disclosure into a question the provider can answer.

With single-value item replacement, the system inserts one chosen value where an answer is absent and then scores as if that value were present. The value may come from an item rule, other answers, or a model. Person-mean scoring, sometimes described as prorating, instead calculates a scale total or average from completed items and adjusts it to the scale’s scoring format. It does not identify what the respondent would have selected on the omitted item. Ask whether the scale permits it and how many answers are required. Do not apply a rule copied from another test or recalculate the report from an unofficial formula.

Model-based approaches estimate missing data using relationships in the observed responses or other variables. Multiple imputation creates several plausible completed datasets and combines results, so it can represent uncertainty differently from inserting one fixed value. The provider should identify the model family and whether the reported personality interpretation reflects that uncertainty; a technical-sounding method name alone does not establish suitability. Ask whether the result is an estimated item response or a directly estimated scale score, since the second does not claim to know the missing answer.

The alternatives clarify what is at stake. Complete-case scoring uses fully answered records; a provider may withhold an incomplete scale. Withholding avoids presenting an unsupported score, though it also leaves the reader without that interpretation. “Missing Data in Multiple Item Scales” compared deletion, regression, hot-deck, and mean substitution in a Monte Carlo analysis; its abstract says regression and within-person mean substitution looked promising, and imputation often outperformed deletion. It does not validate an individual report. “Missing data and psychometric properties of personality tests” simulated missingness in ESQUIZO-Q data and compared eight methods on reliability, factor structure, and score ordering. Those results concern that instrument and simulation. An ETS study, “Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items,” examined an item missing for all test takers in educational assessments. Ask what it did, whether this instrument supports the operation, and how the interpretation changes.

Sources: Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques; Missing data and psychometric properties of personality tests; Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items

Why imputation and dropping incomplete cases are not simple opposites

Ask what alternative the provider considered and why its chosen method fits the instrument. Replacing an answer adds assumptions; excluding an incomplete record can discard information and change who contributes to an analysis. Neither choice is automatically more careful. The comparison depends on what the report estimates and how the answer went missing. Listwise deletion, also called complete-case handling, removes a respondent's record from a calculation when any required answer is missing. This reduces the usable sample and may change its composition. Imputation estimates a missing value or uses a model to calculate a result while retaining available information. Person-mean scoring calculates a respondent's average across answered items on a scale and uses it for the blank. It assumes those items inform the unanswered one. A model-based method uses relationships among observed responses under its own assumptions. More elaborate methods are not automatically better for a particular report. The abstract of “Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques” describes a Monte Carlo comparison of listwise deletion, regression imputation, hot-deck imputation and two kinds of mean substitution. The authors report that regression and a person's mean on other scale items were promising in tested conditions, and imputation often outperformed listwise deletion. Their rationale is specific to multi-item scales: answers to related items can inform a blank. This argues against discarding every incomplete record by rule; it does not establish that a provider's method fits every instrument or individual narrative. A personality-specific comparison adds context, with limits. “Missing Data and Psychometric Properties of Personality Tests” reports two simulations based on ESQUIZO-Q schizotypy data: one used 3,056 adolescents and the other samples of 200, with four simulated missing-response levels and eight procedures. It examined reliability estimates, factor structure and score ordering. Results varied with method, missing-data amount, sample size and outcome. Ask what evidence matches the score at issue; this study cannot validate another test. The target matters. A study-level result does not necessarily justify a precise individual facet statement. The ETS report “Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items” studies an item missing for all test takers on educational tests, unlike one respondent's skipped personality item. Ask the provider to compare permitted options for this instrument and interpretation. The question is not whether blanks should always be filled or deleted, but why this method fits this score.

Sources: Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques; Missing data and psychometric properties of personality tests; Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items

Could the reason an item is blank change what the estimate means?

Ask whether the scoring procedure assumes that the missing answer is unrelated to the answer that would have been given, and what evidence supports that assumption. A blank cannot tell you why it occurred. If wording, language, sensitivity or accessibility affected an answer, estimates from other responses may miss something relevant. These are possibilities to investigate, not motives a report can infer from one omission. Researchers often describe missingness using three patterns. “Missing completely at random” means the chance that an answer is absent does not depend on the answer itself or on other measured information. “Missing at random” means absence may be related to information that is observed, such as a recorded session condition, but not to the unobserved answer once that information is taken into account. “Missing not at random” means the chance of absence may still depend on the answer that is missing, even after observed information is considered. They describe data-process assumptions, not why one person left one item blank. That distinction matters because an imputation rule can only be judged against assumptions that fit the situation. If people who skip an item differ in an observed way, a provider may be able to examine whether accounting for that information changes the estimate. If skipping depends on the unobserved response itself, the available data alone may not settle the question. The provider should explain what was tested and what remains uncertain; a filled field does not show that the assumption holds. Ask what evidence the provider has about the items and response patterns: whether omissions cluster on particular questions, whether the item wording and instructions were reviewed for clarity, and whether the scoring rule was checked across relevant groups or administration conditions. The review *Psychological, psychiatric, and behavioral sciences measurement scales: best practice guidelines for their development and validation* describes target-population evaluation and cognitive interviewing as parts of item development, including checks for ambiguity. It supports asking about item clarity, not assuming that this report’s missing-data assumptions were tested. Also ask whether the provider can explain its findings without sharing other respondents’ private records. Group-level summaries or a description of the validation method may answer the question while protecting individual data. The *Guidelines for Technology-Based Assessment* make a separate, narrower point: their imputation guidance concerns incomplete sessions caused by technological disruption, and they explicitly exclude ordinary omitted or not-reached items. For technology-related loss, they call for empirical validation and attention to score quality and fairness. It is not a universal rule for skipped personality items. If the answer is simply that the software fills blanks, the assumption remains unresolved. Ask what pattern of missingness the rule was evaluated for, what evidence makes that pattern relevant here, and how the remaining uncertainty affects the report’s interpretation. Without that explanation, the estimate may still serve as a prompt for reflection, but it should not carry more weight than its assumptions can support.

Sources: Psychological, psychiatric, and behavioral sciences measurement scales: best practice guidelines for their development and validation; Guidelines for Technology-Based Assessment

What can personality-specific evidence actually establish?

Ask whether the scoring rule was evaluated on this instrument, or a comparable version, with people and response patterns relevant to the report’s intended use. Evidence from another scale can show that a method is plausible under certain conditions; it cannot validate the score in front of you. Ask whether the study’s target and limits resemble the claim the report makes. Check six features: instrument and version, including language; population and intended use; amount and pattern of missing data; method and comparison reference; outcomes measured; and limitations. “Personality assessment” is too broad a match by itself. Evidence about group score ordering, for example, does not necessarily show that an individual narrative is stable. Preserving scale structure also does not establish that a particular work or coaching interpretation is supported. One possible design is a masking study. Researchers start with completed responses, hide known answers, apply a scoring rule, then compare estimates with original answers and scores. They might examine score ordering, structure or reliability. This tests simulated patterns; it cannot establish that real omissions arise in the same way or that findings transfer to another inventory or use. Ask what was masked and which outcome the comparison supports. This is useful evidence, not a universal required test. The personality-specific study “Missing data and psychometric properties of personality tests” shows the value and limits of direct evidence. It used real responses to ESQUIZO-Q, a schizotypy measure, as the basis for two simulations. One drew on 3,056 students aged 14 to 18; the other examined samples of 200. Researchers imposed four levels of randomly missing responses and compared eight procedures, assessing Cronbach’s alpha, factor structure and score ordering. They reported that even simple procedures could be practically reasonable under low missingness in those conditions. At higher missingness, results differed more across procedures; in the larger-sample simulation, no method maintained the examined factor structure at 30% missingness. The study shows why amount missing and outcome matter; it does not validate a consumer report or individual interpretation. “Psychological, psychiatric, and behavioral sciences measurement scales: best practice guidelines for their development and validation” supports asking for documentation tied to systematic scale development and evaluation, including dimensionality, reliability and validity. It is guidance for measure development, not a validation study of a provider’s imputation rule. Ask the publisher to connect its explanation to the instrument, population and intended interpretation. “Measuring the Uncertainty of Imputed Scores” treats uncertainty associated with imputed scores as a distinct measurement issue. That supports asking whether the estimate’s uncertainty is reflected in the reported result; it does not prescribe an interval for personality reports. Evidence becomes relevant when the instrument, population, missingness process and target interpretation fit closely enough to make the comparison informative. If a link is absent, treat the study as background rather than proof, and keep the report’s conclusion provisional until the gap is addressed.

Sources: Missing data and psychometric properties of personality tests; Psychological, psychiatric, and behavioral sciences measurement scales: best practice guidelines for their development and validation; Measuring the Uncertainty of Imputed Scores

Would the report's conclusion change if the blank were handled differently?

Ask for a sensitivity comparison under the instrument's approved alternatives: does the missing answer alter the numeric score, facet, band, percentile or narrative conclusion? If a small change in handling changes the interpretation, that instability matters even if the provider cannot disclose the hidden item or exact response value. It checks whether a conclusion holds when a permitted scoring choice changes. Separate movement in a number from movement in its meaning. A score can shift while the report remains in the same broad band; a smaller shift can cross a cutoff and change the label. Percentiles compare a score with a norm group, while bands depend on stated cutpoints. Neither display becomes stable merely because it shows a precise number. Ask whether the provider can compare results across alternatives it considers acceptable, or at least say whether the facet, band or wording changes. For example, imagine a report that assigns a broad description using a cutoff. If every permitted treatment of one blank leaves the result on the same side, the description is less sensitive to that choice. If one treatment moves it across the boundary, the headline depends on scoring and should be qualified, narrowed or withheld. This is a hypothetical illustration, not a study result. Evidence makes this question useful, but does not answer it for an unrelated report. In two simulation studies using actual ESQUIZO-Q data, a personality self-report measure of schizotypy, researchers examined four levels of missing responses and eight handling procedures. They assessed reliability estimates, factor structure and score ordering. The study reported that simple methods could be practically reasonable at low missingness in its simulated conditions; performance varied by method and outcome, and higher missingness could worsen results. This supports checking which outcome matters rather than assuming one statistic settles the question. It does not give a sensitivity result for your instrument or report. A contrast comes from the ETS report on educational tests where an item was missing for all test takers. Its real-data comparison assessed scaled-score accuracy and reliability; the abstract reports negligible imputation error. Because the missingness pattern and test purpose differ, that finding cannot establish that one omitted personality item is harmless. It does show why a provider should identify the target being compared. The APA's test-taker guidance says results should be explained in commonly understood terms and describes access, upon request, to score interpretation information and possible score change from measurement error. Ask whether the provider can explain stability in plain language, even if proprietary details remain protected. If the conclusion changes across approved treatments, give the narrower interpretation less weight. If it remains stable, that supports cautious use, not proof that the score is accurate or suitable for every decision.

Sources: Missing data and psychometric properties of personality tests; Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items; Rights and responsibilities of test takers: Guidelines and expectations

How should the report show uncertainty?

Ask whether the report’s stated uncertainty includes the effect of imputation, what its interval or precision describes, and whether the interpretation changes when estimated data are involved. A decimal, percentile, or assured paragraph does not show that this uncertainty was counted. Measurement error is expected variation in a score because a measure cannot capture a person’s standing with perfect precision. Imputation uncertainty is uncertainty introduced when an unobserved response or score component is estimated rather than observed. These are related, but one does not automatically cover the other. A report may show a conventional standard error or confidence interval without saying whether it accounts for the missing response and the assumptions used to estimate it. Ask what the reported interval includes, what method produced it, and whether it applies to the score, percentile, or narrative category. A single substituted value can make an incomplete record look complete. A single choice can hide other plausible values. Some model-based approaches, including multiple imputation, can represent variation across plausible values, but procedures differ. Do not infer that a report uses such a method from a polished result; ask the provider to identify the procedure and its documented output. Ask whether uncertainty matches the operation and the conclusion. A practical request is: “Does the uncertainty shown include the missing-response procedure? What does the interval or precision estimate cover, could the missing answer change the score band or narrative, and how is that uncertainty reflected?” Where documentation supports it, a provider might report an interval, standard error, provisional marker, suppression rule, or plain-language caveat. If the report supplies only a point estimate, ask whether a sensitivity comparison is available and whether the interpretation remains stable under permitted alternatives. The ETS research record “Measuring the Uncertainty of Imputed Scores” identifies imputed-score uncertainty as a distinct measurement topic. The record does not provide enough readable findings for a specific procedure or numerical recommendation. This supports asking, not a conclusion about any personality report. The American Psychological Association’s “Disclosure of Test Data and Test Materials: Just the FAQs” encourages understandable explanations of scores, including confidence intervals and significant reservations about accuracy or interpretive limits. It also recognizes that roles and test security can limit disclosure. Disclosure can be limited by roles and test security; the FAQ is not a universal guarantee. If uncertainty is unavailable or not connected to the imputed result, avoid treating a fine-grained rank or narrative distinction as more precise than its evidence warrants. Keep narrow categories and detailed descriptions tentative. This account cannot establish validity for every use or show that the missingness assumption was appropriate. The practical standard is to know whether the displayed uncertainty reaches the estimate that shaped the report, and to narrow your conclusion when it does not.

Sources: Measuring the Uncertainty of Imputed Scores; Disclosure of Test Data and Test Materials: Just the FAQs; Rights and responsibilities of test takers: Guidelines and expectations

What can I reasonably request without asking for secure test content?

Ask for enough information to trace the score, assess the method, and understand its limits. The American Psychological Association’s “Rights and responsibilities of test takers: Guidelines and expectations” says test takers should receive an explanation of results in commonly understood terms; it also describes requests for general scoring information and information about sources used to interpret results. The document calls these professional expectations, not rights with the force of law, and recognizes limits where disclosure would expose proprietary information. A focused message can make the request answerable. First, ask which responses or scale results were incomplete, and whether the issue was an unanswered item, a technical interruption, or another recorded data problem. Second, ask what happened next: was an item value estimated, was a scale score calculated from answered items, or was an interpretation withheld? Ask which facet, score, band, percentile, or narrative statement was affected. The word “imputation” alone does not identify the operation. Third, ask where the rule appears in the manual or scoring policy, including any threshold at which the report withholds a result or requests completion. Fourth, ask what evidence supports applying that rule to this instrument, version, language, relevant population, missingness pattern, and intended interpretation. The APA guidance supports asking for the sources used to interpret results, including technical manuals, reports, norms, and comparison-group descriptions, subject to applicable limits. Fifth, ask whether the provider checked how the interpretation changes under alternatives permitted by its own scoring policy. Ask whether the broad narrative remains stable or a facet or band changes. Sixth, ask what uncertainty is included in the displayed result, what the stated precision refers to, and what use the result is intended to support. If there is a process for questioning a score or correcting a processing error, ask how to use it. Read the reply for traceability rather than reassurance. A specific policy reference and a bounded account of the affected output help you understand the result. “Our system handles missing answers” leaves the rule, evidence, and effect unresolved. A provider may have sound reasons to protect test materials or proprietary details; that refusal alone does not show that its scoring is poor. The APA’s “Disclosure of Test Data and Test Materials: Just the FAQs” discusses the practical and ethical concerns around disclosure. These concerns make a bounded request sensible here. Treat this as a reasonable request for explanation, not a guarantee of access to every method or record in every setting or jurisdiction. If the provider cannot explain the score’s data history, keep conclusions that depend on the estimated information tentative.

Sources: Rights and responsibilities of test takers: Guidelines and expectations; Disclosure of Test Data and Test Materials: Just the FAQs

Choose the next step by the report's stakes

If the provider explains a rule evaluated for the relevant instrument and missing-response pattern, and the interpretation stays stable under permitted alternatives, use it as a tentative prompt for low-stakes reflection. Keep its caveat attached. Stability supports cautious use; it does not establish a diagnosis or settle who you are. The APA test-taker guidance supports asking for scoring and interpretation information, while recognizing test-security and contextual limits. When the provider cannot explain the operation, or the interpretation changes materially under another permitted treatment, ask whether it can review the record, issue a corrected report, or withhold the affected interpretation. Pause reliance on that conclusion for hiring or another consequential decision unless evidence supports this instrument, procedure and use. A general personality report does not diagnose a condition or determine a person’s future. The APA disclosure FAQs support understandable explanations while recognizing limits around secure items; they do not guarantee a particular remedy in every setting. For further reading on scores, evidence and report limits, consult the publication’s [personality assessment and report literacy library](/topics). The standard is an explainable, appropriately bounded score history. The conclusion could change if instrument-specific evidence shows the rule fits this missingness pattern and intended use, and its uncertainty does not materially alter the interpretation. Even then, the procedure estimates; it does not recover an unrecorded answer.

Sources: Rights and responsibilities of test takers: Guidelines and expectations; Disclosure of Test Data and Test Materials: Just the FAQs

Questions readers ask