Can You Compare Self and Observer Personality Scores?

What does cross-rater comparability mean for the score you are reading?

A self score and an observer score can be compared numerically only when evidence supports treating the two forms as measuring the same construct on a sufficiently shared scale. Using the same trait name, response options, or chart does not establish that condition. Cross-rater research shows that comparability is an empirical property of particular forms, dimensions, and score interpretations. It may support some comparisons and leave others unresolved, so a finding for one instrument cannot decide what a different report's numbers mean.

One way researchers examine whether forms work comparably is measurement invariance. In plain terms, it asks whether the relationship between the underlying trait and the answers or scores behaves similarly across groups or forms. Metric invariance concerns whether a unit of change has a comparable meaning; when it holds, associations or relative standing may be meaningfully compared. Scalar invariance adds a comparable starting point, which is needed to interpret group mean differences as differences on the measured construct rather than as a form-related offset. These are specific model tests, not certificates that every score is accurate or that two individuals should agree. Other questions, including reliability and the intended use, remain distinct.

The study titled “Do self-reports and informant-ratings measure the same personality constructs?” examined NEO-PI-3 domains and facets in a sample of 3,253. Its abstract reports that most domains and facets met metric invariance, while Agreeableness and ten facets did not meet scalar invariance recommended for mean comparisons. The result supports a narrower conclusion than “self and observer scores are interchangeable”: relative standing was broadly more comparable than mean levels for those reported exceptions. It also does not show that every non-invariant scale is useless; it identifies a limit on the particular comparison the model supports.

A contrasting result appears in “Psychometric Comparison of Self- and Informant-Reports of Personality.” In that study, self and informant ratings showed configural, metric, and scalar invariance for four dimensions: Well-Being, Social Closeness, Stress Reaction, and Harm Avoidance. This is meaningful counterevidence to any claim that self and observer forms are generally incomparable. But the result belongs to those dimensions and the study's dyad sample; it does not transfer to a similarly named scale in another publisher's report.

The levels matter because a report may invite more than one kind of conclusion. Asking whether people tend to preserve a similar ordering is not the same as asking whether a one-point difference means the same thing, or whether average levels can be contrasted. Metric evidence addresses comparable relationships between the measured construct and scores; scalar evidence addresses whether score origins are aligned enough for mean comparisons. Even the stronger result only answers the modeled comparison for the studied forms and sample. It does not establish that a single respondent's observer saw every relevant situation or that any difference has a practical consequence.

The term “invariance” can therefore sound broader than the inference it warrants. It describes a tested measurement structure, not identical perceptions, perfect agreement, or the cause of a discrepancy. A study can support score comparison under its model while leaving individual-level uncertainty, response context, and other forms of validity open. Conversely, failure to meet one level is not proof that the underlying trait labels refer to wholly unrelated ideas; it means that a particular stronger comparison was not established in that analysis.

Taken together, these studies show why the report itself matters. One cannot infer level comparability from matching labels, and one cannot infer a universal ban from exceptions in one instrument. For the score in front of you, the relevant question is whether technical documentation supports the exact self and observer forms and the level of comparison being made. If that evidence is not available, neither study supplies it by proxy: read the two values as separate ratings rather than as a validated numerical gap.

Sources: Do self-reports and informant-ratings measure the same personality constructs?; Psychometric Comparison of Self- and Informant-Reports of Personality

Why do similar group averages leave an individual report question open?

A finding that self and observer scores are similar on average does not tell you whether your own two scores are directly comparable or how to interpret their gap. A group mean is the arithmetic average across many people; it describes where a sample sits overall, not how close each person's paired ratings are. A correlation describes how two sets of scores vary together across people. Neither statistic, by itself, establishes equal score levels for a particular pair.

“Self–Other Agreement in Personality Reports: A Meta-Analytic Comparison of Self- and Informant-Report Means” pooled results from 152 samples and 33,033 people on Big Five reports. Its abstract reports little average self–informant mean difference overall, with an average delta of −.038, while comparisons involving strangers showed moderate differences. That large synthesis is a reason not to assume that acquainted informants generally produce a systematically higher or lower average than self-raters. Its result answers a population-level question about average differences across studies, however; it does not define an acceptable gap for an individual report or establish that the paired forms share a common metric.

A different view comes from “The Convergent Validity between Self and Observer Ratings of Personality: A Meta-Analytic Review,” which synthesized Big Five self–observer associations. The abstract reports corrected correlations from .46 to .62 across traits, alongside substantial unique variance in both sources and a moderating role for acquaintance duration. These correlations indicate that people who rank relatively higher by one source tend, to a degree, to rank higher by the other. They do not say that the sources assign the same number, or that any particular person's two scores are close.

A simple hypothetical illustrates the distinction without setting a threshold. Imagine two rating sources each score many respondents on the same named dimension. If the overall averages are close, one source may still rate a particular person substantially above the other, balanced by people with differences in the opposite direction. If the scores correlate, higher relative positions tend to travel together across the sample, but that pattern does not establish equal levels for any one pair. These are properties of the distribution, while the reader's question concerns a paired interpretation.

The two meta-analyses also summarize different evidence and should not be collapsed into a single agreement percentage. The mean comparison asks whether source averages differ systematically; the correlation synthesis asks how strongly rankings co-vary. A study can find little mean difference while finding only partial correlation, because positive and negative individual gaps can offset in the average. It can find correlation despite a mean offset, because people may retain similar ordering even when one source tends to use higher values. The reported statistics therefore complement each other, but neither substitutes for evidence that a given report scores its forms on comparable levels.

The findings are compatible rather than contradictory because averages and correlations answer different questions. Two groups of ratings can have nearly equal means while individual pairs vary around those means; a positive correlation can also coexist with a consistent offset between forms. Conversely, a modest correlation does not mean every pair disagrees. The Kim et al. meta-analysis makes broad average self-enhancement an unsupported default; the Connolly et al. synthesis shows meaningful overlap alongside source-specific information. Neither gives a personal tolerance band or tests the comparability of the exact report you are reading.

So do not use the meta-analytic average as a ruler for your own gap, or a correlation as proof that the two displayed values are interchangeable. Those results describe patterns across studied samples and traits. Your individual interpretation still depends on what the report's forms and scoring permit: without that connection, a difference may be worth considering as two perspectives, but its size alone cannot establish a meaningful discrepancy, an error, or who is right.

Sources: Self–Other Agreement in Personality Reports: A Meta-Analytic Comparison of Self- and Informant-Report Means; The Convergent Validity between Self and Observer Ratings of Personality: A Meta-Analytic Review

What can a report document establish—and what remains untested?

To decide whether a report supports reading a self score beside an observer score, look for documentation that connects the exact forms and scoring process to that particular comparison. The joint AERA, APA, and NCME Standards for Educational and Psychological Testing provide a useful framework: score interpretations and uses need evidence appropriate to the claims being made. This is an interpretation-and-use principle, not a special rule prohibiting cross-rater comparison. Applied here, it shifts the question from whether two displays look alike to whether the provider has documented the meaning of comparing these two scores.

Start by identifying the instruments precisely. Does the report or its technical documentation name the self-report version and the observer version, including any differences in items, instructions, response options, or scoring? A shared trait heading is not enough to establish that two forms function alike. A label such as “planning” could organize related questions on both forms while the items still ask about different behaviors or perspectives. Form identification lets a reader tell whether the provider is discussing a tested pair of instruments or simply presenting parallel-looking sections. If version details are absent from the short report, check whether the provider supplies a manual or technical report elsewhere.

Next, examine what the displayed values are and how the provider produced them. A raw total, transformed score, percentile, band, and position on a personal profile each carry different meanings. Useful documentation explains the scoring transformation and reference used, and says whether the scores are intended to be compared with one another, interpreted separately, or read only as prompts for reflection. A common graphic scale can help readers see values together, but a shared axis alone does not explain whether equal distances or levels have comparable meaning across forms. Conversely, a report may omit technical notes from the graphic while making them available in supporting documentation.

Then match the evidence to the interpretation being offered. If the provider invites a numerical comparison, look for technical evidence that addresses the exact forms and the intended comparison, rather than evidence that merely supports each form separately. The Standards' framework makes the claim and its use central: evidence adequate for describing a person's answers in two separate sections may not establish that their numerical difference measures a difference on a shared scale. Documentation should also make clear the relevant limits. A detailed manual is not automatically strong validation; the reader still needs to see what was studied and what conclusion that evidence supports.

A practical conclusion follows from the documentation, not from the design of the chart. When the exact forms, score construction, comparison intended, and supporting evidence are documented, interpret the scores within those stated boundaries. When the report does not establish a direct numerical comparison, the honest reading is that this comparison has not been established by the material available to you. That does not show the scores are false, that either rater is mistaken, or that the report cannot support separate reflection. It marks the edge of what the document has demonstrated; the two ratings can still be considered as distinct accounts while the numerical relationship remains open.

Sources: Standards for Educational and Psychological Testing

What does the shape of two profiles let you compare?

A profile has at least two visual properties that can be confused: its shape, or the ordering of dimensions within it, and its level, or where its scores sit on the displayed scale. Two profiles might place “planning” above “adaptability” in the same relative order while showing different values overall. That resemblance in ordering does not by itself tell you that the levels are comparable. The chart answers only the questions its score labels, axes, and transformations permit. Before reading a gap as a meaningful difference, identify whether the display uses a shared metric or transforms each profile in a way that changes the level information.

Teaching example, invented numbers, not observed data: imagine that a self profile shows three generic dimensions as 8, 6, and 4, while an observer profile shows them as 6, 4, and 2. In both, the first dimension ranks highest, the second is in the middle, and the third ranks lowest. The ordering is the same, yet the displayed values are lower in the second profile. If these numbers were produced on one common scale with the same scoring rules, the level gap might be relevant to interpret. If the report rescaled each profile separately, the same visual pattern would not establish that one rater sees a generally lower level.

A transformation can preserve relative shape while removing or changing absolute level. For instance, centering scores within each profile subtracts that profile's own average from its values. The resulting numbers show which dimensions are above or below that profile's average; they no longer preserve the original average level. Standardizing within each profile can also express scores relative to that profile's spread. In either case, similar peaks and dips may remain visible even though the transformed values cannot answer which rater used higher absolute scores. These are conditional examples of what such transformations do, not a claim that a particular report applies them.

Read the chart's labels before comparing its geometry. Check whether the axis begins at zero or another point, whether it names raw or transformed scores, whether both profiles share identical limits, and whether the notes describe centering, standardizing, or separate rescaling. A chart may overlay two lines for readability while still presenting values that have been prepared differently. Equally, some providers may use a validated common metric and shared axis that support a level comparison. The visual alone cannot settle which case applies; the scoring notes and provider's technical explanation determine the warranted reading.

Finally, visual overlap is descriptive, not a test of statistical equivalence. Lines that nearly touch on a small graphic are not thereby proven equal, while visibly separated lines are not automatically a meaningful or reliable difference. The display may not show uncertainty or the evidence needed to judge either claim. Without a named report and its scoring documentation, keep the conclusion narrow: you can describe whether the displayed profiles have a similar relative ordering, but you cannot infer comparable absolute levels from shape alone. If the axes and documentation establish a common metric, a level comparison may be available; otherwise, treat the chart's pattern and its numeric level as separate questions.

What kind of evidence can each rater contribute?

When the question is what behavior a report may help explain, prediction against an outside criterion is different from agreement between two scores. Predictive validity, in plain terms, asks whether a score helps anticipate a separately observed outcome or behavior. A self rating can predict one criterion better, an observer rating another, or both can contribute information. That evidence concerns a relationship in the studied sample; it does not establish that an observer is the correct scorer of an unnamed report or that a score difference itself predicts anything.

In “Who Knows What About a Person? The Self–Other Knowledge Asymmetry (SOKA) Model,” the researchers studied 165 participants using self ratings, ratings from friends and strangers, and behavioral tests. The design compared perspectives with selected behavioral criteria, rather than asking only how closely the rating forms matched. The reported results varied by trait: self ratings had a relative advantage for neuroticism related traits, friends for intellect related traits, and self and other perspectives were similarly accurate for extraversion related traits. The pattern is conditional, not a ranking of raters overall. It suggests that which vantage point carries useful evidence can depend on the trait and on what behavior is used to assess it.

The SOKA result also clarifies why a behavioral criterion should not be mistaken for a universal ground truth. A task samples a particular behavior under particular conditions. A person's report may concern a wider range of situations, including private thoughts that an observer cannot see; an observer may witness actions the person does not readily notice. The study tests whether ratings relate to its selected behavioral measures. It does not prove that those measures exhaust the trait, that one rater has a complete view, or that its relative advantages transfer to another questionnaire, population, or decision.

A broader synthesis reaches a compatible conclusion at a different scale. “An Other Perspective on Personality: Meta-Analytic Integration of Observers’ Accuracy and Predictive Validity” combined three meta analyses covering 44,178 targets in 263 samples. Its abstract reports that observer ratings can contribute accuracy and predict some outcomes; frequency of interaction and intimacy were related to accuracy in the summarized evidence. These are correlates reported across the included research, not a recipe for selecting an informant or a guarantee that frequent contact improves every trait judgment. The finding gives a reason to take informant information seriously when observers have relevant exposure. It does not mean that closeness automatically makes a rating accurate, nor does an aggregate relation identify the accuracy of one person's chosen observer.

Together, these studies support complementarity as a possibility, not a verdict. The SOKA study shows trait dependent results against particular behavioral tests; the meta analytic synthesis reports observer accuracy and prediction across studied samples and criteria. Neither study compares the exact forms in an arbitrary commercial report, supplies a tolerance for a pair's score gap, or establishes an employment conclusion. Their useful implication is narrower: a perspective may contain information that another vantage point misses, while the value of that information depends on the trait, criterion, observer exposure, and proposed use. Prediction about studied criteria cannot settle who is right about a displayed score.

Sources: Who Knows What About a Person? The Self–Other Knowledge Asymmetry (SOKA) Model; An Other Perspective on Personality: Meta-Analytic Integration of Observers’ Accuracy and Predictive Validity

How can disagreement contain different information rather than a score error?

Two good faith ratings can diverge because the people answering do not have identical access to the behavior behind a broad trait label. A person has direct access to private plans, worries, intentions, and effort, but may not notice how those inner experiences appear to someone else. An observer can see actions and their effects, but cannot directly inspect the thought or intention that preceded them. This difference in vantage point makes distinct answers possible even when both raters understand the same words and answer sincerely.

The SOKA study, “Who Knows What About a Person? The Self–Other Knowledge Asymmetry (SOKA) Model,” gives a trait specific reason to take this possibility seriously: its behavioral criterion findings did not favor the same perspective for every selected trait. This is not evidence that every disagreement reflects complementary knowledge. Rather, it shows why a single general rule about who knows a person best would oversimplify the relation between perspective and the behavior being assessed. The kind of behavior and the access needed to notice it matter to what each account can represent.

Observation windows can differ as well. A self rating may draw on recurring experiences across settings and time, while an observer's answer may be anchored in the occasions they share with the person. If a behavior appears mainly in private, in unfamiliar settings, or under conditions one rater seldom sees, the available examples will naturally differ. Conversely, a repeated interpersonal behavior may be more visible to someone who encounters its consequences than to the person acting. A report item may not tell respondents which occasions to recall, so two answers can summarize different slices of experience rather than opposite judgments about one identical event. A general frequency word can also invite one rater to count occasional exceptions and another to summarize the usual pattern.

Interpretation adds another possible source of divergence. Words such as “organized,” “direct,” or “calm” can point to more than one observable action. One person might understand organization as privately planning ahead; another might mean communicating a timely plan so others can coordinate. An illustrative contrast, not a study finding or a real person's case: a self rater could endorse being organized because they plan carefully, while a close observer could be less certain because plans are not always shared early. The responses concern related but nonidentical behaviors. Neither answer alone reveals whether the other person used the same meaning.

Different access and interpretation are only possible explanations among several. A response can also be affected by an unclear item, inattention, memory, a different reference period, or ordinary measurement error. A person may behave differently across settings, and both ratings can accurately describe the situations each rater sees. The studies discussed here do not diagnose the cause of any one reader's gap. A discrepancy is therefore a prompt to clarify what behavior and occasions each answer refers to, not proof of dishonesty, bias, error, or a stable contradiction in character.

Sources: Who Knows What About a Person? The Self–Other Knowledge Asymmetry (SOKA) Model

When is disagreement complementary information rather than a score contest?

A disagreement can be useful without proving that either rating is the correct number. The relevant question may be what each account can tell you for a distinct purpose: one may describe the respondent’s own experience, while another records patterns visible to someone else. That possibility supports listening to both accounts as separate information. It does not by itself justify calculating a combined score, deciding that the higher rating is more accurate, or treating the gap as a finding about the person.

The systematic review “Concordance and Added Value of Informant- versus Self-Report in Personality Assessment: A Systematic Review” covered 46 studies of personality-pathology assessment in adults and older adults. Its abstract reports moderate concordance and complementary contributions: informants added some interpersonal and externalizing information, while self-reports captured intrapsychic characteristics. This is a bounded clinical literature about personality pathology, unlike validation research for a general personality report or a workplace instrument. The finding cannot be transferred to an unnamed report, used to interpret its scales, or treated as evidence for diagnosis. Its limited relevance here is that a review found reason to retain distinct vantage points in that particular assessment context, rather than declaring one source universally sufficient.

That example counters a simple one-source-wins rule, but it does not establish that every difference contains useful information. A gap might reflect different questions, settings, interpretations, or response conditions; it might also be uninformative for the decision at hand. The review does not tell a reader how much difference to expect in a general report, how to weight a particular informant, or whether one account better represents a person’s work behavior. Complementarity is a possibility to examine, not a property conferred automatically by having two raters.

A careful reading therefore preserves the accounts for the questions each can actually address. If a self-rating concerns a person’s internal experience, keep that answer attached to self-report; if an informant describes what they have observed, keep that observation attached to its source and context. Do not average the values unless the provider has established that the scores can be combined for the intended interpretation. Do not rank one source as the winner, infer a diagnosis, or choose the more familiar rater as inherently more accurate. Familiarity may describe exposure, but it cannot establish that the exact score is valid or comparable.

The practical distinction is between learning from two perspectives and treating them as interchangeable measurements. The first can leave both reports intact and ask whether their different contributions matter to a modest, clearly framed question. The second requires evidence about the particular forms and scoring, which the clinical review does not supply for a general assessment. If there is no decision that needs resolving, leaving the difference open is also reasonable. A reader may recognize that the accounts answer different questions without forcing them into a single verdict.

Sources: Concordance and Added Value of Informant- versus Self-Report in Personality Assessment: A Systematic Review

What is the smallest useful next step when the gap remains?

Use a short, reversible decision path. First, check whether the provider explains the exact self-report and informant forms, how their scores were produced, and whether it supports the comparison you want to make. If it does, read the values only within those stated boundaries. That allows a cautious interpretation of the comparison the documentation supports; it does not make the result a general judgment about the person.

Second, if the provider does not document a comparable interpretation—or the needed information is unavailable—write the two values down separately and leave their numerical relationship unresolved. Keep the source attached to each value so that neither becomes an unqualified description of the person. You do not need to find another test, contact the observer, or decide which rating is more trustworthy. An open question is a complete and responsible outcome when the report cannot settle it.

Third, ask about behavior only if the underlying issue matters to a real, low-stakes question and a conversation is appropriate. Choose one specific context and one observable action, rather than asking someone to defend a score or characterize your personality. For example: “In the last project we worked on together, when did you see me share a plan early enough for others to coordinate?” The answer can clarify what that observer had in mind and whether it bears on the question; one example, or even several, does not prove a stable trait or settle a score.

This sequence is practical reasoning, not a validated assessment protocol. Keep the action proportionate to the consequence: document what is known, note what remains unclear, and stop if further clarification would not change a decision. A reader without access to a manual can simply preserve the two ratings separately. A reader who does not want to discuss them with the observer can leave the numeric interpretation open. Neither choice requires confrontation or another assessment, and either can be revised if trustworthy provider information or a relevant question later becomes available.

Keep the two questions distinct as you decide whether to continue: “What does this score difference mean?” is a measurement question for the provider’s documentation; “What happened in this situation?” is a practical question that may be clarified by a relevant example. An answer to the second can help with that situation, but it cannot retroactively validate the score comparison. If the concern is consequential, such as a formal employment decision, this informal conversation is not a substitute for the process and evidence appropriate to that use. For ordinary reflection, a narrow question and a note of uncertainty are enough.

What should you do next with this report?

Check the provider’s documentation for the exact self-report and observer forms, the score type, and the interpretation it supports. If it establishes how those two results may be compared, use them only within that stated scope. If the documentation is unavailable or does not support a cross-form interpretation, keep the ratings attached to their respective sources as separate perspectives. Do not turn the unresolved gap into a single score. The report provider’s documentation remains the relevant source for interpreting the report you already have.

If a separate, low-stakes question about your own work tendencies would help you decide what to reflect on, the live Work Pattern Report offers a private self-report across decision and collaboration continuums with a local summary. It has no norms, cutoff, or selection score; it cannot arbitrate this disagreement, establish career fit, or recommend a job. **Turn a work-pattern question into observations you can examine.** If you are unsure which tendencies to explore, start with your own pattern and compare it with specific situations. [Explore the Work Pattern Report](/assessment), or [read more Personality Report guides](/topics).

Sources: 100-item work pattern report | Personality Report; Guides | Personality Report

Questions readers ask