Psychometric Tests in Hiring: What Ability, Personality and Judgement Tests Predict

A psychometric test measures ability, personality or judgement. In a 2022 re-analysis, ability tests predicted job performance at .31, conscientiousness at .19.

· NOOVA Cognitive Lab

A thick water-clear glass caliper whose jaws close on a soft frosted glass droplet, the droplet bulging out above and below where the jaws press

A psychometric test is a standardised measure of a mental attribute, such as reasoning ability, personality or judgement, given and scored the same way for everyone and read against a comparison group. The three kinds used in hiring predict job performance unevenly. In a 2022 re-analysis by Sackett and colleagues, ability tests reached a validity of .31, situational judgement tests .26 and a general conscientiousness scale .19. This guide covers what each kind measures, reliability and validity in plain words, sample items and fair use, for the people who choose tests and the candidates who sit them.

What a psychometric test is

A psychometric test turns behaviour into a number under controlled conditions. The Standards for Educational and Psychological Testing, the reference text of the three main US testing bodies (AERA, APA and NCME), define a test as a "systematic sample" of a person's behaviour in a specified domain, "scored using a standardized process". Same items, same instructions, same scoring for every candidate. That is what separates a test from a chat.

The score means little on its own. It is read against norms, the spread of scores in a reference group. The UK Civil Service, for example, reports its judgement test as a percentile against a representative group of applicants, so a percentile of 44 means you did better than 44% of them.

In hiring, three families cover most of what gets called psychometric testing:

  • Ability tests measure maximum performance: how well you reason with numbers, words or patterns when you try your hardest. Items have right answers, and there is usually a clock.
  • Personality questionnaires measure typical behaviour: what you usually do and prefer. Items have no right answers, and you describe yourself. Integrity tests are a close relative aimed at rule-breaking at work.
  • Situational judgement tests (SJTs) show a work scenario and ask you to pick or rate possible responses. They sit between the other two.

The split between maximum and typical performance goes back to the psychologist Lee Cronbach, as McDaniel and colleagues recount, and it explains much of what follows. On an ability test, trying harder helps. On a personality questionnaire, trying harder mostly means describing yourself more flatteringly.

Reliability and validity of a psychometric test in plain words

Reliability is whether a test gives the same answer twice. Validity is whether that answer means what you use it for. A test needs both, and the first does not guarantee the second.

The Standards define reliability as consistency "over repeated applications of a measurement procedure". Developers check it in three ways: the same people take the test twice (test-retest), the items of one scale are compared with each other (internal consistency), or two versions of the test are compared (parallel forms). Each gives a coefficient from 0 to 1. The practical figure is the standard error of measurement: how far one person's score would wander if they could take the test again and again under identical conditions. It is the reason two candidates a few points apart may not differ at all.

Reliability is not automatic. A reliability meta-analysis of SJTs by Kasten and Freund concluded that their scores are "typically rather low and below recommended levels for high-stakes applications".

Validity is subtler. The Standards define it as the degree to which evidence and theory support "the interpretations of test scores for proposed uses". A test is not valid or invalid in itself; a particular use of its scores is, and the Standards call the bare phrase "the validity of the test" incorrect. A bathroom scale can be perfectly reliable and still tell you nothing about height.

In hiring, the main evidence is criterion-related validity: the correlation between test scores and later job performance, on a scale from 0 to 1. For a sense of scale, the 2022 re-analysis puts structured interviews at .42 and unstructured interviews at .19. These are averages: in your own jobs the figure can be higher or lower.

A long water-clear glass spirit level twisted a quarter turn along its length, its frosted glass bubble resting far off-centre

Ability tests: the longest record in hiring

Ability tests are the most studied psychometric test in hiring, and they still predict: .31 in the 2022 re-analysis, down from .51 in Schmidt and Hunter's 1998 review. They come in numerical, verbal, abstract and mechanical forms. Our aptitude test guide goes through each type with examples, and the cognitive ability test post covers the validity debate in detail.

Two sample items, written for this post:

Numerical. A support team closed 240 tickets in March and 300 in April. By what percentage did the number rise? Answer: 25%. The rise of 60 is divided by the March figure, not the April one.

Abstract. What comes next: 2, 6, 18, 54? Answer: 162. Each number is three times the one before.

The cost of ability tests is adverse impact, the legal term for a selection step that screens out one group at a much higher rate. In the same re-analysis, ability tests carry the largest Black-White score gap of the main predictors, d = .79 in standard deviation units. Integrity tests, by contrast, show d = .10. That difference is one of the best arguments for combining test types instead of leaning on one.

Personality tests: what the Big Five predict at work

Personality questionnaires predict job performance weakly, and one trait does most of the work. In the 2022 re-analysis a general conscientiousness scale reached .19 (it was .31 in 1998), while extraversion and agreeableness sat at .10, emotional stability at .09 and openness at .05.

Most work questionnaires are built on the Big Five: extraversion, emotional stability, agreeableness, conscientiousness and openness to experience. The landmark study is Barrick and Mount's 1991 meta-analysis across five occupational groups: professionals, police, managers, sales and skilled or semi-skilled jobs. Conscientiousness was the only trait related to performance in all of them. Extraversion mattered for managers and sales, the two groups whose work runs on dealing with people.

Context helps: in the re-analysis, conscientiousness items framed around work reached .25, with an estimated spread across studies of .00. The difference can be three words. A general item reads "I finish what I start", rated from 1 (strongly disagree) to 5 (strongly agree). The work-framed version reads "At work, I finish what I start".

Integrity tests do better, at .31 (down from .41), but their lower 80% credibility value is .05, so a given integrity test may do far less.

The known weakness of self-report is faking. A meta-analysis of 33 studies by Birkeland and colleagues compared job applicants with people taking the same questionnaires with nothing at stake. Applicants scored higher on conscientiousness (d = .45) and emotional stability (d = .44), with much smaller gaps on extraversion (.11) and openness (.13). In sales jobs, the inflation moved to the traits sales seems to need. Morgeson and colleagues, former editors of the field's leading journals, accepted in 2007 that faking "cannot be avoided" and argued that the bigger problem is low validity.

Situational judgement tests: judgement in a work scenario

A situational judgement test shows a realistic work problem, asks you to choose or rate responses, and predicts job performance at about .26. The figure appears in McDaniel and colleagues' 2007 meta-analysis of 118 validity coefficients from 24,756 people, and again in the 2022 re-analysis. Six scenarios with explained answers are in our free situational judgement test practice.

A sample item, written for this post: A client emails at 5 pm: the report you promised for Friday is now needed tomorrow morning. Your manager has already given you another task for tonight. Rate each response from 1 (very ineffective) to 4 (very effective). (a) Work late and do both. (b) Tell the client tomorrow is not possible and offer Thursday noon. (c) Ask your manager which task matters more, then tell the client what you can deliver and when. (d) Send the client half a report without comment.

The key comes from how experienced people in the role rate each option. For a client-facing role you would expect (c) to come out highest and (d) lowest; (a) looks keen and hides a capacity problem.

The wording of the question changes what the test measures. "What should you do?" (knowledge instructions) and "what would you do?" (behavioural tendency instructions) predicted performance equally well, at .26. But knowledge-instruction SJTs correlated .35 with cognitive ability, against .19 for the other kind, while behavioural tendency SJTs tracked personality more closely: agreeableness .37 against .19, emotional stability .35 against .12. Ask "should" and you get something nearer an ability test. Ask "would" and you get something nearer a personality questionnaire, faking risk included.

SJTs also show a smaller group gap than ability tests (d = .39 and .34), and McDaniel's team found they add validity over cognitive ability and the Big Five combined. Their weak spot is reliability. A situational interview asks the same kind of question face to face.

How the three psychometric test types compare

Read side by side, no single psychometric test is strong enough to decide a hire alone. Validity and Black-White gap (d) in the 2022 re-analysis:

  • Integrity tests: .31, d = .10
  • Ability tests: .31, d = .79
  • SJTs, knowledge instructions: .26, d = .39
  • SJTs, behavioural tendency instructions: .26, d = .34
  • Conscientiousness, work-framed: .25
  • Conscientiousness, general: .19
  • Other Big Five traits, general: .05 to .10

For comparison, a structured interview scores .42 with d = .23, and a work sample test .33. Two patterns stand out. The predictors with the largest group gaps are the ability-heavy ones. And the personality measures that do best are the ones that ask about work, not about life in general.

Timed tests and questionnaires also rarely show how a person notices and corrects their own mistakes, the skill called metacognition. Some teams now look at reasoning in conversation for that. NOOVA HR, for example, runs a 20-minute voice dialogue instead of hours of tests, and a person always makes the decision.

What candidates can expect from psychometric tests

If you are the one taking a psychometric test, each kind rewards a different kind of preparation. Ability tests reward familiarity with the item formats, because they measure your best performance and unfamiliar formats cost time. Personality questionnaires reward consistency more than strategy. Answer as you are at work: the Civil Service guidance, for one, says of its self-assessment questions that you "may be asked about them at interview". SJTs reward reading each scenario for the job's priorities, not for the answer that sounds nicest.

The UK Civil Service Judgement Test shows what an SJT looks like from the inside. You rate each of four actions per scenario as Counterproductive, Ineffective, Fairly Effective or Effective, and each behaviour gets three scenarios. The test is untimed, and most people take two to four minutes per scenario.

You can also ask for changes to how a test is given. In the US, the EEOC says employers must make reasonable accommodations, "including in the administration of tests", unless that would cause undue hardship. The Civil Service test offers subtitles, British Sign Language and written transcripts for its video scenarios.

How to use a psychometric test fairly

Fair use of a psychometric test starts with a job reason and ends with a check on who passes.

Start from the job. The Standards ask for "a clear statement of the intended interpretations of test scores for specified uses" before an employment test is developed and put to use, and for a clear definition of the job content it should reflect. If you cannot say which job behaviour the score predicts, you are not ready to buy the test.

Ask for evidence for your use. The EEOC tells employers to make sure tests are "properly validated for the positions and purposes for which they are used", even when a vendor supplies the documentation, and warns against tests "adopted casually by managers" who do not understand them.

Offer adjustments by default. The same EEOC page says a test must measure the skill it targets, not the applicant's impairment. Put the way to request an adjustment in the invitation, not in the small print.

Check who passes. A neutral test can still be unlawful under Title VII if it excludes a protected group disproportionately and is not job-related. The usual first screen is the four-fifths rule of the Uniform Guidelines: a group's selection rate below four-fifths (80%) of the highest group's rate is generally treated as evidence of adverse impact. Our pre-employment test guide walks through the calculation, and hiring bias covers what else drives the gaps. If software or AI scores the test, the rules in what an AI interview is come on top.

Clinical personality tests count as medical exams

In the US, a personality test that can reveal a mental disorder is a medical examination, and employers may not require one before a job offer. That is the position of the EEOC's 1995 guidance: a psychological test is medical if it provides evidence that would lead to identifying a mental disorder, while a test that measures only "honesty, tastes, and habits" is not.

The line was tested with current employees in Karraker v. Rent-A-Center. Staff seeking promotion to management sat a battery of nine tests that included 502 questions from the MMPI, a clinical personality inventory. In 2005 the Seventh Circuit held that the MMPI "is best categorized as a medical examination" and sent the case back for judgment in the employees' favour on that claim. The practical rule is short: work questionnaires for hiring, clinical inventories for clinics.

A checklist for choosing and running a psychometric test

  1. Write down which job behaviours the score should predict before you compare tests.
  2. Ask the provider for validity evidence from jobs like yours, plus reliability and the standard error.
  3. Choose personality scales written in a work frame over general ones.
  4. Keep clinical inventories such as the MMPI out of pre-offer screening.
  5. Combine any psychometric test with a structured interview instead of using one score as a gate.
  6. Compare pass rates by group at every stage against the four-fifths rule.
  7. Put the way to request adjustments in every test invitation.
  8. Tell candidates what each test measures and how the score will be used.
Psychometric tests predict unevenly: from .31 down to .05: the post in one picture

Sources

Get in Touch

Thank You

We will reply soon