Any HR leader, hiring manager, or recruiter who is interested in using pre-employment tests knows how important is to tests that are validated. After all, the goal of using pre-employment tests is to drive better hiring decisions, which in turn lead to better performance outcomes across the entire organization. The validation process provides the backbone for a test’s ability to predict and drive these positive outcomes.
The validation process is by no means simple, often requiring a careful study that involves gathering different pieces of evidence to provide a scientific basis for interpreting the test scores in a particular way. There are a variety of different validity measures that can be used to provide this evidence, from criterion validity (how well a test correlates with a certain outcome, such as job performance or turnover) to construct validity (whether a test is measuring what it’s supposed to be measuring).
(For a more in-depth explanation of the validation process, read here.)
But another related, equally important concept is what is referred to as the “reliability” of a test. Reliability refers to a test’s ability to produce consistent results over time. For example, let’s say you take a cognitive ability test and receive 65th percentile on the test. Then a week later, you take the same test again under similar circumstances, and you get 27th percentile on the test. These results are vastly different and could suggest that the test doesn’t demonstrate a high reliability.
However, it’s important to keep in mind that to get an accurate measure of the reliability of a test, a testing provider would gather hundreds to thousands of data points to make that judgment. One data point alone isn’t going to determine whether or not a test is valid, so if you individually get an inconsistent result, it’s not necessarily a cause for alarm.
Why would you want a test with reliability?
It goes back to the goals of administering pre-employment tests in the first place. The greatest value of pre-employment tests is their ability to measure innate qualities that are often not apparent to us through the traditional resume and interview process. These qualities are often hard-to-train soft skills like critical thinking, problem solving, conscientiousness, openness, motivation, and learning ability. The main thing these qualities have in common is that they can’t easily be trained because they remain relatively stable within a person over time.
For these types of stable qualities, you’d expect a pre-employment test to be able to measure them in a consistent way over time. If the test cannot produce consistent results, you may question how successful the test is at measuring what it is supposed to be measuring, or its validity in general.
The types of tests that measure soft skills or innate qualities can be contrasted with tests that measure acquired skills, or abilities that are learned over time. For example, let’s think about a high school chemistry test. At the start of class, you could be given a test about the periodic table and receive 20% as your score. Then your teacher goes over the material for an hour, after which you’re given the exact same test and receive 90%. These results are obviously inconsistent and lack “reliability,” but that’s a good thing. The test in this example is evaluating you on learned knowledge, so improvement is expected. It’s important to acknowledge when it’s important that a test provides reliable results, and when it’s not.
Reliability and validity
Reliability does not imply validity. A test can be reliable by achieving consistent results but not necessarily meet the other standards for validity. However, an unreliable test limits the ability for a test to be valid. In other words, a test needs to be reliable in order to be valid.
The biggest takeaway when evaluating testing providers is to look for assessments that are both reliable and valid, because ultimately you want to feel confident that the results are helping you make smarter, data-driven hiring decisions.
