Across nearly every industry, Artificial Intelligence is driving new innovations that are poised to change the very nature of work. Here at Criteria, we agree that AI is a promising area of exploration for pre-employment assessments and the hiring process at large. But AI is not a magic spell that can do everything for us with the flick of a wand. Last year, Amazon experienced this firsthand when the company built a hiring algorithm that ended up discriminating against women, a sign that even the biggest, most technologically savvy companies haven’t quite mastered AI. While AI is still in its emergent phase, we all need to be vigilant about how it is designed, executed, and validated to make sure that the AI is producing the results that we actually intend.
Last week, the Washington Post published an article entitled, “A Face-scanning algorithm increasingly decides whether you deserve a job.” The article casts a lot of doubt on the AI-driven recruiting tools marketed by HireVue, a prominent video interviewing company. Among the skeptics quoted are leading AI researchers, including one who describes HireVue’s offering as “pseudoscience.” They make a convincing case, and in so doing the article raises interesting questions about what kind of “assessments” are fair for candidates.
In the last few years, HireVue has expanded beyond its roots as a video interviewing platform and tried to rebrand itself as an assessment company. This transition makes sense if one considers that video interviewing is an increasingly commoditized field where players are competing against free platforms like Skype or even multi-functional virtual meeting platforms like Zoom or GoToMeeting.
HireVue’s rebrand strategy is driven by two elements. First, HireVue acquired a game-based assessment company called MindX whose assessments measured constructs long known to predict performance, like cognitive ability. Fair enough. But the biggest part of HireVue’s marketing message, as the Washington Post article points out, involves rebranding the video interviews themselves as a type of AI-fueled assessment that analyzes hundreds of thousands of data points, including tone of voice, facial expressions, and other cues, to predict job performance.
Describing these unproven technologies as “assessments” is deeply problematic.
There are very real ways in which AI is enhancing pre-employment assessments, but HireVue’s marketing claims are unsupported by the types of validating evidence that buyers of pre-employment tests should insist on. Quite frankly, the “science” does not pass the smell test. In particular, HireVue’s black box algorithms are troubling for two main reasons:
1. Professionally developed assessments rely on decades of research in social science as well as leading edge data science. This is important because social science research shows us WHY these assessments work in a way that is clear and transparent for employers.
Any organizational psychologist would agree that research shows, for example, that cognitive ability is consistently a good predictor of work performance. In the world of behavioral assessments, similarly, a well-researched personality trait called conscientiousness tends to be predictive of job success pretty consistently. There are in fact thousands of independent, peer-reviewed studies that confirm that cognitive ability and conscientiousness predict performance across many different types of jobs. This is not surprising, as most employers would agree that hiring smart and hard-working employees should lead to good hiring outcomes. These constructs are proven and validated as the CAUSE of good performance, not just a correlate to it. But black box algorithms like HireVue’s that are not grounded in social science research are at their best only chasing correlations. There is no body of peer reviewed evidence on how facial expressions or use of certain words in an interview predict performance, and HireVue has certainly not shared any. These companies are just looking for correlations without context or theory to back it up—and I wouldn’t hold my breath waiting for them to publish data showing the efficacy of their algorithms.
2. For ethical reasons and for the sake of candidate experience, we believe assessments should be based on a model of informed consent.
This is what black box algorithm-driven hiring is usually missing. Candidates who take a Criteria Corp assessment KNOW they are being evaluated, and generally have a pretty good idea of why and what we are trying to evaluate (intelligence, work habits, etc.). AI-driven systems that scrape social media profiles, or even examine facial expressions in video interviews, are to varying degrees evaluating without consent, or at least without INFORMED consent, meaning that a HireVue applicant may know they are being evaluated but really have no idea how or what is being measured. The candidate impressions in the WaPo article are funny. Should I use big words? Should I force a smile? Should I be enthusiastic?
HR people may find benefit in conducting video interviews but should be very wary of using anything marketed as an assessment that isn’t grounded in constructs that have been proven through transparent, peer-reviewed research. At Criteria, we’ve administered over 15 million assessments, and we’ve continually validated their predictive validity by tying the assessments to job outcomes at companies from across all industries. Ultimately, vendors who describe their services as assessments should be able to share details of how AND why their assessments work, and “trust us, it’s AI” is an answer that should only lead to more questions.
