What does your interview process measure?

phone screen interview

There’s a tendency to take the current interview process for granted. This process has stood as is, largely unchanged for decades. Companies generally adhere to some form of pre-screening and resume review, followed by on-site interviewing. We may feel that the interviewing questions we ask are clever and well considered, but does it produce the desired results. Several studies show the pitfalls of the current interviewing process (HBR, Forbes, NYT). This poses the question, what is the interviewing process actually measuring? And how great is the cost of a bad interviewing process?

An anecdote about retail.

To frame our thoughts, we’ll begin with an anecdote. We recently had a conversation with a retail employee – when we asked her how her day was going, she told us that tomorrow her store was being visited by company executives and she was stressed, because she and her peers needed to thoroughly clean the store prior to their arrival. We felt empathy for her, but thinking about this exercise from the perspective of the company executives struck us as odd.

Of course, the store employees should prepare the store for the visit, but from the leadership perspective, arriving at the store and evaluating its cleanliness and orderliness should be an obviously meaningless exercise. The orderliness of the store on the day of the planned visit only tells a story about how concerned the store’s employees were for the visit, not about the normal state of affairs. A surprise visit would have likely been much more revealing. We mean no criticism for the store’s employees, but this did remind us of the process of interviewing.

What does a normal interview tell us?

What does a candidate’s ability to recite past accomplishments and bits of role-related knowledge on the day of an interview tell us? To answer with a question – how does a prudent candidate prepare for an interview? In my experience, a prepared candidate has tailored their resume to match the context of the job listing; broken apart the job listing’s requirements and keywords into a list of things to study; created a list of possible questions with they may be asked for this type of role and written answers; and recited these answers in front of friends and the mirror.

In anticipation for the interview, they’ve tried to temporarily mold themselves into the most attractive candidate possible. This is not unlike the store employees that have tried to temporarily mold their store into the model store. We all know this process of preparation, because we ourselves have likely engaged in it before. But, what does this tell us about the long term success of the candidate? And will we as interviewers be evaluating the candidates answers for their merit alone? Or will we be swayed by their looks, their charm or any other arbitrary attribute?

Critical thinking interview questions.

So how might we overcome this problem? The strategy some companies employ is to sprinkle in questions and exercises which require critical thinking and some skill. This isn’t a bad start, but how do we measure such exercises? A company might have prepared a rubric which describes what criteria warrants a good or bad score. This works well if we’re measuring something like basic math ability. But, in other cases, we’ll be subjectively evaluating their answer and the steps the candidate took in arriving at that answer.

Subjectivity is a problem, because it must be evaluated by the interviewer. Do interviewers score the same first thing in the morning compared to when they are hungry right before lunch? Or at the end of the day when they are tired and ready to leave? Can we possibly have the interviewer always interview at the same time and under the same conditions? If there are multiple interviewers, can we ensure each interviewer is calibrated to produce similar scores for similar responses? If we cannot, what should we do?

Correcting for issues.

Maybe we average out the interviewer scores to try to root out biases. But, this sort of consensus approach may well average out the brilliance of a candidate’s answer. What if the candidate had a clever insight and approached the problem in a way unfamiliar to the interviewers? Maybe some of the interviewers saw the cleverness of their approach and appropriately scored them highly. But, others did not and lowered the candidate’s score. This would be particularly unfortunate, as we should all strive to hire employees even smarter than we are.

But, all of this was under an assumption that our critical thinking problems were even useful in the first place. Are all good candidates good at quickly responding to tricky questions? Does the inability to articulate a strong answer mean they are not good at their craft? Will the candidate perform differently in the unusual and uncomfortable interviewing environment? If they perform poorly, does that mean they could not produce great results in their normal work environment? Does does the interview itself really qualify a candidate to undertake a months long project on behalf of the company? The likely answer to all of these questions is a resounding “no.”

So what can we do?

To a degree, our criticism of the current interview process is not entirely fair, because we do not have all of the answers. The interview process most companies employ today is the “least worst” option. Ideally, we’d want to watch a candidate perform their job for an extended period of time and evaluate them, but this is logistically infeasible for both the candidate and the company. Alternatively, we’d benefit from reviewing samples of their work, but this is likely the confidential property of their current or previous employer and unless their current job is also their personal hobby, they may not be able to provide this to us.

Automated employment testing can help.

So what can we do? We cannot yet solve all of the interviewing problems, but some can be tackled. Automated employment testing can be a helpful way to eliminate some of the issues with interviewing. Using automated employment testing, we can standardize our questions to ensure all candidates are being challenged in the same fashion. Human interviewers can be removed from this step of the process to eliminate biases that cause different interviewers to score differently or the same interviewer to score differently based on their current disposition. Candidates can be tested as early as possible in the process to eliminate most and allow more time to probe the remaining candidates thoroughly. And candidates can be given the freedom to answer some of our questions in an environment of their choosing so that they might not underperform due to the stress of the unnatural interviewing environment.

All of us should take seriously the pitfalls of our current interviewing processes, because hiring the wrong candidate can be one of the most costly and painful decisions a company makes. Re-evaluating our hiring approach can be one of the most productive efforts a company ever undertakes.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to Top