Beware of Snake Oil and Hiring Assessments

One question I'm often asked is what hiring assessments do I recommend for screening customer service employees. 

I'm always cautious about these, as some assessment vendors seem like the modern-day equivalent of a snake oil salesperson. "It will cure anything," they say, even though it actually won't.

It can be hard to tell the difference between a good and bad assessment. Here's my advice if you're thinking of going down this path.

Searching for job candidates who all like the same.

My "Ah-Ha" Moment

Years ago, my role included developing leaders for the company I worked for. We used two pre-hire assessments from a well-known vendor. One assessed cognitive ability and the other was a personality assessment.

Like many assessments, this one came with pre-determined "standards" that were supposed to aid our hiring decisions. Candidates whose assessment results fit within a range determined by the vendor were considered to be ideal.

The company was dutifully following those standards when I arrived. While not the only basis for a hiring decision, the assessment results were weighed heavily.

Ever the nerd, I did a study to compare our most successful leaders to the vendor's ideal profiles. The results were a mild surprise.

Many of our best leaders did not fit the ideal profile.

A senior executive did poorly on the cognitive test, though his many years of exemplary performance suggested he was pretty smart, or at least smart enough to do the job.

One of our best leaders appeared to have been hired by mistake. The results of both assessments were well outside the vendor's "ideal" range, and I wondered how she could have been hired with those results. Yet her actual performance indicated she was one of the company's top performers in nearly every category, from financial results to service quality to employee engagement.

 

The Challenge with Assessments

Hiring managers turn to assessments to help them solve two challenges:

  1. Make better hiring decisions

  2. Speed up the hiring process

Don't get me wrong, there's a place for assessments and they can sometimes work. (More on that in a moment.) There's also a major challenge.

A good assessment must be valid and reliable. Validity means it accurately assesses what you want to assess, while reliable means it does that consistently.

Looking back on my own research, the assessments my company was using were neither valid nor reliable. They had failed to correctly identify some top performers, while other top performers did fit the vendor's profile.

The big question is why?

Some assessments just aren't very good. They're based upon junk science and crackpot theories with no real evidence to back up their claims.

Other assessments have potential, but it's the vendors' suggested "ideal profiles" that are the problem. These profiles are often generic and not calibrated to your employees.

Think of it this way. Costco and The Ritz-Carlton are both known for outstanding customer service. However, it seems reasonable that the ideal employee is probably slightly different for both companies. 

 

How to Assess Your Assessments

First thing's first. Before investing in an assessment, decide which characteristics you are really looking for in a customer service employee. This will help you pinpoint what type of assessment, if any, to use.

You can use this hiring guide to help you.

If you do consider an assessment, make sure you calibrate it first. Here's how:

  1. Start by having your existing employees take the assessment.

  2. Evaluate the results for your top performers, middle performers, and bottom performers.

  3. Identify the differences (if any) between the assessment profile of each group.

This exercise will help you construct a more accurate hiring profile than the generic one provided by your vendor. 

You may also find that there's no rhyme or reason to the assessment results when you compare them to your top performers. That happened to me when I did this exercise. The results of top leaders were wildly inconsistent.

That tells you the assessment is not a valid or reliable instrument and shouldn't be used.

There's one last concern to mention here. Pre-hire assessments can sometimes put your company on shaky legal ground if they disproportionately screen out people of a particular gender or ethnicity. Make sure you consult your HR professional or employment attorney before giving any assessment the green light.