Crosschq Data Labs: A Deeper Dive Into Candidates’ References

Doing reference checks on potential new hires is something 80% of employers have in common, but this is where the similarities end - because it seems like no two employers can agree on the best method. 

Each company can have their own method for performing reference checks, and may conduct them at a different phase of the recruitment process. They may ask different sorts of questions, and zero in on different types of reference providers over others. 

With all of this variance, is it any wonder that the candidate data coming in from reference checking is considered some of the most suspect?

That isn’t stopping employers from acting on information gleaned from reference checks. According to a Robert Half survey of 2,800 senior managers in the U.S., respondents reported removing 34% of job candidates from consideration for a position after a reference check. 

At the Crosschq Data Labs, we’re continually working to spot data patterns in reference checking and correlate them to what actually happens after the hire. We’ve also been observing some pretty noteworthy insights about what can make a reference skew up and down. 

Understanding how the power dynamics in reference checking work can help you evaluate your reports and accurately compare one candidate to another, increase the accuracy of your reference checking, and help you make better data-driven hiring decisions.

[See The Complete Guide to HR Management]

Who Is Giving the Reference?

According to Harvard Business Review, both a candidate’s former managers and colleagues should be contacted as references, because the data gathered from each type of interview can be significantly different. 

Managers typically emphasize task-related behaviors, such as ability to work independently and consistency in meeting deadlines. They may tout a candidate’s commitment and loyalty to a company or their work ethic.

In contrast, an employee's peers provide a better picture of their interpersonal behaviors. They can give you an idea of whether the candidate is easy or fun to work with, and whether or not they have traits like friendliness, compassion, or an ability to listen.

Teamwork is more important every year, especially with the advent of remote work and a distributed labor force. Obtaining references from both sets of former contacts can be critical to obtaining a holistic view of your candidate and creating an effective, proactive reference checking process.

Don’t forget the data you get can be skewed by more than the direct power dynamic between the reference giver and the person submitted for the check. There are a whole host of gender and ethnic dynamics that can also come into play, as our own research has revealed.

Here are some great insights we’ve found in our Crosschq Data Labs…

Race and ethnicity in relation to reference scores 

For example, Caucasians are the gatekeepers of the reference checking process, especially at the managerial level and above, due to inequity and disparity in hiring. Biases inherent in these dynamics can cause better references to be given by a caucasian manager or coworker to a caucasian candidate than for a candidate of color. 

This holds out across most ethnicities, with the exception of Asians (regardless of specific background.) Asians ask other Asians to be their reference when possible, but they are also harder on each other when it comes to providing references. Caucasians come in second in regard to giving the hardest references.

Fig. 1: % of reference providers by ethnicity by candidate ethnicity

Fig. 2: Avg. reference scores (out of 5) by reference and candidate ethnicity

Gender in relation to reference scoring

The gender gap is similar. Female candidates typically ask other women to be their reference, and men make their own requests to other men. Females give lower scores to male candidates when providing a reference, and men give lower scores to women.  

Fig. 3: % of reference providers by gender by candidate gender

Fig. 2: Avg. reference scores (out of 5) by reference gender and candidate ethnicity

Age in relation to reference scoring

The older the person providing the reference is, the lower the scores they give in reference. It’s unclear why this is so. Perhaps it’s part and parcel of a perception on the part of older generations that younger generations are “entitled” or “lazy.” 

Fig. 5: Avg. reference scores (out of 5) by age group of reference providers

Power structure in relation to reference scoring

As noted, managers give different types of answers regarding candidate strengths and weaknesses than peers do. Mentors are also more likely to be stingy with the scores they hand out, while those who previously worked under a candidate are likely to rate them higher. Vendors, customers, and clients typically provide the highest scores of all. 

Fig. 6: Avg. reference scores (out of 5) by relationship of reference providers to candidates

Comparing Reference Scores 

Candidates tend to select people they think will give them the most positive reference, meaning most employers go into the process of performing checks assuming the data is going to be a little skewed. 

However, by using a solution like Crosschq 360 it asks survey questions written by IO-Psychologists, so you can reduce unconscious bias, as well as, learn more about a potential employee’s true strengths and how they are likely to perform. Done right, reference checks can significantly reduce recruitment costs and improve employee retention through enhanced quality of hire.

Once you have scores in hand, it’s worth reviewing the above trends in reference scoring and looking at correlations between who provided each reference, their demographic data, and their former relationship to the candidate. 

For example, if you have a candidate who is Asian and looks good on paper, but his scores are slightly lower than another candidate, be sure it’s not just because all of his references were also Asian, who have shown to score references lower.

The same goes for a female candidate whose references are all male. A woman who doesn’t have any other female references may suffer from lower scores because the data shows men tend to score women lower than their male counterparts when all else is equal. 

Taking these data trends into account and using them to improve accuracy when it comes to your candidate reference checks can naturally improve diversity in hiring. By adjusting for the various effects known to exist, you can balance candidate scores against other criteria being used to make a decision and hopefully hire the best candidate for the job. 

At the end of the day, a glowing reference shouldn't be your only criterion for hiring, just as a bad reference shouldn’t automatically disqualify a candidate.

Stay tuned to see what other great insights we discover in our Crosschq Data Labs! In the meantime, schedule time with one of our team experts to learn more about Crosschq 360 and see how it can help your team hire more effectively and efficiently.