Understanding our latest Quality of Hire release: How do Performance Ratings Perform?

Recently, Crosschq launched the third iteration of our Quality of Hire metric. The update expanded the scope of Quality of Hire to include non-performance post-hire metrics such as employee engagement and culture-fit and now allows clients to customize the relative importance of these metrics. More detail on these changes can be found here. With our latest version of Quality of Hire now live, Crosschq Data Labs is now taking a closer look at how employee performance is measured and included in the Quality of Hire calculation.

[Related: Unlocking the Mystery Around Quality of Hire]

First, let’s start with a broad question: what are we actually measuring in an employee performance review? You might be inclined to say “performance” – and this is not wrong, but it’s not the whole story. Research by Sculle et al. (2000) suggests that the correlation between an employee’s performance and their performance rating (as completed by their manager) is at best, about 25%. Note the distinct language here: performance vs. performance rating. The study shows that a manager’s performance rating contains more information about that manager and the manager-employee relationship than it does about the employee’s actual performance. Put another way, manager-rated performance is highly influenced by idiosyncratic and interpersonal factors, making it only an approximation of capital P performance.

So with that, can we do better? While expanding the scope of Quality of Hire we’ve explored including additional measures of performance, those that are more objective and more closely related to an employee’s actual performance. Examples include sales numbers, views of a marketing post, GitHub pull requests, Slack activity, etc. These metrics are all objective, reflecting none other than the employee's actions – however, their usage involves a noteworthy tradeoff. Crosschq Data Labs describes these kinds of metrics as “gamable”, meaning that strict reliance on them as performance metrics is likely to change the behavior of employees in undesirable ways.

Thinking about the sales team example, we can imagine a situation in which a sales rep makes unrealistic promises to a prospective customer, leading them to sign and then quickly churn. In this scenario, the employee will have performed well despite the business outcome being suboptimal. Alternatively, you can imagine a development team that uses submitted lines of code as a measure of performance. As you might imagine, there will be a lot more code, but not necessarily faster feature delivery or bug fixes. Both situations exhibit a misalignment of incentives, a kind of principal-agent problem as described in economics, whereby the leadership team and individual contributors have different priorities. 

This leads us to Crosschq Data Labs’ first impossibility theorem: measures of workplace performance cannot be both objective and non-gamable. Or rather, there is a tradeoff between how objective a measure of performance is, and how likely it is to change behavior in undesired ways.

[Related: Can Pre-Hire Assessments Predict Post-Hire Performance?]

Alright, but enough musing on economic theory, what does this actually mean for my organization? Well first, subjectivity might not be as big of a problem as I initially made it out to be. Consider the job of a personal assistant, their performance is intrinsically tied to their relationship with their boss – in which case an objective measure cannot exist. In such situations where objectivity is impossible we shift our focus towards a collection of subjective reviews, that together act as a kind of pseudo-objectivity.

For example, Crosschq’s 360 report uses reviews from a candidate’s peers, supervisors, and mentors in tandem to cut through the idiosyncrasies of an individual review. Second, the style and methodology of a performance review can have a big impact on its objectivity.

Crosschq’s Pulse module, for example, was modeled after an NPS score and fine-tuned to capture as much information as possible. While not objective, tools like A/B testing have enabled us to increase the utility of the survey.

At Crosschq, we believe that no single performance measure works in all contexts. Every measure has its strengths and weaknesses and consists of some balance of objectivity and gamability. Relying too heavily on any metric that is not properly aligned with your organization’s business goals is likely to miss the mark. In practice, it’s often the case that a combination of objective and subjective measures best describes an employee’s performance in such a way that aligns with business goals.

Ultimately, performance is just one dimension of a “quality hire” – in practice things like culture fit, engagement, and promotability might be equally important to a new hire’s long-term success. Crosschq’s Quality of Hire metric was born out of this idea; designed specifically to take a holistic view of “quality” while reflecting your organization’s unique business objectives. In our experience, every hiring process can be improved.

[Related: See 5 Reasons Why Quality of Hire is So Important]

Quality of Hire serves as a north star, guiding which improvements will yield the most impact given their effort to implement. For instance, we frequently work with clients to determine the optimal number of interviews for a given position. To see how Crosschq can help improve your Quality of Hire so you can recruit, retain, and develop your talent better, talk to a team expert for a 1:1 free consultation.