Here's an excellent video from Tony Hsiesh, the founder and CEO of Zappos about the biggest mistakes they made when building Zappos into a well-known consumer brand.
It goes without saying that without an employer having 100% confidence in the assessment results, the assessment is not worth much. Employers have repeatedly told us that test security is of paramount importance to them.
Typically, assessments measure the work personality characteristics and the cognitive abilities of a job applicant.
It's generally understood and accepted that work personality assessments pose little security risk because there is generally no right or wrong answer for each specific assessment question - most of the questions are in a format where the job applicant has to answer how strongly he or she agrees/disagrees with the question statement.
However, each cognitive abilities assessment test question has a correct answer. Therefore, the following problems are associated with administering a cognitive assessment remotely without a proctor:
1. An employer could ask a job applicant to take a cognitive assessments from an assessment vendor. The job applicant completes the assessment. Later, a different employer asks the same job applicant to take the cognitive assessment from the same vendor. The job applicant has already taken the assessment once, and if the time interval between taking the assessments is not long, the job applicant may actually remember the questions from his or her previous assessment taking experience, compromising the test results.
2. A job applicant may ask a friend to take the assessment on his or her behalf
3. A job applicant may copy and paste the test questions (copywrighted) on to the Internet anonymously, breaking the law and compromising the assessment system.
If you are planning to use a cognitive assessment, you need to ensure that your assessment vendor can satisfactority answer the above questions so that you feel comfortable trusting the results of the assessment.
When I speak with employers that are interested in using pre-hire assessments, once they understand the value of using assessments, the first question I get is how long it would take them to implement the assessment system. This criterion seems to be their #1 concern, as it should be.
To answer this question, we need to first understand that there are two types of assessments in the market: Custom-built Assessments and Off-the-shelf Assessments.
A custom-built assessment is built for a specific job at the employer using the employer's own benchmarks on what constitutes success in the job at the employer. These assessments are expensive & time-consuming. Therefore, employers typically only build custom assessments for high-volume hiring positions and/or critical positions.
An Off-the-shelf assessment is an assessment that is available off-the-shelf from a vendor for a specific type of job. The assessment comes with a standard set of benchmarks, not particular to any employer, on what consitutes success in that type of job. These assessments are cheap & fast to deploy. Typically, such assessments have generally been available only for sales and customer service types of jobs.
RightHire can offer both custom-built and Off-the-shelf assessments, but our primary differentiation comes from our ability to offer off-the-shelf assessments for not only sales and customer service types of jobs, but for almost every type of job for which you hire - we support 900+ types of jobs in our off-the-shelf assessment product.
As I am speaking with potential customers, I am realizing that few employers understand how to go about selecting a pre-hire assessment product and a vendor that supplies the product.
Here are my quick thoughts on criteria you need to use when making your decision:
Recently, I have been speaking with quite a few employers regarding their usage of pre-hire assessments. Through these conversations, I have realized that the pain felt by HR, the hiring manager, and the CFO/CEO are different.
HR's primary pain point solved by assessments is recruiting efficiency. For example, if an employer is getting 100's of applicants for a job opening, HR does not have the time and resources to go through this stack of applicants, many of who do not even meet the minimum qualifications. Applicant Tracking Systems (ATS) can help with reducing the number of applicants by asking pre-screening questions, but nonethelsess the number of applicants that meet the minimum qualification requirements may be high. In such a case, asking all the aplicants that meet the minimum requirements to take, for example, a work personality assessment tailored to that job, can help reduce the number of applicants that move to the next stage.
A hiring manager's pain point solved by assessments is different. Suppose the manager makea a hire that doesn't work out, the entire team will suffer. The rest of the team members need to carry extra load potentially working late evenings and weekends in order to acheive the team's objectives and the hiring manager needs to spend quite a bit of time managing the new hire through a performance plan.
The effect of bad hires on the bottom line is ultimately felt by the CFO/CEO. If an employer can increase the average performance per employee even by 5%, it has an enormous impact on the bottomline - assessments can help acheive this outcome.
Warning: This is a lengthy post!
One of the key decisions you need to make when embarking on leveraging a predictive analytics solution for your organization is the buy versus build decision.
In order to make the right decision, you need to understand the process by which you would develop a predictive analytics solution. This process is described below:
The predictive analytics model you develop is only as good as the predictors you use to develop the model. As discussed in the previous blog post, developing a predictive analytics solution consists of 3 components:
As can be seen from the above definition and examples, defining the criterion variable and developing the predictive model is one part of the challenge. The other part of the challenge is to understand which predictors are likely to be correlated to the criterion variable.
Step-1: Assemble the list of candidate predictors
The first step in developing predictors is to come up with guesses using your experience and prior knowledge on what would likely be related to and explain the criterion variable. For example, if the criterion variable is employee engagement, example of predictors could be the difference in employee salary versus market salary in the demographic area for the employee’s occupation, the commute time of the employee to work, and multiple variables related to the fit of employee’s ideal manager working style and the existing manager’s working style.
Step-2: Make sure you have chosen your predictor and criterion data wisely
In constructing a predictive analytics model, you need current and/or historical data on both the predictors and criterion variables. Therefore, you need to make sure that you have chosen criterion and predictors variables where you can obtain such data, and that they are relevant for the task at hand. For example, to predict the overall performance of an employee, you could ask the employee’s supervisor to rate the employee on the overall performance using a particular rating scale. Alternatively, to predict the sales performance of an employee, you could observe the sales revenue generated by the employee last year.
Similarly, to collect predictor data on the employee’s salary versus market salary, you could look at the data available in your financial systems. In order to collect data on a specific employee competency predictor such as inductive reasoning, you can assessment the employee using a test.
Step-4: Use statistical methods that allow the best use of the predictor data you have
There are mathematical methods that will allow you to make the best use of the available data. Describing these methods is beyond the scope of this blog post. Any good statistician will be able to help you use such methods to further refine your predictor set.
Taking corrective actions
Once you have built the model and understood the important predictors, can you take actions by influencing these important predictors in order to influence the criterion? For example, if your criterion variable was employee engagement and one of your important predictor variables for employee engagement was the commute time to work, if an employee has a poor score on this predictor because it takes him more than an hour to commute to work each day, do you think that providing flexible work hours so that the employee can avoid the rush hour traffic during work commute will increase the employee engagement score? It may or may not.
Correlation vs. Causation – The predictive model that we build provides a means to estimate or forecast values of the criterion variable. You should be careful not to mistake correlation for causation.
To understand the difference better, consider the following example: It was found that when the ice cream sales were up, the number of drowning deaths increased. Does this mean that the cause of the increased drowning deaths is selling more ice cream? No. Ice cream sales are correlated with drowning deaths, but do not cause drowning deaths. For example, if you create a large advertising campaign to increase ice cream sales, the number of drowning deaths may not increase. Instead, rather than ice cream sales and drowning rates being a cause of each other, a common cause between the two could be the weather. During hotter weather, more people buy ice cream and more people swim.
On the other hand, take the example of an employee working as a software developer. A predictor in this case could be the analytical thinking skills and a criterion could be the overall job performance. Given that analytical thinking is very important in order to be a software developer, it will likely be highly correlated with the overall performance score of the software developer. Let’s suppose that this employee has medium analytical thinking capabilities. Would improving his analytical thinking improve his overall performance? Maybe!
Therefore, we can conclude that the important predictor variables in our model, even though they may be highly correlated to the criterion variable, may or may not contribute to the cause of the criterion. If we use knowledge gained from serious research in choosing the predictor variables that we believe to be contributing to the cause of the criterion, we may be able to affect the criterion by affecting the important predictors.
Assuming that you believe that the important predictors of your model are causative, you may or may not want to take corrective action based on whether this employee is a high or low performer. For example, it may make business sense for you to retain a top-performer and choose not to retain a poor performer. In such a case, if this employee is a top performer, you may want to offer flexible work hours to this employee so that his commute time can be cut in half by commuting during non-rush traffic hours.
The power of predictive analytics is that they allow you to understand the drivers that are specific to an individual versus the entire company, allowing you to individualize the corrective actions that can be taken only on employees that you choose.
Updating the model
It may not be enough to run the model a single time given that the predictor data in your organization may change continuously. For example, in the case of employee engagement, as new employees join your organization and existing employees leave your organization, the predictor data set changes. In addition, the predictor data on each employee may also change over a period of time. Therefore, it is recommended that you update the model periodically or in real-time so that it reflects the currently available data.
As you can see from the above description, developing a predictive analytics model involves several steps and requires both mathematical and deep functional domain expertise to develop the predictors.
The advantages of a packaged solution are that it comes off-the-shelf with:
You could build the model yourself or using third-party consultants, but a packaged solution generally offers you a lower cost and implementation time.
Predictive Analytics for HR
Warning: This is a lengthy post!
Predictive analytics that leverage big or small data is all the rage. There have been numerous articles written in business press on this subject, but few of these articles explain the fundamentals to the person not familiar with this domain.
In this blog entry, I want to explain the fundamentals of predictive analytics, why it is important for Human Resources, and how it can be leveraged by HR.
Fundamentals of Predictive Analytics
According to Wikipedia, Predictive analytics encompasses a variety of techniques from statistics, modeling, machine learning, and data mining that analyze current and historical facts to make predictions about future, or otherwise unknown, events.
The first step in leveraging predictive analytics is to ask the right question(s) relating to a business problem you are solving. For example, one such set of questions could be "In each job type within my organization, what are the characteristics of employees that perform well and stay longer with my organization?"; "How do I hire new employees that have similar characteristics?”
Once you have asked the right set of question(s), you need to develop what are called criteria, which are essentially variables that you are trying to predict. For example, in our case, the criteria variables could be the overall job performance as rated by the employee's supervisor.
Once you develop the criteria variables, you then need to understand the input variables, called predictor variables, required to predict these criteria. For example, in our case, to predict the future performance of an employee for a particular job type in your organization, what characteristics of each job applicant applying for a job of that type in your organization are important?
In our example, for building the initial list of predictors, we could potentially look at the quality of the undergraduate school applicants attended, that overall grade (GPA) they received, their undergraduate majored, how tenure in each of the previous jobs, number and length of gaps between jobs, and other characteristics that we deem important to predicting how well applicants will perform and how long they will stay with your organization.
Once we develop the predictors and the criteria variables, we need to build a predictive analytics model using mathematical algorithms that will use the historical & current data available on the predictors from existing employees working in that job type at your organization, to predict the criteria. So, for example, in our case, we could gather the historical and current predictor and criteria data of all the employees working in a particular job type in your organization and feed this data to the predictive analytics algorithm. The algorithm will then crunch this input and output data and build a predictive model – a predictive model is nothing but understanding the correlation of each predictor variable to the criteria variable. The model will then provide a weight to each one of these predictors in such a way that it can predict the overall job performance of any future job applicants for who the predictor information is available. In a simple example with only 2 predictors, which school applicants attended, what overall grade (GPA) they received, the model may find that it needs to weigh the scores received by a job applicant, say between 0-100, such that the quality of school attended gets 60% weight and the overall grade receives 40% of the weight in order to predict the overall job performance of fresh graduates – you are essentially training the algorithm using your data for prediction.
Once the predictive model is developed, when job applicants apply to this job type, the algorithm will calculate the overall weighted average score of each job applicant on the two predictors, and based on the score, will predict the degree of future overall performance of each job applicant. Essentially, what we are concluding here is that if the predictive model is built using a large enough sample data from your existing employees working in a job type, it can reliably predict job applicant future performance for that job type. The key however is to keep the model up to date by feeding it the latest data.
Predictive analytics has been used by HR departments for years for hiring. Specifically, there is a whole field dedicated to understanding the predictors of human performance called Industrial/Organizational (I/O) Psychology. This field primarily researches the best predictors for predicting the overall performance, but also takes into consideration uniform guidelines on employee selection procedures enforced by EEOC & OFCCP, while constructing the predictive models. Leveraging the research conducted in this field, solutions called pre-hire assessments have been used by HR organizations for years to predict the future job performance of job applicants. The statistical predictive models developed in this field have however been static in nature and have to be kept up to date through annual validation studies.
The application of predictive analytics to other areas of HR is only emerging right now. To summarize, the best way to develop predictive analytics solutions to these areas of HR are again to follow the same process we outlined above:
Why Predictive Analytics is a big business opportunity
Businesses spend around 40% of their total revenues on payroll. This is the largest single expense category for a business. Businesses that optimize the payroll spending can substantially improve their profitability. In addition, although predictive analytics have been used in supply chain and marketing extensively, businesses have rarely used analytics broadly in the HR domain. Therefore, one of the biggest opportunities for a business to leverage predictive analytics is in a function where they spend the majority of their revenue - Human Resources.
In this post, I explained the fundamentals of predictive analytics and why predictive analytics is a big opportunity for HR. In future blog posts, we will not only delve deeper into how to apply predictive analytics for each specific area of HR function, but also provide you real-world case studies on the application of predictive analytics in these areas.