Judgment versus algorithms in hiring decisions

Companies now have massive amounts of data about employee performance.  Is it possible to find meaningful correlations between performance and indicators that can be measured before a job offer is made?  If so, then there is room for algorithms based on these correlations to make improved hiring decisions.  

Economists at Harvard, Toronto and Yale business schools recently did a study that examined this question.  They examined the hiring of low-level service workers at 15 firms.  They focused on what happens when an employee is hired based on the algorithm versus what happens when a hiring manager overrules the algorithm and hires based on his or her own judgment.  

The results, summarized in Bloomberg Businessweek and available in National Bureau of Economic Research Working Paper No. 21709, are sobering.  Job candidates picked by the algorithm stay longer and perform better than those picked by hiring managers.  Also, there was a strong correlation between algorithm predictions and actual performance.  

One caveat: this study looked at relatively unskilled jobs where performance could be measured objectively.  What would happen in more complex jobs such as trial lawyers or university professors?   My guess is that we will find out in the not too distant future.  

Perils in big data

Worthwhile FT piece by economist Tim Harford about the big data phenomenon.  Make no mistake -- the availability of terrabytes of data on all things imaginable will eventually lead to exciting breakthroughs.  Harford's concern goes back to the old question of correlation versus causation.  Just because two variables appear together at a relatively high frequency, it does not mean that one causes the other.  And even if there was causation, it is hard to tell which way causality runs.

Another challenge is whether the correlation is spurious because it is based on biased data.  Harford notes this is a particular challenge for data sources such as Twitter, whose users do not represent the overall population.  Money quote:

“Big data” has arrived, but big insights have not. The challenge now is to solve new problems and gain new answers – without making the same old statistical mistakes on a grander scale than ever.

MBA students at NC State are getting training on a wide range of tools for analyzing big data, but they also are getting the theoretical insight they will need to make sense of the results they obtain.  

More on big data and hiring

Today's NYT business section has a lengthy article about how some firms are using big data to predict performance of job applicants.   Much of the piece centers on Gild, a new startup that has developed algorithms to review programmer code and expertise from a wide range of open source websites.  Gild identified a programmer with no college as the best available person in the LA area, interviewed him and hired him a $115k/year job.  So far he seems to be performing well on the technical aspects of his position. 

Two obvious caveats mentioned in the article: (1) this might work great for jobs where the skills necessary to be productive can be easily identified, but not so well for highly complex jobs and (2) the algorithm focuses on computer programming, not interpersonal skills. 

Big data and human resource management

Two recent pieces on how analytics and big data sets are being used to make more informed decisions about employees:

(1) Today's NYT cites how companies are starting to take a more scientific approach to measuring and evaluating human resource decisions.  Not only are big data dogs IBM, Oracle and SAP in the chase; eHarmony also is looking at how to make better employee-employer matches.  Initial applications appear to be focusing on high-turnover service jobs in places like fast-food restaurants and call centers.  The story is full of anecdotes; the most insightful one for me was Google's decision to stop worrying about GPAs and SAT scores in deciding who to hire.  I wonder what they use now?

(2) A recent piece in the Economist (link courtesy of Freakonomics) also discussed how big data capabilities are being used in HR.  Two of the examples came from Evolv, a San Francisco company that has become recognized as a leader in work place analytics.  One client learned that the choice of browswer used by a job applicant was correlated with higher job performance and less turnover (those who used preinstalled browsers like IE and Safari do not do as well as those who installed their own browser, such as Firefox or Chrome).  Another learned that there was no real difference in performance between those with criminal records and those without. 

Big data's biggest user -- finance

At least that's what WP says today.  Traders track tweets for signs of optimism or lack thereof to anticipate where the stock market is moving.  Analysts now gauge sales performance from Amazon comments; why wait for the monthly or quarterly sales figures from the government or corporate reports?

The article makes another important point that anyone who has tried to do economic or financial forecasting has known for some time: your analysis is only as good as the data and big data does not necessarily mean better data.  The stock market always has been subject to herd behavior, but at least the herd had to buy or sell to move the market before.  Now hedge funds are making the same bets in an attempt to guess which way the herd might move.  Investor beware. 

NC State MBA and Master of Accounting students are starting to get training in how to harness big data, but we are focusing on marketing, technology commercialization and risk management.  These are areas where data-driven decision making has a history of allowing managers to make more informed judgments about how to create value. 

Big data, little talent

That's the abridged headline this online WSJ piece.  Big data seems to be on its way to becoming the next big thing in management.  Companies are swimming in data and do not have the expertise to either manage the data sets themselves or to draw logical inferences from it.

What the industry needs is a new type of person: the data scientist.  According to Pat Gelsinger, president and chief operating officer of EMC Corp., the giant U.S. data company, this isn't an unprecedented problem. "IBM started a generation of Cobol programmers," he said, referring to one of the first dominant programming languages. "Thirty years ago we didn't have computer-science departments; now every quality school on the planet has a CS department. Now nobody has a data-science department; in 30 years every school on the planet will have one."

Hilary Mason, chief scientist for the URL shortening service bit.ly, says a data scientist must have three key skills. "They can take a data set and model it mathematically and understand the math required to build those models; they can actually do that, which means they have the engineering skills…and finally they are someone who can find insights and tell stories from their data. That means asking the right questions, and that is usually the hardest piece."

NC State is already staking out a leadership position in big data.  The MS in Analytics has been a huge success.  The Center for Innovation Management Studies in the Poole College of Management is doing big data research in conjunction with NC State's computer science department.  Stay tuned.