Land O’Spin is an occasional set of writings focused on best practices in coaching and assessment: how do take what you observe, know what it means, and draw conclusions about what outcomes will occur in the future.
Employment testing is (back) in vogue, and with the recession in place it’s likely to continue: the news, similar to an old adage about dogs, is that some employers use employment personality testing for hiring purposes because they can.
A former colleague who was asked to take a set of tests by Korn Ferry for a CEO opportunity brought the point home. She demurred, was still interviewed for the job, which was subsequently offered to someone else. Did her pass on the tests damage her candidacy, or was it simply a case of the employer trying to get additional information whenever it could?
The list of testing options all claiming to offer the ultimate hiring insight is endless: the Kolbe Index , the Birkman , the DISC Personality Test , the Enneagram to name just a few as well as numerous vendor specific diagnostic tools. Each of the tests has their own adherents, who at times seem more like believers pre-Jonestown than balanced, objective practitioners.
In a world of ambiguity and selection caution, the lure of the test score is their promise of magic bullet-like simplicity: Enneagram “5” and the candidate gets validated as scientist material – Enneagram 2 and the sales manager candidate looks misplaced for that role.
But do they work? Do they accurately predict role success? In two words, not usually. Are they frequently misused? Yes.
Malcolm Gladwell’s piece on testing titled Annals of Psychology: Personality Plus has some great thoughts on the flawed side of testing and Erica Klein has some excellent advice in ask the Head Hunter .
Employment testing is great if it predicts performance in a role. If it doesn’t, then test results are suggestive at best, and like firearms, not something that should be in the hands of kids or amateurs. And while, as my former boss Magnus Lundberg might observe, testing diagnoses what’s in somebody’s head, it misses what is in somebody’s heart. And since success is frequently a matter of both head and heart, it misses a good chunk of the story.
My experience while running US Staffing for Barclays Global Investors with one firm named Brainbench , just one of the many vendors in the space, is telling. Spurred by enthusiasm from BGI’s United Kingdom-based (where employment testing is apparently legal) former CTO, I spent time talking to the Brainbench folks about their tests.
I had been assured by the firm’s sales head that their tests had been validated (simply put: high scoring in the tests predicts strong performance in the role, and that the validity holds by gender, racial/ethnic identify, etc.) and the validation studies had been done by Brainbench’s testing head, a “psychologist.”
Terms are just words, but in assessing something those terms whenever possible should be exacting. In many states, such as Oregon where I was raised and in California where I now live, a psychologist is someone who has a PhD in psychology, has at least a two years of appropriate supervision, and meets the other criteria such as being licensed. Brainbench’s psychologist? A master’s degree of psychology from the University of Alaska. No license was held.
While I heard from Brainbench that they had indeed designed their tests, no validation studies apparently had been conducted confirming that strong performance on their tests predicted strong performance on the job. They did predict a certain amount of knowledge – but only if you assume knowledge equals performance did it predict success in a job.
A telling item: would Brainbench warranty their work and absolve Barclays of any liability if BGI was sued regarding test validity? No, came Brainbench’s answer.
Not to pick on Brainbench, but my conversation with them gives an illustration of the type of hype that’s present and through which – whether employer or candidate – you need to navigate.
When considering deploying tests, here are five questions I would ask:
1. What exactly does scoring well on the test tell you – and how do you know?
2. Who constructed the test, what’s their background and experience doing test design
3. How has the test been validated? What was the testing size, and was it cross-validated by gender, age, ethnicity, etc?
4. What other employers have used the test, how did they use it, and who are the contacts you can speak with at those companies?
5. What does the test not tell you – and is the vendor candid about its limitations.
In a little over a month, the US professional football league (known as the National Football League) will hold its annual draft, consisting primarily of eligible students coming out of US colleges. What does that have to do with testing? These atheletes are arguably the most tested people in any employment situation worldwide: films of their college football games are studied, they are run through a batteries of physical and psychological tests, they are extensively interviewed, and additionally placed in a series of scrimmage type assessments. The result? The list of expensive mistakes goes on and on .
Moral of the story: testing can be suggestively helpful in some area things but its history as a predictor is sketchy.