You have a testing problem – you probably just don’t know it.
As noted earlier [here , here and here ], the use of psychological and personality tests to make hiring and promotion decisions is on the uptick. With the recent US Supreme Court Ricci vs. Stefano decision, any number of ill-advised HR types will likely increase the use of these tests. With over 30 years of background in the employee selection and assessment business, I’ll come clean on having a bias against those the use of those tests for hiring and promotion.
You should too.
Your next job, or your next promotion may depend on how well you test – and the test may have little to nothing to do with how people like you will perform on the job. And it’s not only “jobs” – I’ll be surprised if the testing doesn’t extend to contractors and / or consultants: if you believe in something (drug testing comes to mind), might as well have anyone who touches the enterprise be tested.
So why do firms test?
Because testing provides the allure of a “magic bullet.” Something that will tell a company if they should hire or pass on a candidate. And that seduction turns minds to mush when you push the point.
As background, all sorts of people get involved with designing and developing selection tests. Some may be qualified – my hunch is that many are not.
I remember when I headed up US Staffing at Barclays Global Investors and talked to the “psychologist” who then led Brainbench’s (acquired in 2006 by PreVisor) test development program. He was a “psychologist” – which in his case it meant be had a master’s in psychology from the University of Alaska-Fairbanks.
Not to impugn the University of Alaska-Fairbanks, but it’s not known as the epicenter of the personality and assessment testing movement. Brainbench also listed McKesson IT as a client – and I happen to be a McKesson alum – yet no one at the McKesson IT head office had heard of them. Not to sound like a curmudgeon, but this sort of fast and loose characterization of qualifications and references by a testing firm like Brainbench is the sort of thing that I’ve found crops up all over the place.
The Brainbench ad reads like many others: “Having insight into a candidate’s attitudes, interests, and motivations can help you ensure the right fit with the culture of your organization as well as the requirements of the specific job. Our personality tests differ from others on the market because they are designed specifically for employee selection.”
Here are the things you should look for when someone trots out a test to inform promotion and selection decisions. And if you don’t see these factors when using an instrument then you’ve got to wonder why anyone is using the test.
1. Is there a demonstrated direct relationship between scoring well on the test and performing well in the role? In other words, does a high score on the test mean high performance in the role, and does lower test performance mean lower or unsatisfactory performance?
2. Has the test been validated? And by validation, I mean several things: have high performers currently in the role taken the test and scored well? Have lower or unsatisfactory performers taken the test and have their scores been lower. And while I’ll defer to a wonk like Nate Silver to have a point of view regarding data validity, the set of “N” that you likely need is in the 100’s or 1,000’s, not the five or tens: that means that to get decent data validity you must have tested a large set of people and tracked those who scored well to see if they performed well on the job. Unless there’s a strong correlation you don’t have much data predictability.
3. Does the testing company stand behind their test? Will they assume partial or total liability if there are lawsuits? In other words, do they guarantee their work? If they don’t, you should wonder why not.
Psychological and personality testing CAN be helpful in better understanding how people are wired, inferring interests and predispositions, and in a number of other areas. And there are certain knowledge tests that are interesting and can tell you in a broad sort of the level of technical expertise someone possessed. But like most things, any test should be taken with a grain of salt: the people who score highest on tests are frequently not the people who perform best in a role. Tests are suggestive, inferring things rather than predicting things.
No test – unless a job is limited, static, and controlled – measures all the elements at play in someone’s work. The dirty little secret from companies who run candidates through batteries of tests is that not everyone who “passes” does well on the job: many candidates still bounce and don’t pan out.
And unless the test actually predicts, why use them?
New Rules is an occasional set of writings focused on changes in norms, culture, or ways of navigating work, organization and careers. More about executive and team coaching services can be found at the “About J. Mike Smith and Back West, Inc.” sidebar or the “Hire Me” tab above.