Knowing is Good, but Showing is Better

sunny

Consider the following disparate facts: In 2004, approximately 58% of traffic accidents were due to improper driving (source: InfoPlease.com); in August, approximately 36% of employees turned over (source: www.nobscot.com).

- x! E/ @' ~( I! }7 _! f5 c7 }You are probably asking yourself, "What do car accidents and employee turnover have in common?" The answer is, "More than you think!"

) e4 C: z* d( S5 w/ L( A/ MI always like to be up-front. If you believe your job consists of finding candidates and letting someone else screen or train them, save your time. Read something else. If you seriously care about hiring and promoting fully qualified managers, salespeople, and professionals, read on.
( ^. u. t( V. M! j
Suppose some very important government officials woke up this morning with a conscience (yeah, I know it's a stretch, but work with me). These enlightened civil servants unanimously decided to put you in charge of reducing traffic accidents. They gave you the power to renew current licensed drivers as well as issue new ones. They said you could use any criteria you like. Where do you start?
6 \: \- h. O; K$ w3 l+ G
First, you might want to investigate the job. Naturally, you would talk to drivers with good safety records and experienced driving instructors. After many interviews, you would learn the following competencies were critical:
2 ^: q' k+ |3 ]' h6 V
Eyesight
Physical coordination and reaction time
0 ?+ p) }. t, \7 ]' w% t3 g6 N, TBasic operator knowledge
Knowledge of driving laws
Analysis and judgment
. N  Z+ `0 S5 \$ Z! bKnowing vs. Showing

sunny

Knowing that impairment of any factor could lead to driving accidents, you decide to administer a broad range of tests:

Personal opinion. Each candidate must be recommended by another driver. This would screen out the blatantly unqualified (assuming you could trust the reference).

The personal interview. Each candidate would be interviewed about driving skills; basic operator knowledge; driving laws; and past and/or future driving decisions. This would screen-out people who did not know the right answers, but you would have no proof other than their word.
4 I% Q% _4 h: Q
Basic skill tests. Each candidate would be tested for eyesight; coordination; physical reaction time; operator knowledge; and driving laws. This would be considerably more valid and reliable. Candidates who, under controlled conditions, demonstrated physical skills and knew driving laws would be much safer than ones who didn't.

Simulations. Each candidate would "drive" a realistic simulator that presented them with a wide variety of road, weather, and traffic conditions. Again, you could be reasonably sure candidates who successfully completed the simulation would be safer drivers.
9 F# L- S3 i4 O. E* {1 g
! {9 ~; B% s- q# U$ D+ b5 IPsychological evaluations. Each candidate would be evaluated for his or her potential for road rage; use of drugs that would impair judgment; physical health; emotional stability; and so forth. Again, you could be reasonably sure candidates who successfully completed the evaluations would be safer than those who did not.
- w- X; Q' K; i5 }( ~! v% {) tNow, what do you think would happen to traffic deaths and accidents if only the people who passed this battery were permitted to drive? Right! Millions of dollars and tens of thousands of lives would be saved annually.

sunny

Bring it on Home
But what might critics claim? It's too much work. Too many candidates fail. No one cares about the poor candidates. We should give everyone a chance. Some drivers can be "coached" or trained.

Yes, those are possibilities, but what do you think would happen to traffic accidents as we loosened-up on the testing criteria? For example, what would happen if we were to license everyone and let them "sort it out" on the road? Nah! That's silly. Let's drop everything except the interview. Yeah. That should be OK.
  W' Y2 M' v' j8 }5 t
3 K% }) r# v3 `6 @Let's apply this argument to organizations, and more specifically, employment and promotion decisions. Just as driver competency correlates with traffic accidents, employee competency correlates with turnover and productivity. Whether it is a management or professional position, getting the right person into the right job requires multi-trait, multi-method testing.

This method refers to a few simple concepts: carefully determine job requirements; use several testing methods; and measure critical competencies more than once.
1 p1 A4 S' Z3 w4 d/ ^6 }& O
You don't need a rocket scientist to argue each time something is left out of that equation, employee quality (e.g., productivity and turnover) would suffer.
$ C3 V5 E  w/ a  t: ]5 w
1 J  x/ H# H. v0 U) eWhat Kind of Multi-Trait, Multi-Method Test?
) ], h+ }$ Y8 @7 I  C% x9 w' j" P6 hEverything that separates one candidate from another is a test. This includes references, interviews, photographs, application forms, resumes, sources, written questions and answers, and so forth. Put another way, unless you hire everyone who applies, you are using some kind of test.
" Y8 a" H- ]& v4 }* i
3 S! U% Z: ^# u9 O0 uNow that we all agree every recruiter and hiring manager is in the testing business, the decision becomes much simpler: how predictive is the test he or she is using?
0 a, u$ ~- B, X+ y* C! F2 |3 e5 p
Predictive accuracy is not complicated. All we want to know is whether the test measures something critically important to the job, whether scores are accurate, and whether the test is stable over time.
! G* e1 I/ u2 w$ l0 W; v
  t9 R% L3 U- e; f) z7 wConsider the personal opinion "test" in our driving example. Do personal opinions measure something critically important to the job? Are they accurate and consistent? Probably not — unless the candidate behaves like the lead character in a Steven King novel.

" Z+ m, l5 m' j( ]5 q- X/ ]$ KPersonal opinions are filled with subjectivity. Some subjectivity is good; but unless the job is very simple, personal opinions are weak predictors. Why? It's a people flaw. Humans tend to think their personal opinions are good indicators of job competencies. Although we all like to think we are excellent judges of character, research and rational analysis show otherwise.

% F: v- s% P( R6 |; }  G& |8 t  n$ ZWhat about the people who are brimming with confidence about their interview skills? Can they be wrong? As amazing as it seems, sometimes better-crafted interviews lead to better predictions, but interviews are still based on "knowing" not "showing." However, people who have never taken a college course in measurement are often as open to new ideas as a witchdoctor is to germ-theory!
8 }; k8 [0 I* \: a% t: O
Knowing and Showing
3 \$ F5 q1 U" b7 FThe more a test measures knowing instead of showing, decision-quality drops. It's a fact. It's called "test error." Every test, whether it's references, interviews, photographs, application forms, resumes, sources, or written questions and answers, contains some amount of error. Error can never be completely eliminated, but it can be reduced by following a few professional steps:
7 J: C9 O0 ?; G, Z7 W8 g
, A/ e5 s% L1 FInterview job-content experts such as trainers and job holders to clearly understand the key competencies required for the job (i.e., skip the managers).
* o. f+ N* D, J; j$ k* @  }/ I' \Measure every key competency using tests that are job-related and validated (i.e., make sure scores predict job performance).
Use more than one test type (i.e., cross-check).
3 \$ q, V+ _2 H0 O* ]  X& Q0 EClearly understand that "showing" (what you see) is better than "knowing" (what you hear).