The ‘Oh Shit’ test is weak too

originally published here.


Yesterday evening when I opened my Mattermark Daily, I was drawn to a piece on talent that referenced my old home airport entitled “Kissing Goodbye to the O’Hare Test” by the founder of ZumperAnthemos Georgiades.

In essence, he pushes for a move away from valuing pedigree in hiring and instead toward valuing “resilience,” or the folks that have proven themselves loyal and dedicated — “people who will be by your side on the worst day imaginable.”

He formatted these lines to stand alone, and I’ll include them here since they’re the ones I’d like to speak to:

“You should hire for resilience. Over and above everything else. Resilience.”

I agree that academic credentials and performance in a structured and rule-based environment didn’t prepare me for the emotional challenges of working on startups and now on my own company.

Criticism of my performance when it was just a job or a paper was much easier to rationalize than the constant and varied critiques that starting a company invites (if you thought your personal appearance was safe, it’s not — or at least mine wasn’t).

So while my anecdata points to Anthemos’ correctness, something was nagging at me about the piece. As I turned it over with my brother at Zeitgeist last night, I realized it’s the fact that resilience is difficult to define. Things that are difficult to define / “you know it when you see it” are a problem, because they make us more susceptible to hiring bias.

I attended a recent (not yet publicly available, will update with link when it is) Google Ventures event on Unconscious Bias that specifically called out this type of hiring methodology as risky. Afterward, I skeptically examined my own (measurable!) bias with Project Implicit tests and reviewed some of the academic literature around Implicit Bias, and now I’m confident in saying:

Don’t hire based on personality characteristics you think you see in candidates.

Instead [even though you are busy, even though they came via a fantastic referral, even though it completely makes sense on paper to choose an ‘action-oriented person’ over a ‘person that seems like a rule-follower’]:don’t trust your gut when it comes to hiring — trust defined, objective hiring criteria.

Eric Luis Uhlmann and Geoffrey Cohen’s paper “Constructed Criteria”documents exactly why you shouldn’t look at people ad hoc: you probably are moving the goalposts, but you do it subtly and structure a logical case around it, so it doesn’t feel like bias to you.

Even worse, there’s evidence that if you’re primed about being unbiased, you may make even more biased decisions — see “I think it, therefore it’s true…” another of their (relatively short!) papers.

So, I don’t want to take down the embrace of resilience, grit, whatever the term of the moment is, but how would someone that wanted to structure a minimally-biased hiring process do it?

(Why would you want to hire better and more diverse? ROE/outcomes for investors, or so your company has higher collective intelligence, which improves performance on unique challenges and is different from the average iqs of the group members)

Uhlmann and Cohen suggest the answer is structure. This does mean time investment.

What are you looking for on a resume? Define it like product requirements — clear, concrete. Show it to other people.

What will make someone successful in the position you’re hiring? Define it in a way that is measurable, objective and real. Show it to other people.

What kinds of interview questions tie directly to the job description or job criteria? Define them. Show them to other people.

What makes for great answers? You can see where I’m going.

My feeling, in short, is that there’s no reason to use any quick and dirty tests when it comes to hiring — they are likely subjective. Despite the tax on speed in creating a structured process, I have a hard time imagining I’ll ever be too busy to hire intentionally for a small team.

Whom I work with and for (customers) is, after all, why I’m passionate and loyal in the first place.

I hope this was worth your time! If not, let me know: @kathl2en_ on the blue birdy app

Kathleen MeilComment