4 Problems with Software Interviews

Earlier this week I posted a tweet about my frustrations with software interviews. After seeing so many of the stories that were shared with me, I want to summarize my beef with software interviews.

I do think that interviews should be challenging and companies should have a high standard for the candidates they offer roles at their company. Recruiting and hiring is very hard, and it's even harder when you have 100s or 1000s of candidates in the pool. I'm very proud of how my current team and organization interview candidates, we have very high standards but do our best to set clear expectations.

This post is not about making hiring decisions, even though it kind of comes with the territory. I've had many great interviews that I was proud of that didn't result in an offer. That's a story for another time.

Problem #1: Candidates are evaluated on material they weren't able to prepare for, material that doesn't match the role

This is probably the #1 problem that I come across.

Typically when talking to a recruiter or hiring manager the first thing you'll hear about technical interviews is: "data structures and algorithms". The range for this is very broad. Even if you read all of Cracking the Coding Interview there's a risk you'll be asked questions beyond what you've prepared for.

As Joshua calls out, the pool of coding questions you're prepared to answer is much smaller than the possible questions you might be asked. Does this luck really translate into better data when evaluating a candidate? I don't really think so.

As William has pointed out, it's really frustrating to come in with a certain skillset and knowing something about what the role entails... and being evaluated on something completely different. In my opinion, the format of the interviews should not be a surprise to candidates.  

It's fair to ask really challenging questions that shows how a candidate works through a new problem. If you're a hiring manager, you can reduce interview churn by giving candidates a chance to prepare for these interviews.

If you find that complex problem solving and using computer science or math domain knowledge helps give you better insight into candidates, just make clear that that's the case. If I was interviewing for a web UI role (as an experienced web UI engineer) and I was asked questions about iOS development or combinatorics, I'd be pretty upset and confused if nothing told me to prepare for that.  

Recommend: Set realistic expectations with candidates & standardize question pool with interviewers

If your team has a pool of interview questions that you pick from, let candidates know some of the things that will/won't be expected for them to know. Things like "data structures and algorithms" doesn't really narrow down what a candidate should brush up on.

Picking a few areas of interest that are relevant to the team gives candidates a chance to brush up on topics and show up better during interviews. When I interview, I try to minimize the number of "prior-knowledge" questions I ask that can't be worked out during the interview.

When I do use prior-knowledge questions, it's because I think it will help a candidate shine. If someone says they know a lot about React, I'll ask really specific questions about it and it always blows me away. However, I wouldn't ask those questions to someone who says they're vaguely familiar with it.

Problem #2: Candidates are put through intimidation trials or 'gauntlet' challenges

I've been through plenty of technical interviews, and 99% of them have been 1:1 with an engineer, designer, agile coach, or engineering leader. During my internship interviews with Goldman Sachs I had 2 separate interviews with 2 engineers interviewing me. It wasn't awkward and even maybe felt a little more social and comfortable.

In high school, I had an interview where a team of 5 engineers sat on one side of the table and interviewed me from the other. It was a bunch of questions and was all over the board. The interview went well, but I was definitely in a defensive posture the whole time.

When interviews are described as intentionally making you uncomfortable, that's a red flag for me. There's anxiety that comes with the territory, and I really do not believe intentionally stressful interviews demonstrates "how you work under pressure".  

Here are some other stories:

Most of these are pretty cringeworthy on their own. I always go back to the question: does this interview give me any signal about how this candidate will perform at this role?

I'm not against pop-quiz style questions to start out an interview; I use them myself a lot. However, I explicitly state that my questions cover a range of topics with the intention of probing for depth and breadth on technical subjects. It's not a pass/fail evaluation.

Recommend: Review your interviewing practices & optimize for getting useful signals from candidates

• If your team values the traditional "work under pressure", "think on your feet", etc. values; ask yourself whether or not intentionally making the interviews more stressful is the right way to gain that signal.
• If you're an engineering leader, do you know how folks in your organization are interviewing candidates? Could changes be made to get more useful data from these interviews?
• In my experience, getting the anxious energy out of a candidate in the first few minutes usually pays off for the rest of the conversation. They're much more willing to share relevant knowledge and experience when they're comfortable.
• Would your hiring loops produce different results if the interviewing methods were changed? Is it possible you're passing on skilled talent?

Problem #3: Some people are assumed to be non-technical or "not technical enough". Companies ghosting candidates.

I've seen this one play out a lot too. It basically comes down to recruiters and interviewers having a bias for what a "technical person" looks like. Many times women and other underrepresented folks in tech are assumed to be "non-technical".

On the surface, I don't think there's anything with finding other roles within the company when the one they're interviewing for doesn't match. However, it's important to be aware of when this makes a candidate feel like they're being typecast into a non-technical role.

There are lots of reasons to not offer someone a role, and that's normal. That's business. However, when providing feedback to candidates it's important to consider what message is being sent. Sometimes the feedback sent to candidates is incongruent with the actual hiring decision that was made.

Recommend: Check your biases. Think about how they impact your decisions and actions.

• If someone in your company makes a candidate feel stereotyped or marginalized, they will remember it forever.
• During one of my interviews, I was mistaken for the coffee guy (innocent mistake) and was berated at the door for being late (not so innocent). I'll remember that story forever, and I'll tell it forever.
• Educate yourself and your employees about the biases that show up in recruiting and hiring, re-think your interview and evaluation process to ensure the process is equitable for candidates.

Problem #4: Lack of professionalism during interviews and failure to communicate with candidates

These are pretty self explanatory.

Recommend: Do better.

These stories will permeate and be shared with friends, families, and coworkers. I have a friend who once heard his interviewer take a Snapchat video of the interview (because the audio of it started to play back before he could mute it).

These are first impressions that last a life time. Regardless whether or not a candidate is going to move forward or not, it's important to treat them with dignity throughout the process.

I know I personally have a short list of companies/individuals I'll never interview or work with based on these kinds of bad interactions.

If your interview process is the front door to your engineering organization, how do you feel about how people are greeted at your door?