I read Reddit regularly, as apparently half the world does. One of the subreddits I follow is the ExperiencedDevs subreddit because sometimes I learn something or find an interesting article there.

Today I saw this post and it struck a chord:

Are interviews REALLY providing better talent as they get harder, longer, and more complex?
by u/Groove-Theory in ExperiencedDevs

In my experience the answer is NO. Not just no – but NO with capital letters because it is that serious.

Software Interviewing is Broken

I’ve done hundreds of interviews in my career and I’ve hired dozens. I can remember 2 bad hires that should have been caught in the interview process.. One of those was someone who we gave a job as he was desperate. The other one who was a young recent college grad who found a better job and ghosted us. I accept blame for the first but not the second.

I’ve never used a coding test as part of my hiring process.

Ideally, I’d like to see some past work and code. Typically a technical screen will be 2-3 people asking questions. That’s pretty much all that’s required. We’re talking 2-3 hours tops and you know or don’t know if the person is a good hire. That’s all it takes.

Yet, somehow the software industry has gone the way of high schools. For some reason we now have more and more steps to get through and more and more tests. As if tests somehow prove someone is a developer. Some interview processes are a 4-6 step process over weeks or months. I’ve done interviews where they expect you to complete a 8-20 hour long assignment and turn it in.

These tests are a huge waste of time. They keep candidates from applying. (Who has time to waste on such nonsense?). Plus, these tests weed out a ton of great candidates.

Tests Suck

Let’s talk about tests for a second, because really, that is what so much of software interviewing has become. Almost every interview now consists of some kind of coding test and sometimes even multiple coding tests. Sometimes you’ll run into personality tests, IQ tests or a CCAT test. There is a whole cottage industry now that provides tests for companies looking to hire and another whole industry providing practice problems and tests for software interview tests. LeetCode.com is one of the best sites for practicing interview problems.

That’s all well and good except tests SUCK.

Tests are artificial, contrived and often poorly constructed. Passing or failing a potential hire on a 1 to X hour long software test is stupid. Even if its longer or a whole day test, it is still pretty much useless.

Why?

In your test, how much are you testing someone’s skill with a particular framework, tool or library vs their ability to learn, collaborate and produce great code? What if your hire has been doing Ruby for 10 years and your test is using some new Gem that he’s never used or he used once 6 years ago? So you’re going to pass on someone who spent half the 1 hour test learning about the framework? Unfortunately that is exactly what most companies do.

Lets Think for a Second

Students in school study for the test and not for the material. For examples you don’t have to look very far. We have the SAT, ACT or <insert your state> standardized tests in just about every state of the country. Students spend a ton of time practicing and preparing for the test itself. In some states, the last month or more of a school year is spent ONLY preparing for standardized tests. During this time students are not actually learning, they are just getting better at the test.

Now in the software industry, we’re doing the exact same stupid thing.

Hiring companies are requiring these waste of time coding tests and failing the vast majority of their candidates. So, in response candidates now look online before they apply or after they hear they will need to take a test. They will find out exactly what coding test you use and prepare. (For almost any large tech company, the entire hiring process is available online, including what tests they use.)

Even if your a small company and your interview process is not public knowledge – once you tell the candidate what test you use they will find the answers online. It is not hard. So, they look great on your test and pass. Except – really, they “cheated” because they knew what was coming, prepared and did well. Thus the test provided nothing and you let through a candidate who can pass a test where they already know the answers.

Best case as the hiring company, your candidate didn’t already know about the test so its a “surprise” and they haven’t already seen it. So they have to take it live and complete it in the time given. (As the hiring company you will never really know this by the way.)

So where do we go from here?

Let’s say the candidate passes. Great, they move on to the next step. If they fail, you boot them.

What if though, the person who failed was brilliant but didn’t know your specific library? Or they had an urgent phone call during the timed test they had to take? Possibly the test tool had an issue? Or maybe the candidate was nervous and didn’t do well? Could it be you’re looking at something like structure and your structure expectation is different from what they did. I could think of a thousand reasons you might not like their test result.

In all of those cases, you’d never know how good the candidate was because you just booted them based on a test result. You could have booted a great developer who had a bad day or a bad few hours. Or maybe your test sucks and you need to re-look at it. You will never know – and that is the whole point: the test proves nothing.

Let’s look at the different cases:

Do Tests Work?

Checkout this table. Along the top we have columns showing whether or not the candidate knew the test ahead of time, and along the side we have rows for whether or not the candidate passes:

	Candidate DID know the Test Ahead of Time	Candidate DID NOT know the Test Ahead of Time
Candidate Passes	– So, they cheated? Or do we call that prepared? – Or they are just really good at the test? – Or are they actually a good dev and the test proves it?	– Great, so I guess they know the tech we tested on? – Or they found the answers online as they did it… – Or are they actually a good dev and the test proves it?
Candidate Fails	– Did they have a bad day and do poorly? – Did they study an old test version and miss some small change? – Do they not know the libraries you were using or testing? – Or maybe your test sucks? – Or maybe they are a bad developer?	– Did they have a bad day and do poorly? – Do they not know the libraries you were using or testing? – Did the test software crash or break? – Or maybe your test sucks? – Or maybe they are a bad developer?

In each case, we really don’t know if the developer is good or bad. We just know they passed or failed a test. But we don’t know why. This is a pretty bad way to figure out if someone should be hired or not. Yet this is all too common.

So What is Better?

Every company, team and project is unique. So, there isn’t a perfect one size fits all approach. But, I can tell you what has worked very well for me over the years. The following steps lay out the process in detail:

Hiring Step 1: Job Post Instructions

Obviously you will have a job posted up online in various locations. When writing your job description, include at least 1 or preferably 2 specific instructions for candidates. For example:

1. Send your resume to <x> with the subject line <y>
2. Please include a brief paragraph describing your most recent use of <z> technology

These are simple instructions that anyone who reads your job post will follow. Right?

So what’s the point then?

Most applicants don’t really read the job post in detail. They see a headline like “JavaScript Developer” for a job post, think “Hey I know JavaScript” and apply. People like this with no attention to detail will get weeded out right away. This will get rid of at least 50% of your candidates.

Hiring Step 2: Quality Screening

The next step in the process of hiring is quality screening candidates.

Hopefully you now have a nice collection of resumes from people who made it through the initial screening we did in Step 1 “ie. Can you follow instructions”. Now, we need to do some screening of candidates. For this stage I do a quick review of their resume, application and cover letter.

My goal in this stage is to quickly remove non-qualified and unlikely to succeed candidates. I am looking for obvious problems such as:

• Obvious typos or other boneheaded errors in the resume
• Obvious lack of knowledge or experience in required technologies
• Large or unexplained gaps in employment
• Changing jobs too frequently
• Deception

The last bullet point above – Deception – I think warrants some additional comments. A lot of folks exaggerate a bit on their resume. That’s not what we’re going for here, though that can be a problem. You won’t necessarily know if they are exaggerating at this point just from reviewing a resume.

Deception is not always easy to spot. But there are some telltale things I look for:

• Abbreviated Names: Yes, really, this is a problem. A LOT of less scrupulous head hunters will submit fake resumes under names like A. Jones. The game they play is that they hope you like the resume, reply back and then they can pitch you on their great placement services that provide cheap talent for a small fee of $20000. Yes, this really happens. How do you avoid this gotcha in your resume? Put your full first and last name on there.
• Dates Don’t Add Up: I have seen a lot of resumes over the years claiming way more years of experience than someone has. If you are 2 years out of school, chances are you don’t have 8 years of professional Java experience, for example.
• Resume does not match Linked In: When I see resumes from promising candidates I look at their LinkedIn and other social media accounts. When their resume is quite a bit different than the info they post on LinkedIn for jobs and experience, that is a big red flag.

Those are the most common deceptions I find in resumes. Anything I consider deceptive gets tossed. If the resume still looks promising it gets added to a list of folks we would try to phone screen.

Hiring Step 3: Initial Phone Screen

What I do here is probably pretty typical for this part of the hiring process.

The goal of the initial phone screen is to check for 3 things: communication skills, personality fit, basic technical competence. This phone call is usually 15-20 minutes and is very casual. Typically during this call I will introduce myself, talk about the open position and then ask the candidate to talk about themselves, the experience they have and why they are a great fit for the role.

If the candidate is impressive, I will advance them to the next level. If the candidate is only good, then we hold them but do not advance them further. Every other candidate is discarded.

Hiring Step 4: Technical Interview

The next step is to invite the most promising candidates to a full technical interview. I like doing these in person, but remote works as well. For these I like to set aside at least an hour, but 2 is better. I also like to have a committee of 3 people on the hiring side on the call. The ideal mix would be the candidate’s manager should they be hired, and 2 technical resources.

In this technical interview, we ask pointed questions about the candidates technical history.

What we’re most focused on is understanding what the candidate has actually done. This is an important distinction. Just because candidates were on a team that did something does not mean they actually contributed in a meaningful way.

To help understand a candidate’s actual work I ask them what they are working on. Some will say “I am part of team X that is building Y”. OK, that’s fine but I want the candidate to tell me what exactly you are doing on the team. I will drill down deep into the details as far as they can go. The more they can tell me about what they are working on the better. Specific things I like to hear about include:

• What the purpose of the software you are building?
• What is the system architecture?
• What tools are you using?
• What are the pros and cons of the system?
• What specific components have you worked on and what did you build?

I also like to ask some more general questions:

• What is the biggest mistake you have made? How did you handle it?
• If you were to build a new system today, what would you use and why?
• What technologies do you love / hate and why?
• What was the last technology you learned? Why did you learn it.

For these general questions, I’m looking to learn about the candidates experience and thought process. If they can explain their choices well, that is a great sign and what we’re looking for. If they can’t that is a big red flag.

At the conclusion of the interview, we hold a small conference with the 3 interviewers. A unanimous yes will result in us making an offer to the candidate. Anything less is a no and we reject the candidate.

Conclusion

Hiring technical staff is hard. But there are ways to find great people.

We need to remember that great people is what we’re looking for. We’re not looking for just a skill – we’re looking for people we will work with, train, grow and hopefully people who will buy into the vision of your company.

I love building new things, and coming up with new ideas. The ability to create new products and solutions has always been great fun. One of the things I wanted to do since early in my career was to have a patent in my name.

At my first professional software job at Kodak back in 1997, I remember closely working with a lot of really brilliant engineers. One engineer in our team who’s name I sadly cannot remember had over 100 patents issued. His entire job was to come up with radical new ideas and see if he could make them work. He had no other responsibilities which at that time was quite rare. At the time I thought it was pretty incredible that was his only job. Since then, I always wanted to be a patented inventor.

Well, today I can say that I am officially a patented inventor!

So What is the Invention?

During my work at Pleiotek, I was a co-inventor on a series of ideas that resulted in Patent 10786395: “Apparatus for Processing Healthcare Data and Storing and Transmitting Large Amounts of Data via a Bandage or Sticker”. Yeah that doesn’t roll of the tongue I know.

In short, this patent captures the idea behind storing patient medical data on the patient itself. Imagine a soldier injured in a battle, or someone hurt in a mass protest. In these urgent and tense situations, timely patient care is critical. But in these types of scenarios, the patient will need to be made safe, and ultimately moved to higher echelons of care.

We must retain records of patient injuries and care. Incorrect or incomplete patient records can cause mistakes. For example, a patient in pain may be administered a dose of morphine. Morphine can be fatal in high doses. So it is critical that providers know if a patient has already received the medication.

The current state of the art for tracking patient data in these situations is weak. Patient data is written on very small cards, or on the patient via sharpie. Yes really. In some cases mobile devices are used to track data, but it is rare in the theatre of war or in mass casualty situations.

The system that we invented helps care givers both track medical data and provides a way for the data to easily travel with the patient. We invented various techniques for embedding data storage in bandages or very small patient worn devices. This allows care givers to record the patient’s medical information directly on the smart bandage or device on the patient. Now patient data follows the patient wherever they go. This ensures their medical record is complete and accurate. It also ensures care providers have all the information they need to provide the best care.

The idea and technology is great. Although I am no longer part of Pleiotek, I am hopeful that they can bring this technology to market. This idea can be a great help to our military as they treat soldiers who sustain non-combat or battle injuries. It will also help our local emergency healthcare teams as they treat large events with lots of patients and hand offs between providers.