Why Do We Ask Coding Questions?

By Olympia Interviewer Simon Carroll

Eng Manager at Etsy, Former EM at Alphabet (Wing and Google) 

Why do we ask coding questions at Olympia? It goes beyond figuring out if someone can write Python, sort a list, or find nodes in a graph. Software engineering requires a wide range of skills, but an interviewer spends less than an hour with a candidate. Simply put, a coding question is the most efficient way to get signal about all of a candidate’s abilities to perform in a software engineering job. 

There are several things an interviewer needs to determine about a candidate in the space of their interview. You need to find out about a candidate’s thought process, communication and their ability to explain their thinking. You also need to determine if the candidate can solve the problem, and whether or not they can implement that solution in code. We’ll cover each of these in detail below. 

Thought Process 

The first area of assessment is their thought process. What is their approach to solving the question? The first thing to notice here is the candidate’s initial goal. Some candidates will hear the first few words of the problem, pattern match it to something they’ve done already, and start working on the solution they already know. That strategy doesn’t lead to the best outcomes. Many software problems (both in interviews and on the job) appear to be similar to things we’ve seen before, but upon closer inspection have interesting quirks or twists that make our previous solutions either incorrect, or inefficient. A successful candidate is likely to spend time understanding the problem first. This means understanding both the question being asked, and the desired output. Successful candidates will wonder how the inputs to the problem are being received, in what format the output should be, and what the optimization goal is (computational efficiency, memory efficiency, speed, correctness, etc). 

Once the candidate has understood the question, notice their approach to gathering more information. Are they trying to find the boundaries of the solution space? Are they looking for hidden efficiencies? Are they looking for any crucial information that was left out of the problem description? Are they asking any questions at all? This stage is very important. Before a candidate starts solving the problem, they should have a good sense of the scope of the problem, the types of inputs they’re likely to see, and if there’s anything intrinsic to the problem that will make a solution easier. 

Finally, it’s helpful to see if and how the candidate keeps track of what they’ve learned about the problem. In an interview, a candidate can succeed without doing this (after all, we have less than an hour together), but in a software engineering job, keeping track of what you’ve learned helps you avoid round trips with other team members to cover information you’ve already learned. If the candidate takes notes during the interview this is easy. However, some candidates are able to keep the information from the interview entirely in their head, and shouldn’t be penalized for this.

Communication 

The next area of assessment is communication. Obviously the candidate will need to be able to communicate to successfully pass the interview. How else will they convince you that they’ve solved the problem? But there are several levels where a candidate can shine in their communication skills that are not directly related to sharing a successful solution. 

The first comes during the discovery phase, while the candidate is trying to understand and scope the problem. Are they asking questions that get them the information they need? There’s a difference between just asking questions, and asking specific questions that uncover a fundamental truth. This is a skill that can be learned, and the best software engineers have mastered this essential art. 

The second is when they have an idea of how they will solve the problem, and they’re traveling down the path to that solution. Are they bringing you along for the ride? Different candidates prefer to do this in different ways. Some will explain while they’re coding, telling you what they’re doing and what they’re not doing. Others will prefer to explain first, then write the code second, only speaking up when something unexpected happens. Still a third way of doing this is for a candidate to begin by writing the code, then walk you through it once they’re done. A primary point of attention is how openly they’re sharing their thinking. Openly communicating to help the interviewer understand the solution shows an investment in teamwork, which is another crucial skill for a successful software engineer. A candidate can prove their code works by simply running the code, but proactively explaining to the interviewer why their code works, and where its limitations are, demonstrates an investment in the interviewer’s understanding. 

This ability to explain their thinking is what separates an average engineer from a great one. An engineer who can write the code to solve the problem has value. The problems you give them become solved problems, as long as that person is around. But an engineer who can not only solve the problem, but also explain their solution to their colleagues has more value. The problems you give them become solved problems not only while they remain there, but for as long as the team exists, because they’ve distributed their understanding and solution to everyone else on the team. 

Ability to Explain Their Thinking 

A specific communication skill, the ability to explain their thinking merits its own discussion. 

The first trait to consider here is initiative. Does the candidate recognize that what’s clear to them may not be clear to someone else, and proactively try to close that gap? Having that recognition solves a lot of problems on a team because it removes room for confusion. Knowing that two people can understand the same thing differently, and proactively taking steps to build a shared understanding reduces the possibility for misalignment because of different understanding. 

The second is clarity. Everyone communicates differently, so clarity is the ability of the candidate to build in the mind of the interviewer the same picture that exists in their mind. This will happen

differently with each candidate and interviewer. Is the candidate able to work with the interviewer to find a common place of understanding, and work forward from there? Being able to do so demonstrates a solid ability to successfully share their ideas with others. 

Finally, we have completeness. Does the candidate’s explanation give sufficient detail appropriate to the context. An interviewer will probably not care why a candidate named their variables this or that, but will care about which sorting algorithm the candidate selected and why. Do they pick an appropriate level of detail for their explanation? And if they get probing questions, do they answer them and keep going, or do they change their approach. Being able to give a complete picture saves time, and helps in ensuring understanding. 

Can they solve the problem? 

Up to this point, we’ve considered a lot of factors around how the candidate approaches the problem, and how they take that journey with the interviewer, but we haven’t asked the foundational question. Can they solve the problem? While successful candidates don’t always solve the problem, and unsuccessful candidates don’t always fail to solve the problem, this is still an important piece of information to note. 

There is usually a set of factors required for a candidate to be able to solve the problem. The first few we’ve covered above. Do they have a productive thought process for approaching the problem? Are they communicating to ensure they understand the problem, and verifying that understanding with the interviewer. Next, do they have sufficient background knowledge of data structures, algorithms, and common software patterns to be able to identify when to apply them to a novel problem? Most coding questions center around a specific algorithm or data structure, or have a few options that will work acceptably. 

A successful candidate is likely to recognize that inherent aspect of the problem, and call it out as the correct, or optimal way forward. They should also be able to explain why it’s the correct choice, and consider how the choice might be different if the circumstances of the problem were to change. Upon recognizing that, the candidate will know how to solve the problem, and be able to explain that solution to the interviewer. This is an important milestone to a working solution. 

Can they write code to implement their solution? 

Finally, we ask the candidate to implement their solution in the programming language of their choice. With the problem already solved, and the technology being the choice of the candidate, this step is a final verification. It shows they feel comfortable enough in some structured language, and they can use the syntax of that language to express their solution and test it. Writing code is a specific skill that an engineer will be doing every day, so this step is to verify that they still have this skill. This is also where we can tell if the candidate can readily spot common errors in code, or if they need to be pointed out. This isn’t usually a big deal. These are readily caught by compilers or unit tests, but a candidate who identifies them before those steps is more obviously comfortable in their chosen environment.

Wrap up 

Asking a coding question successfully gives you a chance to learn all of the above about a candidate and more, and to offer an informed opinion on their suitability for a software engineering position. You’ve assessed how they think, how they communicate and explain their thinking, determined whether or not they could solve the problem, and reviewed the code they wrote to produce a solution. With this combination of factors, you should feel comfortable giving your recommendation on this candidate.