The Risk of Doing Things by Default

We all know that there are only two hard things in Computer Science: cache invalidation, naming, and off-by-1 errors. I recently realized that besides these technical obstacles, preventing myself from doing things by default is the toughest thing in daily work.

As engineers, we usually implement solutions that allow ourselves to do things by default. We extract shared logic from different services into a single package so we can reuse them; We develop specific algorithms to solve specific problems; We design boilerplates so we can bring up a well-formed working application from 0 to 1 in no time. Things are so handy that we usually can just start working in the default way. Don’t take me wrong, these are great things, and improve our productivity so we can spend more time in bars🍸.

I mean really, who wants to spend 10 hours to scaffold an Angular.js based app instead of just hitting <CR> after a single command `yo angular my-awesome-app`?

While reproducibility is a holy heaven, philosophy tells us that there are always risks we want to take into account.

Defaults miss out outliers, sometimes mission critical ones. One biggest difference between school and industry is the way we look at the importance of special cases. In school, an edge case that you didn’t include in your Proofs might just make your homework A- instead of A; In real-world business, missing an index bound checking can lead to loss of $10000+ in revenue. This is not just kidding.

In Yelp, we have a postmortem culture where people send postmortems after accidents. The purpose of postmortems is to educate people in a blameless way. I’ve read literally every postmortem since three months ago, and among those documents, I learned that causing revenue loss because of “default thinking” is not a rare case.

For example, a production error was identified one hour after it was deployed because people were monitoring wrong graphs, which are usually used for error monitoring but somehow didn’t present a specific kind of error. Those default charts are vulnerable hence we shall be proactive and document vulnerabilities (things the defaults can’t do) and create action items to fix them. It will never be perfect because it’s Software Engineering.

Defaults are constraints and people are afraid of breaking them. This is a variation of Kellan’s 2nd Law of Software Engineering from an industry conference - “Your answers are wrong. Or they will be soon”.

Current solutions are great for status quo, not necessarily for the future. I think this is something almost everyone will agree. This is also the reason we often talk about short-term plans versus long-term ones because we want to make ourselves aware of these potential constraints.

The problem is not that defaults can be constraints, but once they become constraints, people are afraid of breaking them or just lazy to make a change. This can spawn a lot of head scratchers when you are afraid of challenging what’s already been designed and implemented. Instead, you will be consuming energy and think really hard about how to accommodate status quo.

For instance, say your JavaScript file has a list of HTML IDs that you want to attach an event listener. Everybody knows this is a bad idea, but because it’s so simple to add something to a list comparing to implementing HTML Class based solution, many people end up appending IDs to that list and give the performance nightmare indulgence.

But don’t get me wrong, please do NOT break defaults all the time. My point is that we should be untiring of identifying default constraints. It might sound less actionable, but how about pausing 10 seconds whenever you think something is good enough or something has to work?

Defaults are based on assumptions. Redux is a powerful frontend framework that allows easy and fast web app development. That’s great, but what are the assumptions? The assumptions are …

1) You can afford costs for engineers to learn the framework

2) You are building a superhero (giant) app that cannot be done with jQuery within 3 hours

3) You can afford costs for building frontend - backend separated app, which usually takes double commitments than it’s counterpart

4) …

Again, no kidding, in real business, failing to realize any of these concerns can potentially fail your project, cause catastrophe in revenue. For startups, a wrong technical stack can kill its whole business.

My team in Yelp (traffic team) does a lot of experiments to boost Yelp’s organic traffic from search engines. When I was designing my first experiment, the only thought I had was “this is just yet another experiment, I can just replay the same thing that guy did”. Most of my experiments implemented in a default way eventually took me and our team more time than others. It didn’t take me too long to realize that very experiment is unique and we should always have a thorough discussion before sitting down and typing.

Defaults are just human intuitions. As you know, intuitions can be very helpful and also can be the complete opposite. Let’s face it, there are risks. 

Being risks themselves, defaults are still important, but they can’t be strongly held. Identifying risks of default thinking in decision-making process helps us to break stereotypes and make better decisions so that we can achieve healthier business growth.