Every Byte Counts

While it is painfully obvious that computing power has grown exponentially since the invention of the integrated circuit, what many do not realize is that this appears to be nearing an end [see MIT Technology Review].

Even without the death of Moore's law, the observation that computing power doubles every two years, we must be far more conscious of resource utilization at the most basic and fundamental layers. Take the average personal computer in 1998: A nimble 333 MHz Pentium II with 64 MB of RAM and a boat load of storage in its 16 GB drive. This speed demon ran its Windows 95 operating system using only 50 MB of space taken up on the hard drive and just 8 MB of RAM (even in Windows 98, which seemed to include every driver known to man, it took up just 500 MB of space). Now, when you think about Windows 95 or Windows 98, it arguably had the same features and functionality as your present day mobile phone or Windows 10 PC. Word processing, spreadsheets, calculator, internet browser, email reader, clock and calendar, etc. So outside of a few more bells and whistles, and a superior graphical experience, why is the size of a single mobile app now pushing 2 GB? It could very easily be the bill for all of the accrued technical debt.

Taken from Wikipedia: technical debt (also known as design debt or code debt) is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer. It is important to note that technical debt isn't always accrued from poor design and implementation, unfortunately it can also be due to incompetence or laziness. Regardless of the cause, the effects are pretty obvious. By not properly designing and implementing code, you tax entire infrastructures in an exponential fashion.

Let's take a very simple function, one that adds 1 to a number passed in for our customer Bob (the guy who requested this new feature) - we'll code this in C#:

public int AddOneForBob(int addTo)
{
     int newVal = addTo + 1;
     return newVal;
}

Taking this function as it sits is not technical debt (unless you consider that it is pointless to begin with, but let's not digress). Now, if a few other customers requested this same feature, and I copied and pasted it for say Mary and Tom, it becomes technical debt.

public int AddOneForBob(int bobsNumber)
{
     int newVal = addTo + 1;
     return newVal;
}

public int AddOneForMary(int marysNumber)
{
     int newVal = addTo + 1;
     return newVal;
}

public int AddOneForTom(int tomsNumber)
{
     int newVal = addTo + 1;
     return newVal;
}

This is certainly the easiest and most cost effective way to implement this feature for Mary and Tom right now, but it accrues huge technical debt. We can fix this by taking the few additional minutes to implement proper design:

public class MathLibrary
{
     public int AddOne(int addTo)
     {
          int newVal = addTo + 1;
          return newVal;
     }
}

Simple right? What about data itself, say beyond the argument over JSON vs. XML, and into the payload? For an example, let's say that we have an API that provides customer data, and our API queries and returns every detail about a customer (we'll call this the customer object). If our client calls this customer API, they get every detail, every time - even when they only need the phone number for example. The full customer data looks like this:

{
  "id": 8,
  "name": "Troy Aikman",
  "phone": "214-555-1234",
  "email": "troy@dallascowboys.com",
  "state": "Texas",
  "favorites": [
     "touchdowns",
     "pizza",
     "efficiency"
  ]
}  

But what if the client could customize their responses, and get only the data they need back?

{
  "id": 8,
  "phone": "214-555-1234"
}  

The full customer data: 149 bytes, custom data: 31 bytes. Let this type of customer data get requested 10,000 times per day, and you've used 1.49 MB for full customer data and only 310 KB for custom data. When you're talking bandwidth utilization, that's a savings of almost 80% per day.

When these sorts of problems are looked at individually, the impact is minimal, arguably non-existent. However when you get to the details, these extra bytes need CPU cycles to process, RAM to temporarily store it, disk space to support it, and network utilization to transfer it. Wasting time worrying about a few more KB here and there seems silly considering the massive power of computing at our fingertips. It seems silly until you realize these things compound even with compression, and ask Warren Buffett about compounding if you don't believe me.