The floating point "bug"

Building on my previous article, instead of simply checking homework by using a computer to calculate answers, both the question and the answer have a variable. Let's just dive into the question and its solution, and then make some observations.

The worst verification we can attempt, in general, is using either 0 or 1. If my answer was simply "4x", then 0 would satisfy both the question and answer, and our answer seems to be correct. If we then use 1, both the question and the answer (4x) resolve to "4", which means that 4x is seemingly the correct response. However, once we replace x with 2, our answer is obviously wrong. 1 is usually a bad test because of the reflexive property. We need to use a better selection of numbers for the verification process.

The first issue we must tackle is how to represent the question in a computer language. Let's try python:

import math

math.pow(64 * math.pow(x, 12), 1/3)

Just about every computer language has a library which can take a square root, but not a cubed root, as required in this case. Mathematically, a cubed root is the same as raising something to the 1/3 power (not -3 power, which we'll see later).

We can then use different values for x, and that's as simple as creating a list of numbers:

 [1, 5, 20, -3]

We end up with the following code and results:

That CANNOT possibly be correct. Where did we go wrong? The answer lies in the fact that there are strong-typed languages such as C and Java, and loosely-typed languages, such as JavaScript and python. With python, we even go as far as calling it "duck-typed", where if something looks like a duck, walks like a duck, and quacks like a duck, then it IS a duck. Notice the 1/3: python sees "1" as an integer and "3" as an integer; therefore the result will be an integer. The integer (truncated) value of 1/3 is 0, which means that no matter what x is, we are raising the entire expression to the 0 power, and that is always 1. How do we tell python that the answer should be a floating point result? Make one of the numbers look like it is a floating point number. We can either make 1 into 1.0, or 3 into 3.0, or both. Now our results are quite different:

Now that we know how to represent the question, let's see if our answer is correct:

It works! How about a language where we don't have to remember to explicitly to cast integers to floats? Will Java work?

public class MathCheck
{
    public static void main(String[] args)
    {
        int[] nums = {1, 5, 10, -3};
        for(int num: nums) {
            System.out.print(num);
            System.out.print("\t");
            System.out.print(Math.pow(64 * Math.pow(num, 12), 1/3));
            System.out.print("\t");
            System.out.println(4 * Math.pow(num, 4));
        }
    }
}

If we run the code, we get:

We seem to be experiencing the same issue as with python, but there's a subtle difference. In python, a variable can be of any type; it can hold an integer, a floating point value, a string, a list, etc. In the Java code above, we did not specify variables, and simply used "1" and "3", so at compile time, it inferred the types. There are two ways we can tell Java to use floating point: make either the 1 or 3 a 1.0 or 3.0 respectively, or explicitly cast either of them. Let's try them both:

System.out.print(Math.pow(64 * Math.pow(num, 12), 1.0/3));
System.out.print(Math.pow(64 * Math.pow(num, 12), (float) 1/3));

Now when we run this, we get:

That's even more confusing than before. The answers look fine, but both representations of the questions have fractional parts that shouldn't be there, and neither representation agrees with the other. The reality is that Java doesn't infer 1.0 to be a float; it infers it to be a double. If we replace "float" with "double" in the above code, our output is now:

At least they match now, but what is with all this fractional nonsense?

The floating point "bug"

Computers are funny with numbers.

In 1993, we had some in-house accounting software written in Paradox 3.5, a then-popular database system. One user caught a bug when the system should have yielded 15.15 when adding 12.03 and 3.12. Instead, it was more like 15.1499999999. We then performed the identical operation in FoxPro 2.5, and it did not exhibit the same behavior; it returned the correct result. We called Borland (back in the day when you didn't need a support contract) and their response was "Computers are funny with numbers." We then told them that FoxPro did not have that problem, and they had no response. Back then, Intel processors did not natively support floating point operations. One would have to get a "math coprocessor" to perform true floating point calculations. This is no longer the case, and you will not encounter the problem we did back then.

The problem is still related since it has to do with representing factional values. How does the computer represent 1/3? Our decimal system is based on exponents of 10:

Computers are based on binary (base 2). That means that the number system gets represented as follows:

How would 1/3 be represented in binary? I can't be 0.1, because that's equivalent to 0.5 in our decimal system. How about 0.01? That's 0.25 in our decimal system. That means it must be at least 0.01 binary. What about the next binary digit, one-eight? That's 0.125 decimal, but if we add that to 0.25, we get 0.375, which is more than 1/3. That means that there are no eights, but that leaves us at 0.25, so we have to consider sixteenths. 0.0001 binary is 0.0625 decimal, and if we add that to 0.1 binary, we get 0.0101 binary, or 0.3125 decimal. If we iterate through the 20 subsequent decimal positions, we end up with 0.01010101010101010101 binary, which is 0.333333015442 decimal. As you can see, binary is not well-suited to represent some decimal numbers.

Third time's a charm?

How about if we try JavaScript?

Note the floating point issue, but it's the only language where we didn't have to explicitly state that 1/3 should be a fractional value.

I would say that your best bet is to use python, since it does not exhibit the floating point issue seen in other languages. This really has to do with how languages implement floating point arithmetic. If we were to go as low-level as possible without using architecture-dependent machine language, we would use C, which surprisingly gives the following result:

I would not recommend using C since some systems may not have a compiler installed. The above code was compiled on Intel x86 hardware. It would interesting to see what happens if I attempt this on ARM-based hardware, like my Raspberry Pi 3. I'll leave that exercise to the reader.