Finally, I got it! “Unpivot” Python, without using “pandas.melt()”

Ok, first of all, to be very honest, I am new to Python, having my experience very recently this summer... learning by doing... So, let's start from the very beginning: why do we care about “Unpivot”? Is it really important and necessary for Data Scientist or Business Analyst? Do we need it to perform well on a daily basis? The answer is YES! Because the data we collect or download is often organized in a “report” format. Below is just an example of demographics data (Figure 1) from CBS (Het Centraal Bureau voor de Statistiek). Before starting the analysis, it’s important to clean the data and sort the data in its original way, for example, to “unpivot” the columns “Unmarried”, “Married”, “Widowed” and “Divorced” in to a new column (Figure 2).

Figure 1: Original .csv file from CBS

Figure 2: Target dataset after “Unpivot”

So, here we go. How to make it? In #PowerBI, there is a direct “UNPIVOT” function under Transform when editing query (Figure3). It is very much similar function named “UNPIVOT” within #Excel. For #Python, if you are a fan of google like me, you would quickly find the pandas.melt() function here (https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.melt.html). And it works!!!

Figure 3: Screenshot from PowerBI “Unpivot” function

Now, what I want to share next is how to use other basic Python codes to unpivot, without using advanced function “pandas.melt()”, see below my codes (Figure 4).

Figure 4: Python code for “Unpivot” without using “pandas.melt()”

Right, you might wonder why I did this? Seriously, it took me almost 2 hours to intensively google and tryout on #Jupyter till I get the unpivot table… To be honest, it was indeed a very struggling and frustrating process. As a Python starter, I am so happy and grateful to see many advanced and ease-in-use functions already available out there for application. However, for learning point, I don’t understand the backside of those advanced functions, e.g. how they work in principles and how the machine processes it, line by line. By doing this basic programming, I have a better understanding now.

It’s a great learning process. I'm very sure there are many great tips and amazing ways using Python out there. So, if you have any tips and advice on the above related subjects, please don't hesitate to reach out and share. Your insights, experience and knowledge are highly appreciated and very much welcomed always 😊

#Learningbydoing #lifelonglearning #Python #DataScience #Analytics #Unpivot #PowerBI #Excel