Classification by decision tree

Decision Tree is a supervised learning method used in data mining for classification and regression methods. The decision tree creates classification or regression models as a tree structure. It is a tree that helps us in decision-making purposes. It separates a data set into smaller subsets, and at the same time, the decision tree is steadily developed. The final tree is a tree with the decision nodes and leaf nodes.

A Decision tree is a flowchart like tree structure, where each internal node denotes a test on an attribute, each branch represents an outcome of the test, and each leaf node (terminal node) holds a class label. Decision trees can deal with both categorical and numerical data.

Algorithm

A machine researcher named J. Ross Quinlan in 1980 developed a decision tree algorithm known as ID3 (Iterative Dichotomiser). Later, he presented C4.5, which was the successor of ID3. ID3 and C4.5 adopt a greedy approach.ID3 uses Entropy and Information Gain to construct a decision tree. In this algorithm, there is no backtracking; the trees are constructed in a top-down recursive divide-and-conquer manner.

In ZeroR model there is no predictor, in OneR model we try to find the single best predictor, naive Bayesian includes all predictors using Bayes' rule and the independence assumptions between predictors but 'Decision tree' includes all predictors with the dependence assumptions between predictors.

Types of Decisions

There are two main types of decision trees that are based on the target variable, i.e., 

1. Categorical variable decision tree
2. Continuous variable decision tree

Construction of Decision Tree : 

A tree can be “learned” by splitting the source set into subsets based on an attribute value test. This process is repeated on each derived subset in a recursive manner called recursive partitioning. The recursion is completed when the subset at a node all has the same value of the target variable, or when splitting no longer adds value to the predictions.

Decision Tree Representation : 

Decision trees classify instances by sorting them down the tree from the root to some leaf node, which provides the classification of the instance. An instance is classified by starting at the root node of the tree, testing the attribute specified by this node, then moving down the tree branch corresponding to the value of the attribute as shown in the above figure. This process is then repeated for the subtree rooted at the new node.

The decision tree in above figure classifies a particular morning according to whether it is suitable for playing tennis and returning the classification associated with the particular leaf.(in this case Yes or No). 

For example, the instance 

(Outlook = Rain, Temperature = Hot, Humidity = High, Wind = Strong )

would be sorted down the leftmost branch of this decision tree and would therefore be classified as a negative instance. In other words we can say that decision tree represent a disjunction of conjunctions of constraints on the attribute values of instances.

(Outlook = Sunny ^ Humidity = Normal) v (Outlook = Overcast) v (Outlook = Rain ^ Wind = Weak) 

Applications of Decision Trees

1. Assessing prospective growth opportunities
2. Using demographic data to find prospective clients
3. Serving as a support tool in several fields

Advantages of decision tree methods are:

• It can perform classification without requiring much computation.
• It does not require any domain knowledge, easy to comprehend.
• It can handle both continuous and categorical variables.
• The learning and classification steps of a decision tree are simple and fast.
• Provides clear indication of which fields are most important for prediction or classification.

Disadvantages of decision tree methods:

• Working with continuous attributes (Estimation tasks)
• Avoiding overfitting
• Super Attributes (attributes with many unique values)
• Working with missing values

Summary

Decision trees are used for handling non-linear data sets effectively. The decision tree tool is used in real life in many areas, such as engineering, civil planning, law, and business.