What is Binary Classification?

Binary classification is a supervised learning task where a model predicts one of two possible classes. Examples include:

• Email Filtering: Spam (1) or Not Spam (0).
• Medical Diagnosis: Disease present (1) or Absent (0).
• Loan Approval: Approved (1) or Rejected (0).
• Sentiment Analysis: Positive (1) or Negative (0).

The goal is to train a model that correctly separates these two categories based on input features.

How Binary Classification Works

1. Data Preparation

• Identify features (X) and labels (y) (e.g., in email spam detection, features might include the presence of specific words).
• Encode the labels as 0 and 1 (e.g., spam = 1, not spam = 0).
• Split the dataset into training and testing sets.

2. Choosing an Algorithm

Common algorithms for binary classification include:

• Logistic Regression (probability-based).
• Support Vector Machines (SVMs) (hyperplane separation).
• Decision Trees & Random Forests (rule-based models).
• Neural Networks (deep learning models).

3. Model Training

The model learns patterns from the training data to distinguish between the two classes.

4. Making Predictions

The model outputs a probability score between 0 and 1. A threshold (often 0.5) is used to classify:

• If probability ≥ 0.5, predict 1.
• If probability < 0.5, predict 0.

5. Model Evaluation

To measure performance, we use metrics like:

• Accuracy = (Correct Predictions) / (Total Predictions).
• Precision & Recall (for imbalanced datasets).
• F1-Score (balances precision & recall).
• Confusion Matrix (shows correct & incorrect classifications).

Hands-On Mini-Challenge: Binary Classification with Logistic Regression

Let’s classify emails as spam or not using a simple Logistic Regression model.

Starter Code:

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

# Load dataset (Replace with actual dataset)
emails = pd.read_csv('https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms-spam.csv')

# Prepare features and labels
X = emails['message']
y = (emails['label'] == 'spam').astype(int)  # Convert 'spam' to 1, 'ham' to 0

# Convert text data into numerical vectors
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(X)

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train logistic regression model
model = LogisticRegression()
model.fit(X_train, y_train)

# Predictions and evaluation
y_pred = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))
print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))
print("Classification Report:\n", classification_report(y_test, y_pred))
        

Best Practices for Binary Classification

• Balance Your Data – If one class dominates (e.g., 95% non-spam), use oversampling or SMOTE.
• Choose the Right Evaluation Metric – Accuracy alone isn’t enough! Use Precision, Recall, and F1-Score.
• Feature Engineering Matters – Text data requires vectorization; numerical data may need normalization.
• Hyperparameter Tuning – Optimize your model with GridSearchCV or RandomizedSearchCV.