Riya Upadhyay

Building My First Machine Learning Model: A Step-by-Step Guide

Riya Upadhyay's photo
Riya Upadhyay
·

4 min read

Introduction

In this guide, I'll walk you through building a machine learning model to predict house prices. I chose this project because it's relatable and perfect for beginners. I'll share my mistakes and lessons learned along the way.

Project Overview

Goal: Build a model to predict house prices based on features like square footage, number of bedrooms, etc. Type: Regression problem (predicting a continuous value) Dataset: Housing dataset (we'll use the Boston Housing dataset from scikit-learn)

Step 1: Setting Up the Environment

# Essential libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

# Set random seed for reproducibility
np.random.seed(42)

Mistake #1: Initially, I forgot to set a random seed, which meant I got different results each time I ran my code. Setting a seed ensures reproducibility.

Step 2: Loading and Exploring the Data

# Load dataset
boston = load_boston()
data = pd.DataFrame(boston.data, columns=boston.feature_names)
data['PRICE'] = boston.target

# Basic data exploration
print("Dataset Shape:", data.shape)
print("\nFirst few rows:")
print(data.head())
print("\nData Info:")
print(data.info())

Mistake #2: I jumped straight into modeling without understanding my data. Always explore your data first!

Step 3: Data Analysis and Visualization

# Check for missing values
print("Missing Values:")
print(data.isnull().sum())

# Basic statistics
print("\nBasic Statistics:")
print(data.describe())

# Correlation analysis
plt.figure(figsize=(10, 8))
sns.heatmap(data.corr(), annot=True, cmap='coolwarm', center=0)
plt.title('Correlation Matrix')
plt.show()

# Relationship between price and most correlated feature
plt.figure(figsize=(10, 6))
plt.scatter(data['RM'], data['PRICE'])
plt.xlabel('Average Number of Rooms')
plt.ylabel('Price ($1000s)')
plt.title('Price vs. Number of Rooms')
plt.show()

Lesson Learned: Visualizations helped me understand relationships in the data that weren't obvious from just looking at numbers.

Step 4: Data Preprocessing

# Separate features and target
X = data.drop('PRICE', axis=1)
y = data['PRICE']

# Split the data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print("Training set shape:", X_train.shape)
print("Testing set shape:", X_test.shape)

Mistake #3: Initially, I didn't split my data into training and testing sets, which meant I couldn't properly evaluate my model's performance.

Step 5: Building the Model

# Create and train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Make predictions
y_pred = model.predict(X_test)

# Evaluate the model
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print("Mean Squared Error:", mse)
print("R² Score:", r2)

Mistake #4: I originally only looked at R² score, which didn't give me the full picture of model performance.

Step 6: Model Analysis

# Feature importance
feature_importance = pd.DataFrame({
    'Feature': X.columns,
    'Coefficient': model.coef_
})
feature_importance = feature_importance.sort_values('Coefficient', 
                                                  key=abs, 
                                                  ascending=False)

plt.figure(figsize=(10, 6))
sns.barplot(x='Coefficient', y='Feature', data=feature_importance)
plt.title('Feature Importance')
plt.show()

# Predicted vs Actual values
plt.figure(figsize=(10, 6))
plt.scatter(y_test, y_pred)
plt.plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'r--', lw=2)
plt.xlabel('Actual Price')
plt.ylabel('Predicted Price')
plt.title('Actual vs Predicted House Prices')
plt.show()

Lesson Learned: Visualizing predictions helped me understand where my model was making mistakes.

Step 7: Making Predictions

# Example: Predict price for a new house
new_house = np.array([[0.00632, 18.0, 2.31, 0, 0.538, 6.575, 65.2, 4.0900,
                       1, 296.0, 15.3, 396.90, 4.98]])
predicted_price = model.predict(new_house)
print(f"Predicted Price: ${predicted_price[0]*1000:.2f}")

Mistake #5: I forgot to scale my features initially, which can be important for many algorithms (though not critical for linear regression).

Common Mistakes and Lessons Learned

Mistakes to Avoid

  1. Not setting a random seed

  2. Skipping data exploration

  3. Not splitting data into train/test sets

  4. Relying on a single metric

  5. Not handling outliers or missing values

  6. Overfitting to the training data

  7. Not documenting your process

Best Practices

  1. Always explore your data first

  2. Use multiple evaluation metrics

  3. Visualize results

  4. Start simple and iterate

  5. Keep track of your experiments

  6. Document your code

  7. Test your model thoroughly

Improvements for Next Time

  1. Feature Engineering

    • Create new features from existing ones

    • Transform skewed features

    • Handle categorical variables properly

  2. Cross-Validation

    • Use k-fold cross-validation for more robust evaluation

    • Try different validation strategies

  3. Model Selection

    • Try different algorithms

    • Use grid search for hyperparameter tuning

    • Ensemble methods

Conclusion

Building my first machine learning model was a journey filled with mistakes and learning opportunities. The key is to start simple, understand each step, and iterate based on what you learn.

Next Steps

  1. Try different algorithms

  2. Add feature engineering

  3. Implement cross-validation

  4. Handle outliers and feature scaling

  5. Deploy the model

Remember: Every data scientist started with their first model. Don't be afraid to make mistakes – they're your best teachers!

Machine Learning