XGBoost Regression on Boston Housing Dataset

Code Explanation

Importing Necessary Libraries

import xgboost as xgb
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

• xgboost: A library for implementing the XGBoost algorithm, which is an optimized gradient boosting framework.
• load_boston: A function from sklearn.datasets to load the Boston housing dataset.
• train_test_split: A utility from sklearn.model_selection for splitting data into training and testing sets.
• mean_squared_error: A function from sklearn.metrics to compute the Mean Squared Error, a common evaluation metric for regression.

Loading the Dataset

data = load_boston()
X, y = data.data, data.target

• data: The Boston housing dataset.
• X: Features/data points.
• y: Target variable/labels.

Splitting Data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

• X_train, X_test: Training and testing data set for the features.
• y_train, y_test: Training and testing data set for the target variable.
• test_size=0.2: 20% of the data will be used for testing.
• random_state=42: A seed is set for reproducibility.

Initializing XGBoost Model

xg_reg = xgb.XGBRegressor(objective='reg:squarederror', colsample_bytree=0.3, learning_rate=0.1,
                          max_depth=5, alpha=10, n_estimators=10)

• objective='reg:squarederror': The loss function used.
• colsample_bytree=0.3: The fraction of features to consider for each tree.
• learning_rate=0.1: The step size for updating weights.
• max_depth=5: The maximum depth of a tree.
• alpha=10: L1 regularization term on weights.
• n_estimators=10: Number of trees in the model.

Training the Model

xg_reg.fit(X_train, y_train)

• Fits the model on the training data.

Making Predictions

preds = xg_reg.predict(X_test)

• preds: Predictions made by the model on the test data.

Evaluating the Model

rmse = mean_squared_error(y_test, preds, squared=False)
print(f"RMSE: {rmse}")

• rmse: Root Mean Squared Error, calculated as the square root of the mean squared error. It serves as an evaluation metric for the model.
• print(f"RMSE: {rmse}"): Prints the RMSE value for the predictions.

Summary

The code demonstrates the use of the XGBoost library to perform regression on the Boston housing dataset. The process involves:

1. Importing necessary libraries.
2. Loading and splitting the dataset.
3. Initializing the model with specific parameters.
4. Training the model on the training data.
5. Making predictions on the test data.
6. Evaluating the model performance using RMSE.

For more in-depth understanding of algorithms like XGBoost and their practical application, consider exploring courses on the Enterprise DNA platform.