Implementing a Neural Network from Scratch in Python

This guide will walk you through the process of implementing a basic neural network from scratch in Python. The neural network will perform binary classification using the sigmoid activation function and the mean squared error loss function.

Table of Contents

1. Necessary Imports
2. Neural Network Class
3. Training and Predictions
4. Example Usage
5. Conclusion

1. Necessary Imports

We will need to import numpy for numerical operations.

import numpy as np

2. Neural Network Class

Code Implementation

class NeuralNetwork:
    """
    A simple feedforward neural network for binary classification.
    
    Attributes:
        input_size (int): Number of input features.
        hidden_size (int): Number of neurons in the hidden layer.
        output_size (int): Number of output classes.
        weights_input_hidden (ndarray): Weights from input to hidden layer.
        weights_hidden_output (ndarray): Weights from hidden to output layer.
        learning_rate (float): Learning rate for weight updates.
    """

    def __init__(self, input_size, hidden_size, output_size, learning_rate=0.01):
        """
        Initialize the neural network with specified sizes.
        
        Parameters:
            input_size (int): The number of input features.
            hidden_size (int): The number of neurons in the hidden layer.
            output_size (int): The number of output classes.
            learning_rate (float): Learning rate for the optimization.
        """
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size
        self.learning_rate = learning_rate
        
        # Initialize weights with small random values
        self.weights_input_hidden = np.random.rand(self.input_size, self.hidden_size) * 0.01
        self.weights_hidden_output = np.random.rand(self.hidden_size, self.output_size) * 0.01

    def sigmoid(self, z):
        """Sigmoid activation function."""
        return 1 / (1 + np.exp(-z))
    
    def sigmoid_derivative(self, z):
        """Derivative of the sigmoid function."""
        return z * (1 - z)

    def forward(self, x):
        """Forward pass through the network."""
        self.hidden_layer_input = np.dot(x, self.weights_input_hidden)
        self.hidden_layer_output = self.sigmoid(self.hidden_layer_input)
        
        self.output_layer_input = np.dot(self.hidden_layer_output, self.weights_hidden_output)
        self.output = self.sigmoid(self.output_layer_input)
        
        return self.output

    def backward(self, x, y):
        """Backward pass for the network."""
        output_loss = self.output - y  # Error in output
        output_gradient = self.sigmoid_derivative(self.output) * output_loss
        
        hidden_loss = np.dot(output_gradient, self.weights_hidden_output.T)
        hidden_gradient = self.sigmoid_derivative(self.hidden_layer_output) * hidden_loss
        
        # Update weights
        self.weights_hidden_output -= self.learning_rate * np.dot(self.hidden_layer_output.T, output_gradient)
        self.weights_input_hidden -= self.learning_rate * np.dot(x.T, hidden_gradient)

    def train(self, x, y, epochs):
        """Train the neural network."""
        for _ in range(epochs):
            # Forward and backward pass
            self.forward(x)
            self.backward(x, y)

    def predict(self, x):
        """Make predictions on new data."""
        output = self.forward(x)
        return np.round(output)  # Round the output to get binary predictions

3. Training and Predictions

Training the Model

To train the model, you need datasets with input features and corresponding binary labels.

Example Training Code

# Assume X is the input matrix and y is the labels
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])  # XOR problem

# Create a neural network with 2 input features, 2 hidden neurons, and 1 output
nn = NeuralNetwork(input_size=2, hidden_size=2, output_size=1)

# Train the model for 10000 epochs
nn.train(X, y, epochs=10000)

# Make predictions
predictions = nn.predict(X)
print(predictions)  # Output should be close to [[0], [1], [1], [0]]

4. Example Usage

Here's a complete example that demonstrates usage of the NeuralNetwork class:

if __name__ == "__main__":
    # Define input and output datasets
    X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
    y = np.array([[0], [1], [1], [0]])  # XOR output
    
    # Initialize the neural network
    nn = NeuralNetwork(input_size=2, hidden_size=2, output_size=1)

    # Train the neural network
    nn.train(X, y, epochs=10000)

    # Predict the outputs
    predictions = nn.predict(X)
    print("Predictions:")
    print(predictions)

5. Conclusion

This simple neural network implementation demonstrates the fundamentals of how a neural network operates, encompassing forward propagation, loss calculation, backpropagation, and weight updates. You can build upon this foundation for more complex networks and tasks. For deeper learning in data science and related fields, consider exploring courses on the Enterprise DNA Platform.