Implementing a Neural Network from Scratch in Python

Creating a neural network from scratch in Python involves understanding the underlying mathematics and programming. Below is a structured approach to implementing a simple feedforward neural network using NumPy.

1. Necessary Imports

import numpy as np

2. Neural Network Structure

Class Definition

We'll define a NeuralNetwork class which will comprise methods for initializing weights, forward passing, backpropagation, and predicting outputs.

Code Structure

class NeuralNetwork:
    """
    A simple feedforward neural network with one hidden layer.

    Attributes:
        input_size (int): Number of input features.
        hidden_size (int): Number of neurons in the hidden layer.
        output_size (int): Number of output classes.
        learning_rate (float): Learning rate for weight updates.
        weights_input_hidden (np.ndarray): Weights from input to hidden layer.
        weights_hidden_output (np.ndarray): Weights from hidden to output layer.
        bias_hidden (np.ndarray): Bias for hidden layer.
        bias_output (np.ndarray): Bias for output layer.
    """

    def __init__(self, input_size, hidden_size, output_size, learning_rate=0.01):
        # Initialize input and output sizes, weights, and biases
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size
        self.learning_rate = learning_rate

        # Randomly initialize weights and biases
        self.weights_input_hidden = np.random.rand(self.input_size, self.hidden_size)
        self.weights_hidden_output = np.random.rand(self.hidden_size, self.output_size)
        self.bias_hidden = np.random.rand(self.hidden_size)
        self.bias_output = np.random.rand(self.output_size)

    def sigmoid(self, x):
        """Applies the sigmoid activation function."""
        return 1 / (1 + np.exp(-x))

    def sigmoid_derivative(self, x):
        """Computes the derivative of the sigmoid function."""
        return x * (1 - x)

    def forward(self, X):
        """Computes the forward pass through the network."""
        self.hidden_layer_input = np.dot(X, self.weights_input_hidden) + self.bias_hidden
        self.hidden_layer_output = self.sigmoid(self.hidden_layer_input)
        self.final_input = np.dot(self.hidden_layer_output, self.weights_hidden_output) + self.bias_output
        self.final_output = self.sigmoid(self.final_input)
        return self.final_output

    def backward(self, X, y):
        """Computes the backward pass and updates weights."""
        # Calculate output layer error
        output_error = y - self.final_output
        output_delta = output_error * self.sigmoid_derivative(self.final_output)

        # Calculate hidden layer error
        hidden_error = np.dot(output_delta, self.weights_hidden_output.T)
        hidden_delta = hidden_error * self.sigmoid_derivative(self.hidden_layer_output)

        # Update weights and biases
        self.weights_hidden_output += np.dot(self.hidden_layer_output.T, output_delta) * self.learning_rate
        self.weights_input_hidden += np.dot(X.T, hidden_delta) * self.learning_rate
        self.bias_output += np.sum(output_delta, axis=0) * self.learning_rate
        self.bias_hidden += np.sum(hidden_delta, axis=0) * self.learning_rate

    def train(self, X, y, epochs):
        """Trains the neural network over a specified number of epochs."""
        for epoch in range(epochs):
            self.forward(X)
            self.backward(X, y)

    def predict(self, X):
        """Predicts the output for given inputs X."""
        return self.forward(X)

3. Implementation Steps

1. Define the Neural Network: Create an instance of the NeuralNetwork class.
2. Train the Model: Use the train() method with your dataset.
3. Make Predictions: Call the predict() method to get the output.

4. Example Usage

Here’s a practical example demonstrating how to use the neural network:

# Example data (X: inputs, y: outputs)
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])  # XOR problem

# Create neural network instance
nn = NeuralNetwork(input_size=2, hidden_size=2, output_size=1, learning_rate=0.1)

# Train the model
nn.train(X, y, epochs=10000)

# Make predictions
predictions = nn.predict(X)
print("Predictions:\n", predictions)

5. Conclusion

Creating a neural network from scratch allows for a deep understanding of neural architecture and learning processes. This implementation demonstrates the essential components: initialization, forward propagation, backpropagation, and weight updates.

For those looking to dive deeper into data science and neural networks, consider exploring the Enterprise DNA Platform, which offers a plethora of resources and courses tailored to enhance your skills.