Implementing a Neural Network from Scratch in Python

This guide outlines a step-by-step approach to implementing a basic neural network from scratch in Python. The neural network will include a single hidden layer and will be trained using backpropagation.

Table of Contents

1. Prerequisites
2. Importing Necessary Libraries
3. Defining Neural Network Architecture
4. Creating Functions
   • Activation Function
   • Forward Propagation
   • Backward Propagation
   • Training the Model
5. Code Usage Example
6. Conclusion

---

1. Prerequisites

Before diving into the code, ensure you have the following libraries installed:

• NumPy: For numerical calculations
• Matplotlib: For plotting (optional, for visualization)

pip install numpy matplotlib

2. Importing Necessary Libraries

import numpy as np
import matplotlib.pyplot as plt  # Optional for visualization

3. Defining Neural Network Architecture

We'll create a class for our neural network that will handle the operations for a simple architecture.

class NeuralNetwork:
    def __init__(self, input_size, hidden_size, output_size):
        """
        Initializes the neural network with weights.
        
        Parameters:
        input_size (int): Number of input features
        hidden_size (int): Number of neurons in hidden layer
        output_size (int): Number of output neurons
        """
        # Weights initialization with small random values
        self.W1 = np.random.randn(input_size, hidden_size) * 0.01
        self.W2 = np.random.randn(hidden_size, output_size) * 0.01
        self.b1 = np.zeros((1, hidden_size))
        self.b2 = np.zeros((1, output_size))

## 4. Creating Functions

### Activation Function

Define the sigmoid activation function and its derivative.

```python
    def sigmoid(self, z):
        """Applies the sigmoid function."""
        return 1 / (1 + np.exp(-z))

    def sigmoid_derivative(self, z):
        """Returns the derivative of the sigmoid function."""
        return z * (1 - z)

Forward Propagation

Implement the forward pass through the network.

    def forward(self, X):
        """
        Performs forward propagation.

        Parameters:
        X (np.ndarray): Input data

        Returns:
        np.ndarray: Output of the neural network after forward pass
        """
        self.z1 = np.dot(X, self.W1) + self.b1
        self.a1 = self.sigmoid(self.z