Neural Networks Explained: What They Are and How to Build One in Python

In 2026, neural networks are the backbone of countless AI applications, from voice assistants and recommendation engines to autonomous vehicles and medical diagnostics. While the term might sound intimidating—conjuring images of advanced calculus, high-performance computing clusters, and thousands of lines of code—the fundamentals are simpler than you'd think.

At its core, a neural network is a mathematical model that takes numeric input, processes it through a series of calculations, makes a prediction, evaluates the error, and adjusts its parameters to improve over time. With just Python and NumPy, you can build a fully functional neural network from scratch.

What Is a Neural Network?

A neural network is a computational system inspired by the biological neurons in the human brain, though the analogy is loose. It consists of interconnected nodes (called neurons) organized into layers. Each connection has an associated weight and bias, and each neuron applies an activation function to its input. By adjusting these parameters during training, the network learns to map inputs to outputs.

Why Are They Called Neural Networks?

The name comes from their loose resemblance to biological neural networks—structures of interconnected neurons that transmit electrical signals. However, artificial neural networks are purely mathematical constructs; the biological inspiration mainly serves as an intuitive heuristic. They are not simulations of the brain but rather powerful pattern-recognition tools.

The Three Main Parts of a Neural Network

Input layer: Receives the raw data (e.g., pixel values, feature vectors).
Hidden layers: Perform intermediate transformations, extracting features and patterns.
Output layer: Produces the final prediction (e.g., a class label or a continuous number).

Each layer consists of multiple neurons. Data flows from input to output, with each neuron computing a weighted sum of its inputs, adding a bias, and passing the result through an activation function.

What Is a Neuron?

In the context of a neural network, a neuron is a computational unit that takes one or more inputs, multiplies each by a weight, sums them, adds a bias, and applies an activation function. The output is then passed to the next layer.

Mathematically, for a neuron with inputs \( x1, x2, \dots, xn \), weights \( w1, w2, \dots, wn \), bias \( b \), and activation function \( f \):

\[ y = f\left(\sum{i=1}^{n} wi x_i + b\right) \]

What Is a Weight?

A weight determines the strength and direction of influence of an input on the neuron's output. During training, the network adjusts weights to minimize prediction error. Higher weights amplify the input's effect; negative weights can inhibit it.

What Is a Bias?

A bias is an additional parameter that allows the activation function to shift. Without a bias, the neuron's output would always pass through the origin, limiting its ability to fit data. The bias acts like an intercept in linear regression, enabling the network to represent more complex functions.

Why Do We Need Activation Functions?

Activation functions introduce nonlinearity into the network. Without them, stacking linear transformations would collapse into a single linear operation, making deep networks trivial. Common activation functions include:

ReLU (Rectified Linear Unit): \( f(x) = \max(0, x) \) – simple and efficient, widely used in hidden layers.
Sigmoid: \( f(x) = \frac{1}{1+e^{-x}} \) – outputs between 0 and 1, useful for binary classification.
Tanh: \( f(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}} \) – outputs between -1 and 1, often used in recurrent networks.
Softmax: Used in the output layer for multi-class classification, converting logits into probabilities.

Building Our First Neuron in Python

Let's implement a simple neuron using NumPy.

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

class Neuron:
    def __init__(self, n_inputs):
        # Initialize weights and bias randomly
        self.weights = np.random.randn(n_inputs)
        self.bias = np.random.randn()

    def forward(self, inputs):
        # Weighted sum plus bias
        z = np.dot(self.weights, inputs) + self.bias
        # Apply activation (sigmoid)
        return sigmoid(z)

# Example usage
neuron = Neuron(3)
inputs = np.array([0.5, 0.8, 0.2])
output = neuron.forward(inputs)
print(f"Neuron output: {output}")

In a real network, you'd have many neurons arranged in layers, and you'd perform forward propagation by iterating through layers.

From One Neuron to a Layer

A layer is a collection of neurons that all receive the same input (from the previous layer) but have their own weights and biases. The output of one layer becomes the input to the next.

Here's how to create a dense (fully connected) layer:

class DenseLayer:
    def __init__(self, n_inputs, n_neurons):
        self.weights = np.random.randn(n_inputs, n_neurons)
        self.biases = np.zeros((1, n_neurons))

    def forward(self, inputs):
        z = np.dot(inputs, self.weights) + self.biases
        return sigmoid(z)  # or any activation

To build a full network, stack multiple layers and feed the output of each layer to the next.

How Does a Neural Network Actually Learn?

Learning is the process of adjusting weights and biases to minimize a loss function that quantifies prediction error. The most common method is gradient descent:

Perform forward propagation to get predictions.
Compute the loss (e.g., mean squared error or cross-entropy).
Use backpropagation to calculate the gradient of the loss with respect to each weight.
Update weights in the opposite direction of the gradient, scaled by a learning rate.

In code, a simple update step looks like:

# Assuming gradients are already computed
weights -= learning_rate * dW
biases -= learning_rate * db

Gradient descent comes in variants: stochastic, mini-batch, and batch. In 2026, adaptive optimizers like Adam and RMSprop are standard, but the core principle remains the same.

Predictions and Loss

During training, you compare the network's output to the ground truth. For regression, use Mean Squared Error (MSE):

\[ L = \frac{1}{n}\sum{i=1}^{n} (yi - \hat{y}_i)^2 \]

For classification, cross-entropy loss is more appropriate.

Training Loop in Python

Here's a minimal training loop for a one-layer network (logistic regression) to illustrate:

# Assuming X is input data, y is labels (0/1)
learning_rate = 0.1
epochs = 1000

# Initialize weights
W = np.random.randn(X.shape[1])
b = 0

for epoch in range(epochs):
    # Forward pass
    z = np.dot(X, W) + b
    preds = sigmoid(z)
    
    # Loss (binary cross-entropy)
    loss = -np.mean(y * np.log(preds) + (1 - y) * np.log(1 - preds))
    
    # Backward pass (gradients)
    dz = preds - y
    dW = np.dot(X.T, dz) / len(y)
    db = np.mean(dz)
    
    # Update parameters
    W -= learning_rate * dW
    b -= learning_rate * db

    if epoch % 100 == 0:
        print(f"Epoch {epoch}, Loss: {loss:.4f}")

This is the essence of training—it scales up to deeper networks with more complex architectures.

Building a Complete Neural Network in Python

Now let's put it all together: a two-layer neural network that can perform binary classification.

class NeuralNetwork:
    def __init__(self, n_inputs, n_hidden, n_output):
        # Initialize weights and biases
        self.W1 = np.random.randn(n_inputs, n_hidden) * 0.5
        self.b1 = np.zeros((1, n_hidden))
        self.W2 = np.random.randn(n_hidden, n_output) * 0.5
        self.b2 = np.zeros((1, n_output))
    
    def forward(self, X):
        self.z1 = np.dot(X, self.W1) + self.b1
        self.a1 = np.tanh(self.z1)  # activation hidden layer
        self.z2 = np.dot(self.a1, self.W2) + self.b2
        self.a2 = sigmoid(self.z2)
        return self.a2
    
    def backward(self, X, y, output, learning_rate):
        m = X.shape[0]
        # Output layer gradient
        delta2 = output - y
        dW2 = np.dot(self.a1.T, delta2) / m
        db2 = np.mean(delta2, axis=0, keepdims=True)
        # Hidden layer gradient
        delta1 = np.dot(delta2, self.W2.T) * (1 - np.tanh(self.z1)**2)
        dW1 = np.dot(X.T, delta1) / m
        db1 = np.mean(delta1, axis=0, keepdims=True)
        
        # Update
        self.W2 -= learning_rate * dW2
        self.b2 -= learning_rate * db2
        self.W1 -= learning_rate * dW1
        self.b1 -= learning_rate * db1
    
    def train(self, X, y, epochs, learning_rate):
        for epoch in range(epochs):
            output = self.forward(X)
            loss = -np.mean(y * np.log(output) + (1 - y) * np.log(1 - output))
            self.backward(X, y, output, learning_rate)
            if epoch % 100 == 0:
                print(f"Epoch {epoch}, Loss: {loss:.4f}")

This is a toy example, but it demonstrates the core mechanics. In practice, you'd use libraries like TensorFlow or PyTorch for efficiency and scalability, but understanding the underlying math is invaluable.

Limitations and Modern Developments (2026)

While basic neural networks are powerful, modern architectures—transformers, convolutional nets, and generative models—build on these fundamentals. Key advancements since the early days include:

Attention mechanisms: Allowing networks to weigh the importance of different parts of the input.
Advanced optimizers: Adam, RMSprop, and lookahead methods that accelerate training.
Regularization techniques: Dropout, batch normalization, and data augmentation to reduce overfitting.

Moreover, in 2026, there's a strong push toward interpretable AI and efficient training (e.g., federated learning, quantization). But every model still relies on the same principles we've implemented here.

Conclusion

You've just built a neural network from scratch in Python. The key takeaway: a neural network is a stack of layers that transform data through weighted sums, biases, and activation functions. Learning is simply an optimization problem—minimizing loss via gradient descent.

Understanding these fundamentals demystifies AI and provides a foundation for exploring more advanced topics. Whether you're working with deep learning frameworks or developing your own experiments, you now know exactly what's under the hood.

Further Resources

This article was reviewed by an AI technology editor for clarity and technical accuracy.

via FreeCodeCamp