How Neural Networks Work: Why Straight Lines Aren't Enough

A straight line equation is a good place to start understanding how a neural network works. That's what we explored in my previous article, How Neural Networks Work – Explained Using the Straight Line Equation (y=ax+b).

But what if the relationship between the input and the output isn't linear?

In the real world, data doesn't always follow a straight-line pattern. This is where things get interesting: sometimes a single rule isn't enough to describe the relationship between variables.

That's where non-linearity comes in. It allows neural networks to capture changing behavior in data—an essential capability as models in 2026 tackle increasingly complex, high-dimensional datasets.

In this article, we'll explore the concept of non-linearity and how it captures complex relationships in data. Using a simple example, we'll examine piecewise linear curves, ReLU functions, S-shaped curves, and sigmoid functions—and how they help neural networks learn patterns that a straight line can't describe.

Table of Contents

• Prerequisites
• When a Straight Line Is Enough
• When a Straight Line Isn't Enough
• Piecewise Linear Curve
• What Is a ReLU Function?
• S-Shaped Curve
• What Is a Sigmoid Function?
• Modeling a Deep Neural Network...

via FreeCodeCamp