Introduction: The Fixed-Size Input Problem
In the past, a CNN model required a fixed input dimension. This was a significant limitation because objects in real-world images naturally vary in shape and size. For instance, humans typically have a vertical orientation, while cars are better approximated with a horizontal bounding box. The most straightforward solutions—cropping or warping—force images into a uniform size, but each has drawbacks: cropping may truncate parts of the object, while warping distorts the object’s appearance. Both transformations alter the visual characteristics of objects, which can reduce model accuracy when training a classification model. Figure 1 illustrates these issues.
In this article, we will explore SPP-Net, a CNN-based model that accepts images of varying sizes, thereby avoiding the need for cropping or warping entirely. Beyond the theory, I will implement SPP-Net from scratch using PyTorch to provide a deeper understanding of its architecture.
A Brief History of SPP-Net
SPP-Net was introduced in the paper “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition” by He et al., published in June 2014 [1]. Notably, the concept of Spatial Pyramid Pooling (SPP) predates this work, but the paper proposed SPP-Net—the first model to integrate SPP into a CNN architecture, marking a pivotal advancement in handling variable-size inputs.
CNN vs. SPP-Net
The upper flowchart in Figure 2 depicts the conventional CNN workflow, which requires cropping or warping to standardize image sizes before input. In contrast, the lower flowchart shows the SPP-Net flow, where the SPP layer sits between the convolution layers and the fully-connected (FC) layers, eliminating the need for pre-processing transformations.
Looking deeper into CNN architecture, the fixed-size constraint stems from the fully-connected layers. Convolutional layers can naturally handle varying input sizes because they use shared weights across spatial dimensions. For example, a 3×3 kernel has 9 trainable weights (or 10 with bias), regardless of input image size. This contrasts with FC layers, where each element in the flattened tensor requires a corresponding weight. For a 10×10 image, flattening produces a 100-element vector, necessitating 100 weights (or 101 with bias). Thus, any change in input shape would require a different number of weights, making FC layers unsuitable for variable-size inputs.
To address this, the authors proposed a mechanism that leverages CNNs' flexibility in handling variable input sizes while converting the extracted feature maps into a fixed-size vector for the FC layers. This is why, as shown in Figure 2, the SPP layer is positioned between the convolutional stack and the FC layers.
How the SPP Layer Works
Figure 3 below outlines the detailed processing steps of SPP-Net. Consistent with the lower flowchart in Figure 2, the network first applies a series of convolutional layers to extract feature maps, which can vary in spatial dimensions. The SPP layer then partitions each feature map into a set of spatial bins at multiple scales (e.g., 1×1, 2×2, 4×4). Within each bin, a pooling operation (e.g., max pooling) produces a fixed-length summary, regardless of the original feature map size. These pooled outputs are concatenated to form a fixed-dimensional vector, which is then fed into the fully-connected layers for classification or detection. This design allows SPP-Net to handle arbitrary input sizes while maintaining compatibility with standard FC layers.
In practice, for an input image of any size, the SPP layer adaptively adjusts the pooling window size and stride to produce the desired spatial pyramid levels. This eliminates the need for fixed-size inputs and enables the model to process images at their native resolutions, preserving fine details that might otherwise be lost through cropping or warping.
By 2026, this approach has influenced modern architectures, such as feature pyramid networks (FPNs) and vision transformers, which also aim to handle multi-scale representations. However, SPP-Net remains a foundational milestone in deep learning for object recognition, offering a simple yet effective solution to the fixed-size constraint.
For those interested in practical implementation, the following section provides a PyTorch code walkthrough of SPP-Net, demonstrating how to build and integrate the SPP layer into a custom model.