Input scaling and batch normalization AI

Distinguishing fixed sensor scaling from normalization layers inside an image network.

Input standardization expresses a value relative to a reference mean and standard deviation: $x'=(x-\mu)/\sigma$. Fitting those statistics on training data and reusing them for evaluation avoids adapting preprocessing to held-out data. The StandardScaler documentation describes this operation.

For the GRU, each of the nine channels has one training-derived mean and standard deviation, computed across training windows and time. The report plots acceleration in g and angular velocity in rad/s; the model receives standardized values. Units on a plot therefore do not describe the numeric scale entering the GRU.

The image model uses ImageNet RGB normalization at its input. It also contains batch-normalization layers with learned scale and offset and stored running statistics. These are different from input preprocessing; see BatchNorm2d.

In this pet pilot, running statistics stay frozen. Full-network fine-tuning can still update learned scale and offset parameters. “Normalization is fixed” would therefore be too broad a description of what is held constant.

Explore the neural-network demo.