Little Bits of Artificial Intelligence

1. Story-Style Explanation: Why Normalization?

Story: The Fair Running Race

Imagine we’re organizing a school race where kids of different ages are competing. But here’s the twist:

• Some kids run 10 meters,
• Some run 100 meters,
• Some run 1 kilometer.

Now, if we directly compare the time taken to finish the race, that would be unfair, right? The one running 10 meters would obviously finish faster.

So what do we do?
We normalize the data — we calculate speed = distance / time — which puts everyone on the same scale. Now we can compare who ran fastest fairly, regardless of distance.

In neural networks, different features (like age, income, temperature, etc.) may have different scales, just like different race distances.
If we don’t normalize, features with larger values dominate the learning, leading to biased or slow training.

2. Basic Math Concepts Behind Normalization

Mean Normalization (Z-Score or Standard Score):

Normalized value=(x−μ) / σ

Where:

• x: original value
• μ: mean of the column
• σ: standard deviation of the column

This makes the feature have mean 0 and standard deviation 1.

3. Encoding and Normalization – Key Difference

• Encoding is for categorical data (like One-Hot or Label Encoding).
• Normalization is for numerical data (like height, age, income).

They often go hand-in-hand when preparing inputs for a neural network.