Little Bits of Artificial Intelligence

1. Story: The Tale of the Nervous Painter

Imagine a neural network is like a painter who tries to copy a beautiful painting (the true function) by mixing different colors (features).

But the painter receives paint tubes with random sizes and pressures:

• Red paint comes in huge tubes (values between 100 to 200),
• Blue paint comes in tiny tubes (values between 0.1 to 0.5),
• Yellow paint is sometimes strong and sometimes faint.

Because of this imbalance in color strength, the painter messes up the painting—it’s too red and not balanced.

Enter Standardization: A wise art teacher tells the painter:

“Normalize your paint colors—make them all have the same mean (0) and spread (1) before painting!”

This way, all the paint colors behave the same and the painter can focus on blending them well.

2. What is Standardization?

Standardization (or Z-score normalization) transforms data as:

z=(x−μ) / σ

Where:

• x = original value
• μ = mean of the feature
• σ = standard deviation of the feature
• z = standardized value

3. Why is Standardization Important in Neural Networks?

• Speeds up training.
• Helps gradients flow evenly.
• Prevents some features from dominating the learning.
• Works especially well when using activation functions like sigmoid or tanh.

4. Story: The Train Station Balance Problem

The Setup:

Imagine we’re the operator of a metro train that runs through two different cities:

• City A is a small town where people carry light bags (1–5 kg),
• City B is a commercial hub where people carry heavy luggage (50–100 kg).

Every day, the train picks up passengers from both cities and logs the weight of their luggage and whether they reach their destination on time (label = 1) or not (label = 0).

We’re trying to predict on-time arrival using a simple AI model, based on:

• luggage_weight
• number_of_passengers

The Problem:

Our model gets confused. The large numbers from City B’s luggage overpower everything else.

The AI gives too much importance to luggage weight and ignores the passenger count, even though both are important!

The Fix: Standardization

A train engineer gives you advice:

“Why don’t you scale both inputs to a common level? Calculate the average (mean) and variability (standard deviation) of each, then express every value in terms of how far it is from the average!”

So now, instead of feeding raw weights like 75 kg and passenger counts like 3, we transform them like this:

Standardized_value = (original−mean) / std_dev

Now, both features are centered around 0 and have a similar influence on our model.
No single city dominates — the model becomes fair and accurate.

5. Real-Life Parallels

• Doctors standardize test scores to detect outliers regardless of unit.
• Schools standardize grades when merging marks from different exams (one out of 100, another out of 10).
• Credit systems normalize income and spending behavior to detect risky customers.

Key Takeaway

• Without standardization, features with larger values dominate learning.
• With standardization, the model learns from patterns, not magnitudes.