Little Bits of Artificial Intelligence

1. What is Gradient Boosting Regression?

Imagine we’re trying to guess a friend’s weight just by looking at them.

• Our first guess is way off.
• But our friend gives us a clue: “We’re 10 kg too low.”
• So, we adjust our guess to reduce the mistake.

Now imagine doing this:

• Again and again.
• Each time learning from the previous error.

That’s the idea behind Gradient Boosting Regression:

“Keep learning from the mistakes of the previous guesses, and get better each time.”

Let’s break it down simply:

Step-by-step Concept
1. Start with a simple guess (like average of all values).
2. Measure how far off that guess is (called error).
3. Build a small decision tree to predict that error.
4. Add this correction to our previous guess.
5. Repeat steps 2–4 for many rounds.
6. Final prediction = sum of all corrections.

Each new tree focuses on the residual error (the remaining mistake) of the previous trees.

Why is it called “Gradient” Boosting?

The word gradient comes from calculus — it means “direction of steepest improvement.”
Here, we’re using gradient (the direction to reduce error) to boost performance.

So we are:

• Minimizing error
• Using gradients to guide us
• Boosting (improving) the overall prediction

2. Real-Life Use Case

#1: House Price Prediction

Scenario:

We want to predict house prices.

• First model: Predicts based on average price in the city.
• Second model: Learns from mistakes—maybe notices “bigger houses are undervalued.”
• Third model: Learns that “houses near parks are still under-predicted.”

Each model focuses on errors made before and corrects them.

The final result is much more accurate than any individual model alone.
#2: Predicting Sales in a Store

Scenario:
We want to predict how much a store will sell next week.

• First guess: Average weekly sales.
• Next model: Adds corrections based on discounts.
• Next one: Adds correction based on holidays.
• Next one: Adds effect of weather.

By stacking these smart “mistake correctors”, the final prediction becomes highly reliable.