Little Bits of Artificial Intelligence

1. Visual Flow

Initial State

We start with an average guess — everyone gets the same predicted price (450).
Error is large, especially for very small or very large houses.

Round 1 – Learn from the Residuals (Mistakes)

We’re now a bit closer to the actual values — still wrong, but better!

Round 2 – Learn from New Residuals

We can see how each step gets us closer, not perfect, but less wrong.

Round 3 – Tiny Corrections

Now the residuals are nearly zero, and the predictions match closely with actual prices.

Stopping Criteria

We stop when:

• Residuals are consistently small
• Further corrections become negligible
• Model stops improving on validation data

Overall Process as a Flowchart

2. Step-by-Step Process of Gradient Boosting Regression

Step 1: Prepare Your Dataset

• Organize input features X and output target y.

Example:

Step 2: Start with an Initial Prediction

• This is usually the mean of all target values (for regression).
• E.g., If our target prices are [150, 300, 450, 600], our first guess is the average: 375 for every house.

Step 3: Calculate Residuals

• Subtract the prediction from the actual value.
• Residual = Actual – Predicted
• These residuals are the mistakes the model needs to fix.

Step 4: Train a Weak Learner (like a small decision tree)

• Fit a small tree to predict the residuals, not the final target.
• This tree learns how the model is wrong and tries to correct that.

Step 5: Add the Learner’s Output to the Previous Prediction

• Update the model:
  New Prediction = Previous Prediction + Learning Rate × Correction
• The learning rate controls how big a step we take.

Step 6: Repeat Steps 3–5 for Multiple Rounds

• Use the new predictions to calculate new residuals.
• Train another weak learner on these new residuals.
• Keep adding corrections until the model improves very little.

Step 7: Final Prediction

• After all boosting rounds, combine all the learners’ predictions.
• This gives the final, refined prediction — much closer to real values than the initial guess.

Step 8: Evaluate Model Performance

• Use metrics like Mean Squared Error (MSE) or R² score to measure accuracy.
• Optionally validate using cross-validation or a test dataset.

Summary Flow (Plain Words)

1. Guess something simple (like average)
2. See how wrong you were (errors)
3. Train a model to fix the error
4. Add the fix to the previous guess
5. Repeat until you can’t fix much more
6. Combine all guesses for a final answer