Little Bits of Artificial Intelligence

1. Real-Life Story: Predicting House Prices

Imagine an estate agent wants to predict house prices based on these features:

• Area of the house
• Number of bedrooms
• Number of bathrooms
• Garden size
• Wall paint color
• Type of flooring

We’ve collected data on 50 houses. Now we want a formula to estimate the price of a new house.

2. Step-by-Step Walkthrough of ElasticNet Regression

Step 1: Understand the Goal
We want to build a model like:

price=w1×area+w2×bedrooms+…+b

Where:

• wi are weights (how important each feature is)
• b is a constant (bias term)
• We adjust the weights so the predicted prices are close to actual prices

Step 2: Why We Need Regularization

Some problems arise:

• Too many features (we don’t know which ones matter most)
• Some features are similar (e.g., area and bedrooms)
• Our model may “memorize” the data (called overfitting)

Solution? Add a penalty to keep the model simple and general.

Step 3: What ElasticNet Does

It adds two kinds of penalties when adjusting weights:

Loss=Error+λ1×(L1 penalty)+λ2×(L2 penalty)

• Error: Difference between actual and predicted prices
• L1 penalty (Lasso): Pushes some weights to zero → removes unimportant features
• L2 penalty (Ridge): Keeps weights small → prevents big jumps due to noise

Think of ElasticNet as a “coach” that:

• Sweeps away the useless features (L1)
• Keeps the useful ones under control (L2)

Step 4: Preparing the Data

We normalize each feature. For example:

• Area is between 1000 and 2000 sq ft → scaled to 0 to 1
• Bedrooms is between 2 and 5 → scaled to 0 to 1

This makes all features equally important at the start.

Step 5: Start with Random Weights

• Assign a starting weight of 0 for each feature.
• Also set a “bias” (intercept) — maybe 0 for now.

Example:

area = 0.7
bedrooms = 0.5
bathrooms = 0.3
…

Step 6: Predict Prices & Calculate Error

For each house, use the formula and predict the price.

y^=w1⋅x1+w2⋅x2+…+b

Then compare with the actual price to get the error.

error=y^−y

Step 7: Update the Weights (Gradient Descent)

Now update each weight based on:

• How much it contributed to the error (standard part)
• + L1 penalty (push it toward 0)
• + L2 penalty (shrink it if it’s too large)

So the formula becomes:

wj=wj−α×(error part+λ1⋅sign(wj)+2λ2⋅wj)

Where:

• α\alphaα is learning rate
• λ1\lambda_1λ1​ and λ2\lambda_2λ2​ control how strong the Lasso and Ridge parts are

3. Real-Life Intuition for Regularization Terms:

Step 8: Repeat the Process

Repeat the above process many times (epochs) until:

• The error becomes small
• Weights stabilize

Final Outcome

We now have a smart, simplified model:

price = 42000 * area + 21000 * bedrooms + 15000 * garden – 500 * wall_color + 4000 * flooring + 12000

• Irrelevant features (like wall_color) may have zero weight
• Correlated features (like area + bedrooms) are balanced
• Our model now generalizes better on unseen houses