Little Bits of Artificial Intelligence

1. Let’s simulate a basic neural network with 1 input, 1 neuron, and calculate MSE manually:

# Sample dataset
X = [1, 2, 3, 4]           # Inputs
Y_true = [2, 4, 6, 8]      # Expected outputs (2x relationship)

# Initial weight and bias (you can try different values)
weight = 1.0
bias = 0.0

# Predictions and MSE calculation
predictions = []
squared_errors = []

for i in range(len(X)):
    y_pred = weight * X[i] + bias
    predictions.append(y_pred)
    error = Y_true[i] - y_pred
    squared_errors.append(error ** 2)

mse = sum(squared_errors) / len(squared_errors)

print("Predictions:", predictions)
print("Squared Errors:", squared_errors)
print("Mean Squared Error:", mse)

2. Extending the previous example into a mini neural network training simulation with:

• One input
• One output neuron
• Manual weight and bias adjustment
• MSE-based learning across multiple epochs

Step-by-Step Logic for Training with MSE

We’ll add:

1. Forward pass: Calculate predicted values using current weight & bias
2. Loss calculation: Use MSE to compute error
3. Backward pass (gradient descent): Adjust weights and bias based on error
4. Repeat the above steps for multiple epochs

Python Code: Training with MSE over Epochs (No Libraries)

# Dataset
X = [1, 2, 3, 4]
Y_true = [2, 4, 6, 8]  # Goal is to learn y = 2x

# Initialize weight and bias
weight = 0.0
bias = 0.0

# Learning rate
lr = 0.01

# Number of training iterations (epochs)
epochs = 50

# For visualizing the loss
loss_history = []

# Training loop
for epoch in range(epochs):
    total_error = 0
    dw = 0  # gradient for weight
    db = 0  # gradient for bias

    # Loop through each data point
    for i in range(len(X)):
        x = X[i]
        y = Y_true[i]

        # Forward pass: prediction
        y_pred = weight * x + bias

        # Error calculation
        error = y - y_pred
        total_error += error ** 2

        # Gradients (partial derivatives of MSE w.r.t weight and bias)
        dw += -2 * x * error
        db += -2 * error

    # Average the loss
    mse = total_error / len(X)
    loss_history.append(mse)

    # Update weight and bias using gradient descent
    weight -= lr * (dw / len(X))
    bias -= lr * (db / len(X))

    # Print progress
    print(f"Epoch {epoch+1:2d} | MSE: {mse:.4f} | Weight: {weight:.4f} | Bias: {bias:.4f}")

# Final model
print("\nFinal Model: y = {:.2f}x + {:.2f}".format(weight, bias))

What You’ll See in Output:

• MSE goes down over epochs
• Weight gets closer to 2
• Bias gets closer to 0
• Shows how learning happens step by step!

3. Flowchart: MSE-Based Training with Weight Adjustment

flowchart TD

Start([Start: Initialize weights and bias])
A1[Set hyperparameters (learning rate, epochs)]
A2[Loop over Epochs]
A3[Initialize total error, dw, db = 0]

subgraph Sample Loop [Loop through all training samples]
   B1[Get x and y_true]
   B2[Forward Pass: y_pred = weight * x + bias]
   B3[Error = y_true – y_pred]
   B4[Accumulate total_error += error²]
   B5[Accumulate gradients: dw += -2 * x * error, db += -2 * error]
end

A4[Compute MSE = total_error / n]
A5[Update weight: weight -= lr * (dw / n)]
A6[Update bias: bias -= lr * (db / n)]
A7{More Epochs?}
A8([End Training: Output final weight and bias])

Start –> A1 –> A2 –> A3 –> Sample Loop
Sample Loop –> A4 –> A5 –> A6 –> A7
A7 — Yes –> A2
A7 — No –> A8

Flow Summary:

• Each epoch: all samples are used to calculate total error and gradients.
• MSE is computed to assess prediction error.
• Weight and bias are updated to reduce error (Gradient Descent).
• Process repeats for several epochs until error reduces sufficiently.