Little Bits of Artificial Intelligence

1. Inputs

Think of inputs as numbers representing features. Say, we’re trying to predict if someone likes mango juice:

• input1 = sweetness (say 8)
• input2 = thickness (say 5)

Inputs are just numbers.

2. Weights

Each input has a weight, which is a number that tells us how important that input is. If sweetness is more important than thickness, its weight should be higher.

For example:

• weight1 = 0.6
• weight2 = 0.2

We multiply each input by its weight.

3. Summation (Linear Combination)

Now we combine all the weighted inputs:

z = (input1 × weight1) + (input2 × weight2) + bias

Let’s say bias = 1.0, then:

z = (8 × 0.6) + (5 × 0.2) + 1.0 = 4.8 + 1.0 + 1.0 = 6.8

This gives us a raw score z.

4. Activation Function

Now we apply a function to this score. Why? Because we want to add non-linearity or decision logic — like: “If z is high, likely yes. If z is low, likely no.”

The most basic activation function is:

output = 1 / (1 + e^−z) (Sigmoid)

This turns the score into a number between 0 and 1.

If z = 6.8, the sigmoid output ≈ 0.998, which means: “Yes, very likely!”

(We’ll approximate this in code without math.exp soon.)

5. Output

That number (e.g., 0.998) is the final output. It tells us:

• How confident the model is that the answer is “Yes”

6. Learning (Error + Adjustment)

We compare this prediction with the actual answer (called the “label”). If predicted value ≠ actual label, we calculate the error. Then we update the weights to reduce this error:

New weight = Old weight − (Learning rate × Error × Input)

This is how the model learns from mistakes.

7. What Are Input Layer Weights?

In a neural network, input layer weights are numbers that decide how much importance to give to each input feature when calculating the output. Think of them like “volume knobs” that control how loudly each input “speaks” to the next layer.

Example:

Imagine you’re trying to predict if someone likes mango juice based on:

• sweetness = 8
• thickness = 5

We might start with:

• weight_sweetness = 0.6
• weight_thickness = 0.2

Then calculate:

z = (8 × 0.6) + (5 × 0.2) = 4.8 + 1.0 = 5.8

What Is the Basis for These Weights?

1. Random Initialization

At the very beginning, weights are not known. We randomly assign small values (like between -1.0 and 1.0). This is like saying:

“Let’s guess how important each input might be, and learn from experience.”

This randomness:

• Breaks symmetry (neurons don’t all learn the same thing)
• Starts learning process with diverse possibilities

2. Updated Through Learning (Backpropagation)

Once the model makes a prediction and compares it to the actual result, it computes an error. Then it adjusts the weights to reduce that error. This update uses a process called gradient descent:

“If this weight caused too much error, reduce it. If it helped, increase it.”

Weight update formula (simplified):

new weight = old weight + learning rate × error × input

3. Why Weights Matter

Weights capture learning. Over time:

• Higher weights → input is very important
• Near-zero weights → input is not so important
• Negative weights → input has inverse relationship with output

Basic Math Behind It