Little Bits of Artificial Intelligence

1. Goal:

We’ll simulate a single neuron:

• Takes 1 input
• Applies a weight and bias
• Uses a simple step activation (or sigmoid if you’d like later)

Mathematical Formula

output = activation(weight * input + bias)

We’ll use a simple step function as activation:

def step(x):
return 1 if x >= 0 else 0

Simulation Code

# Step activation function
def step(x):
    return 1 if x >= 0 else 0

# Single neuron simulation
def neuron_output(input_value, weight, bias):
    linear_combination = weight * input_value + bias
    activated_output = step(linear_combination)

    print(f"Input: {input_value}, Weight: {weight}, Bias: {bias}")
    print(f"Weighted sum: {linear_combination}")
    print(f"Activated Output: {activated_output}")
    print("-" * 40)
    return activated_output

# Try different weights and biases
inputs = [0, 1]
weights = [-1, 0.5, 1.5]
biases = [-1, 0, 1]

# Testing all combinations
for w in weights:
    for b in biases:
        for inp in inputs:
            neuron_output(inp, w, b)

What to Observe:

1. When weight is high, even small inputs trigger activation.
2. Negative weights reverse the effect of input.
3. Bias shifts the threshold:
   • Positive bias makes it easier to activate.
   • Negative bias makes it harder to activate.

Example Output (Snippets)

Input: 1, Weight: 1.5, Bias: -1
Weighted sum: 0.5 → Output: 1

Input: 1, Weight: 0.5, Bias: -1
Weighted sum: -0.5 → Output: 0

Input: 0, Weight: 1.5, Bias: 1
Weighted sum: 1 → Output: 1

2. Why is Bias Important in a Neural Network?

Think of bias as the neuron’s “freedom to shift” its decision boundary.

Imagine This Setup Without Bias:

output = activation(weight * input)

In this case:

• The neuron can only pass through the origin (like y = mx, always goes through point (0,0)).
• It means: if input = 0 → output will always be activation(0) = 0 or 0.5 (sigmoid), etc.
• The neuron becomes very limited in what it can learn.

Now Add Bias:

output = activation(weight * input + bias)

Now:

• The neuron can shift the decision line up or down (or left/right in multi-dimensional input).
• It becomes more flexible and powerful.
• Even if the input is zero, the neuron can still fire if the bias is high enough.

Simple Analogy

Think of a light sensor that turns on a fan.

• Weight: How much light is coming in (input influence).
• Bias: A built-in offset; like saying “Don’t turn on until there’s at least this much light”.
• Without bias, it would always react the same way when input is 0, which isn’t very smart.

Mathematical Intuition

For a neuron:

z=w1×x1+w2×x2+…+wn×xn+b

• b shifts the whole decision function.
• Helps fit data that doesn’t center around the origin.

Without it, we’d often need more neurons or deeper layers just to compensate.

Visual Understanding

Imagine we’re classifying emails as spam or not.

• If bias = 0, and weight for “has_money_word” = 0.9
  • We’ll only catch spam if the signal is very strong.

• But adding a bias of -1 helps you catch spam with even slight signals.

In Summary

Bias gives the model freedom. Without it, the model becomes a robot with strict rules; with it, the model becomes smarter and adaptable.

Next – Hidden Layers relevancy in Neural Network