❮ Previous Next ❯

Perceptron & Multi-Layer Perceptron (MLP)

  • This lesson explains perceptron and multi-layer perceptron models used in deep learning neural networks.

  • Single Layer Perceptron

    The Perceptron was introduced by Frank Rosenblatt (1958).

    It is a binary classification model based on a single artificial neuron.

    Structure of Single Layer Perceptron

    Z=∑(WiXi)+bZ = \sum (W_i X_i) + bZ=∑(Wi​Xi​)+b Y=f(Z)Y = f(Z)Y=f(Z)

    Where:

    • XXX → Inputs

    • WWW → Weights

    • bbb → Bias

    • fff → Activation function (usually Step function)

    Working

    1. Multiply inputs with weights

    2. Add bias

    3. Apply activation function

    4. Output 0 or 1

    Limitation

    A Single Layer Perceptron can only solve linearly separable problems.

    Can solve:

    • AND

    • OR

    Cannot solve:

    • XOR


    XOR Problem

    The XOR (Exclusive OR) problem is a classic example showing the limitation of single-layer perceptron.

    X1

    X2

    Output

    0

    0

    0

    0

    1

    1

    1

    0

    1

    1

    1

    0

    Why XOR Fails?

    • XOR data is not linearly separable

    • No single straight line can separate classes

    This limitation led to the development of Multi-Layer Perceptron (MLP).


    Multi-Layer Perceptron (MLP)

    MLP is an extension of the perceptron that contains:

    • Input Layer

    • One or more Hidden Layers

    • Output Layer

    It can solve non-linear problems like XOR.


    Hidden Layers

    Hidden layers are the key reason MLP works.

    Role of Hidden Layers:

    • Learn intermediate features

    • Introduce non-linearity

    • Enable solving complex patterns

    The activation functions commonly used:

    • ReLU

    • Sigmoid

    • Tanh

    With hidden layers + non-linear activation → model becomes powerful.


    Feedforward Network

    An MLP is also called a Feedforward Neural Network.

    Why "Feedforward"?

    • Information flows in one direction only

    • Input → Hidden → Output

    • No loops or feedback connections

    Input Layer → Hidden Layer(s) → Output Layer

    It is different from Recurrent Neural Networks (RNN), where feedback loops exist.

    Example: MLP Solving XOR (Python)

Multilayer Perceptron (MLP) Example in Python using TensorFlow

This Python example demonstrates how to build a Multilayer Perceptron (MLP) using TensorFlow and Keras to solve the XOR classification problem. The code creates a neural network with one hidden layer using the ReLU activation function and an output layer with Sigmoid activation. The model is compiled with the Adam optimizer and binary cross-entropy loss, trained for multiple epochs, and evaluated to display the prediction accuracy. 

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
import numpy as np

# XOR Dataset
X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([0,1,1,0])

# Create MLP Model
model = Sequential()
model.add(Dense(8, input_dim=2, activation='relu'))  # Hidden Layer
model.add(Dense(1, activation='sigmoid'))            # Output Layer

# Compile
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Train
model.fit(X, y, epochs=500, verbose=0)

# Evaluate
loss, accuracy = model.evaluate(X, y)
print("Accuracy:", accuracy)

  • Output:

    Output will be close to 1.0 accuracy, showing MLP solves XOR.

❮ Previous Next ❯