Web Architecture on Noman Chowdhury Munna | Blog

Web Systems Architecture Foundations: From Sockets to Application Servers

Thu, 20 Aug 2026 00:00:00 +0000

Welcome back, everyone. Take your seats.

Today we are kicking off our deep-dive series on modern backend engineering and infrastructure architecture. Before we talk about Kubernetes, load balancers, or fancy service meshes, we must build an unshakeable understanding of fundamental web mechanics.

If you don’t understand what actually happens at the OS and network level when a user clicks a button on a web page, you will spend your entire career Cargo-culting configuration files and scratching your head during production outages.

So let me strip away the magic. Grab your notebook, and let me examine the foundations of web systems architecture.

1. The Client-Server Model & Life of a Request

Let’s start with the baseline paradigm: Client-Server Architecture.

In software engineering, client-server is a distributed application structure that partitions tasks or workloads between service providers (servers) and service requesters (clients).

1
2
3
4
5
6


 +------------------+ +-------------------+
 | | HTTP GET Request | |
 | Client Browser | -------------------> | Web Server |
 | (e.g., Chrome) | <------------------- | (e.g., Nginx) |
 | | HTTP 200 Response | |
 +------------------+ +-------------------+

The Client: Usually a web browser (Chrome, Firefox), a mobile application, or a CLI tool like curl. It initiates communication by asking for a resource or sending data.
The Server: A machine (or software process) that waits passively for incoming network connections, processes requests, and returns responses.

What Happens When You Type `https://example.com` into a Browser?

This is a classic software engineering interview question, but more importantly, it tests your holistic understanding of networking. When a user hits Enter, a precise chain of events triggers:

1

[User Types URL] -> [1. DNS Lookup] -> [2. TCP Handshake] -> [3. TLS Handshake] -> [4. HTTP Request] -> [5. Server Processing] -> [6. HTTP Response] -> [7. DOM Render]

URL Parsing & Protocol Identification: The browser parses https://example.com, recognizing the protocol (https://), domain name (example.com), and implicit default port (443).
DNS Resolution: The computer needs an IP address (e.g., 93.184.216.34). The browser checks its local cache, OS cache, router cache, and finally queries a Recursive DNS Resolver.
TCP Connection (3-Way Handshake): The browser opens a socket to port 443 at IP 93.184.216.34. The OS executes the TCP handshake: SYN $\rightarrow$ SYN-ACK $\rightarrow$ ACK.
TLS Handshake: Because the protocol is HTTPS, the client and server negotiate encryption keys, exchange cryptographic certificates, and verify host identities.
HTTP Request Transmission: The browser sends an HTTP request payload over the encrypted TCP connection (e.g., GET / HTTP/1.1\r\nHost: example.com...).
Server Processing & Response: The server receives the bytes via socket buffers, processes the request (executing business logic or fetching static files), and writes an HTTP response back (HTTP/1.1 200 OK).
Client Rendering: The browser receives HTML, parses the document tree (DOM), requests secondary assets (CSS, JS, images), executes JS, and renders the UI.

2. Dissecting URLs, Hostnames, IPs, Ports, and Sockets

Students often mix up these terms. Let’s make the distinctions crystal clear.

Term	What it is	Real-world Physical Analogy
IP Address	The numerical network address of a machine on the internet (e.g., `192.168.1.1` or `93.184.216.34`).	The street address of an apartment building.
Port	A 16-bit integer (0–65535) identifying a specific process on that machine.	The apartment number inside the building.
Hostname	A human-readable name that maps to an IP address (e.g., `api.example.com`).	The name of the building (“Empire State Building”).
URL	Uniform Resource Locator: host + port + path + query parameters + protocol (`https://example.com:8080/users?id=5`).	The complete mailing instructions to deliver a letter to a specific person.
Socket	An OS file descriptor representing an endpoint bound to an IP + Port pair (`192.168.1.1:443`).	The actual telephone plug in the wall connected to a phone line.

What does it mean for a server to “listen” on a port?

When you run a backend application (say, Python’s FastAPI or Node.js), the program makes system calls to the operating system kernel:

socket(): Requests a network socket file descriptor.
bind(IP, Port): Tells the kernel “associate this socket with IP 0.0.0.0 and Port 8000”.
listen(): Tells the kernel “accept incoming TCP connections on this socket and queue them up”.

When a server is “listening,” the OS kernel handles the raw TCP packets in the background. When a client connects, the OS places the established socket in an accept queue for your application code to read.

Exposing a Port to the Internet

When you expose a port, you configure your network interface and firewall (e.g., AWS Security Group, ufw, or router port forwarding) to allow external IP addresses on the public internet to reach that listening socket port. If a port is bound to 127.0.0.1 (localhost), it is strictly private to the machine. If it is bound to 0.0.0.0 (all interfaces) and the firewall allows incoming traffic, it is publicly exposed.

3. Protocols: HTTP vs. HTTPS & DNS

HTTP (Hypertext Transfer Protocol)

HTTP is an application-layer request-response protocol running over TCP. It is completely plaintext. Anyone eavesdropping on the network path (Wi-Fi router, ISP, transit provider) can inspect, alter, or inject data into your traffic.

HTTPS (HTTP Secure)

HTTPS is standard HTTP wrapped inside TLS (Transport Layer Security) encryption.

1
2
3
4
5
6
7
8
9


+------------------------------------+
| Application Layer: HTTP |
+------------------------------------+
| Security Layer: TLS (Encryption) |
+------------------------------------+
| Transport Layer: TCP |
+------------------------------------+
| Network Layer: IP |
+------------------------------------+

HTTPS guarantees three fundamental security principles:

Confidentiality: Payload is encrypted; eavesdroppers see only random bytes.
Integrity: Data cannot be tampered with in transit without detection.
Authentication: Digital certificates issued by Certificate Authorities (CAs) prove that the server actually owns the requested domain.

DNS (Domain Name System)

DNS is the internet’s phonebook. It translates human-friendly hostnames (example.com) into machine-routable IP addresses (93.184.216.34). It operates primarily over UDP port 53 for quick request-response lookups.

4. The Server Hierarchy: Web Server vs. Application Server vs. Backend

One of the biggest sources of confusion for junior engineers is distinguishing between Web Servers, Application Servers, and Backend Servers.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


[ Public Internet ]
 |
 v (Port 80/443 - Public)
+-------------------------------------------------+
| WEB SERVER (e.g., Nginx, Caddy, Apache) |
| - TLS Termination |
| - Serves static assets (HTML/CSS/Images) |
| - Rate limiting & request filtering |
+-------------------------------------------------+
 |
 v (Port 8000 - Internal Socket / Loopback)
+-------------------------------------------------+
| APPLICATION SERVER (e.g., Gunicorn, Uvicorn) |
| - Runs dynamic code (Python, Node, Java, Go) |
| - Manages WSGI/ASGI worker process pools |
+-------------------------------------------------+
 |
 v (Internal Database Protocol)
+-------------------------------------------------+
| BACKEND DATABASE (e.g., PostgreSQL, Redis) |
| - Data persistence & business state |
+-------------------------------------------------+

1. Web Server

A software application dedicated to handling HTTP protocols, serving static files directly from disk, managing TLS certificates, and forwarding dynamic requests. Examples: Nginx, Apache HTTP Server, Caddy.

2. Application Server

A runtime engine designed to execute dynamic programming code and business logic (Python, Java, Ruby, JS). It translates incoming raw HTTP payloads into programmatic objects (e.g., Python dict or Express req/res objects). Examples: Gunicorn / Uvicorn (Python), Puma (Ruby), Tomcat (Java), Node.js runtime.

3. Backend Server

A broader architectural term encompassing everything behind the user-facing tier—including application servers, background worker queues (Celery, BullMQ), microservices, and databases.

5. Why Can’t You Expose Your Application Server Directly to the Internet?

Students frequently ask me: “Professor, my FastAPI/Node.js app has a built-in server. Why can’t I just bind it to port 80/443 on my VPS and expose it directly?”

Technically, you can, but in production, this is an architectural sin. Here is why:

Slow Client Attacks (Slowloris): Application servers (like Gunicorn or Node.js event loops) are optimized for complex business logic, not holding open thousands of idle connection threads. An attacker sending 1 byte every 10 seconds can quickly exhaust all worker threads, taking your app offline.
Poor Static Asset Performance: Application servers execute code to serve files. Reading a static logo.png off disk via Python or JS is 10x to 100x slower and consumes vastly more CPU/RAM than Nginx serving it using OS zero-copy sendfile() kernel calls.
Security & Privilege Escalation: Binding to ports below 1024 (like port 80 or 443) requires root privileges on Linux. Running your application code (FastAPI/Node) as root means any Remote Code Execution (RCE) vulnerability gives the attacker full control of your server.
Lack of Robust TLS Management: Application runtime TLS stacks are rarely as optimized, battle-tested, or easily automated (via Certbot) as dedicated web servers like Nginx.
No Zero-Downtime Reloads: Modern web servers can reload their configuration and worker processes without dropping a single active TCP connection. Application runtimes typically require restarting the whole process.

Classroom Recap & Concept Check

Let’s review what we learned today:

Client-Server: Clients request, servers listen on TCP sockets.
Port vs IP: IP is the machine’s address; Port is the process doorway.
DNS & Sockets: DNS translates hostnames to IPs; Sockets are the OS file handles for networking.
Web vs App Server: Web servers (Nginx) handle HTTP/TLS/Static files blazingly fast; App servers (Uvicorn/Gunicorn) run business logic.
Exposure: Never expose app servers directly; place a dedicated web server or reverse proxy in front.

In our next lecture, we will take this foundation and explore Reverse Proxies & Nginx Implementation in depth. Class dismissed!

Demystifying Reverse Proxies & Nginx: Architecture, Configuration & Production Pitfalls

Fri, 21 Aug 2026 00:00:00 +0000

Welcome back to Systems Architecture. Today, we are taking our foundational knowledge of web servers and diving headfirst into one of the most vital components of modern infrastructure: The Reverse Proxy.

If you look at the infrastructure of Google, Netflix, or any modern startup, you will find reverse proxies standing at the edge of their networks. Today, we’re going to demystify what a reverse proxy is, why it’s called “reverse,” how Nginx implements it, and how to troubleshoot it when things inevitably explode in production.

1. Proxies 101: Forward Proxy vs. Reverse Proxy

Let’s start by clarifying the word proxy. A proxy is simply an intermediary process that acts on behalf of another entity. But which entity is it acting on behalf of?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


FORWARD PROXY (Protects/Hides Clients)
[ Client A ] --\
[ Client B ] ---> [ Forward Proxy ] ---------> [ Public Internet / Web Server ]
[ Client C ] --/ (e.g., Corporate VPN) (Server sees ONLY Proxy IP)

-------------------------------------------------------------------------------

REVERSE PROXY (Protects/Hides Servers)
[ Public Client ] ---> [ Reverse Proxy ] ---\---> [ Backend Server 1 (FastAPI) ]
(Client sees ONLY (e.g., Nginx) |---> [ Backend Server 2 (Node) ]
 Reverse Proxy IP) \---> [ Static File Storage ]

Forward Proxy

A Forward Proxy sits in front of clients. When you are on a corporate office network or using a VPN, your laptop sends traffic to the forward proxy first. The proxy forwards the request out to the internet, receives the response, and hands it back to your laptop.

Goal: Protects/hides clients, enforces content filtering (blocking social media at school), and circumvents geo-blocks.
Identity Awareness: The client knows the proxy exists (it’s configured in browser/network settings). The destination server on the internet has no idea who the real client is—it only sees the proxy’s IP address.

Reverse Proxy

A Reverse Proxy sits in front of backend servers. It intercepts all incoming requests from the public internet and routes them to appropriate internal backend application instances.

Why is it called “Reverse”?: Because it acts on behalf of the servers rather than the clients. It sits at the destination network edge rather than the source network edge.
Identity Awareness: The client believes it is talking directly to the destination server (e.g., api.example.com resolves to Nginx’s IP). The client usually has no idea that a reverse proxy is secretly sitting in front of 10 internal microservices.

Proxying vs. Redirecting

Do not confuse proxying with HTTP redirecting (301 Moved Permanently or 302 Found):

Redirecting: Server responds to the browser with a 30x header and a Location: https://new-site.com URL. The browser’s URL bar changes, and the client opens a brand-new connection to the new URL.
Proxying: The reverse proxy receives the request, internally fetches data from an upstream backend server, and returns it to the client. The browser’s URL bar does NOT change.

2. Core Problems Solved by a Reverse Proxy

Why do we put Nginx or HAProxy in front of our applications?

Infrastructure Obfuscation & Security: Hides internal backend IP addresses, network topology, and private ports. Attackers cannot directly port-scan or target internal application servers.
Host & Path Routing: Exposes multiple applications (e.g., Next.js frontend, FastAPI backend, Go microservice) under a single public IP or single domain name (example.com/api vs example.com/).
TLS/SSL Termination: Offloads CPU-heavy RSA/ECC cryptographic handshakes to Nginx, keeping application server code clean and fast.
Compression & Caching: Compresses HTTP responses (gzip/brotli) and caches static assets in memory/disk before requests ever touch application servers.
Rate Limiting & Basic Auth: Intercepts abusive traffic at the network perimeter.

3. TLS Mechanics: Termination vs. Passthrough

When handling HTTPS, a reverse proxy can operate in two distinct modes:

1
2
3
4
5
6
7


TLS TERMINATION
[ Client ] === (Encrypted HTTPS) ===> [ Nginx Proxy ] --- (Plain HTTP) ---> [ Backend App ]
 (Decrypts here)

TLS PASSTHROUGH (SSL Passthrough)
[ Client ] === (Encrypted HTTPS) ===> [ Nginx L4 Proxy ] === (Encrypted HTTPS) ===> [ Backend App ]
 (Passes raw TCP) (Decrypts here)

TLS Termination

The reverse proxy holds the SSL/TLS private key and certificate. It decrypts incoming HTTPS traffic from the client, inspects the HTTP headers/body, and forwards plaintext HTTP (or re-encrypted HTTP) over a secure private network to backend servers.

Pros: Centralized certificate management (via Certbot/Let’s Encrypt), enables HTTP header inspection/modification, response caching, and rate limiting.

TLS Passthrough

The proxy operates at Layer 4 (TCP) without decrypting traffic. It routes raw encrypted TLS bytes directly to the backend server, which holds the private key.

Pros: Zero-trust security (the proxy cannot read sensitive payload data).
Cons: Proxy cannot inspect HTTP headers, route by path (/api), or inject X-Forwarded-For headers.

4. HTTP Headers & The `X-Forwarded-For` Security Trap

Because a reverse proxy creates a new TCP connection to the backend server, the backend server’s socket naturally sees 127.0.0.1 or the proxy’s private IP as the client address!

To fix this, the proxy injects standardization headers into the request forwarded upstream:

X-Forwarded-For: A comma-separated list of client and proxy IPs (X-Forwarded-For: <client-ip>, <proxy1-ip>).
X-Forwarded-Proto: Indicates the original protocol (http or https).
X-Forwarded-Host: Indicates the original Host header sent by the client.

1
2
3
4
5
6
7


Incoming Request: Client IP 203.0.113.19 -> Nginx (10.0.0.1) -> FastAPI (10.0.0.2)

Nginx injects headers upstream:
 Host: api.example.com
 X-Real-IP: 203.0.113.19
 X-Forwarded-For: 203.0.113.19
 X-Forwarded-Proto: https

⚠️ The Dangerous Vulnerability: Trusting `X-Forwarded-For` Blindly

Suppose your application checks client IP for rate-limiting or admin access using request.headers.get("X-Forwarded-For").

If an attacker on the internet sends a request with a fake header: X-Forwarded-For: 127.0.0.1

If Nginx simply appends to the incoming header, the header reaching FastAPI becomes: X-Forwarded-For: 127.0.0.1, 203.0.113.19

If your backend code simply reads the first IP in the list, the attacker just bypassed your IP restrictions!

Mitigation: Always configure Nginx to clear or explicitly override X-Forwarded-For with $remote_addr, or configure your backend framework’s proxy middleware to trust only specific proxy IP ranges.

5. Nginx Hands-On Implementation

Let’s look at real Nginx configuration files.

Anatomy of Nginx Location Block Matching

When a request arrives, Nginx selects a location block using this strict precedence order:

location = /exact: Exact string match (Highest priority).
location ^~ /images/: Preferential prefix match (Stops regex searching).
location ~ \.(png|jpg)$: Case-sensitive Regex match.
location ~* \.(png|jpg)$: Case-insensitive Regex match.
location /prefix: Standard prefix match (Longest prefix wins).

`proxy_pass` Trailing Slash Gotcha!

Pay extremely close attention to the trailing slash in proxy_pass—this trips up developers every single day:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


# CASE A: Trailing slash present on proxy_pass
location /api/ {
 proxy_pass http://localhost:8000/;
}
# Request to /api/users/5 --> forwards to http://localhost:8000/users/5 (Strips /api)

# CASE B: NO Trailing slash on proxy_pass
location /api/ {
 proxy_pass http://localhost:8000;
}
# Request to /api/users/5 --> forwards to http://localhost:8000/api/users/5 (Preserves /api)

Production Config: Next.js Frontend + FastAPI Backend + WebSockets

Here is a full production-ready Nginx configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


server {
 listen 80;
 server_name example.com www.example.com;
 return 301 https://$host$request_uri; # HTTP -> HTTPS Redirect
}

server {
 listen 443 ssl http2;
 server_name example.com www.example.com;

 ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
 ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

 # 1. Next.js Frontend
 location / {
 proxy_pass http://127.0.0.1:3000;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto $scheme;
 }

 # 2. FastAPI REST Backend
 location /api/ {
 proxy_pass http://127.0.0.1:8000/; # Strips /api/ prefix
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto $scheme;
 client_max_body_size 10M; # Handle large uploads
 }

 # 3. WebSocket Connection Upgrade
 location /ws/ {
 proxy_pass http://127.0.0.1:8000/ws/;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "Upgrade";
 proxy_read_timeout 86400s; # Prevent WS idle timeout drop
 }
}

6. Troubleshooting Nginx Errors in Production

When Nginx throws an error, don’t panic. Check your /var/log/nginx/error.log and use this diagnostic guide:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


 INCOMING REQUEST
 |
 v
 [ Is Nginx Running? ] --- No ---> Connection Refused
 |
 Yes
 |
 [ Can Nginx reach Backend? ] --- No ---> 502 Bad Gateway
 |
 Yes
 |
 [ Does Backend respond within timeout? ] --- No ---> 504 Gateway Timeout
 |
 Yes
 |
 200 OK / 400 / 500

1. `502 Bad Gateway`

Meaning: Nginx contacted the upstream backend socket, but received an invalid response, connection reset, or connection refused.
Common Causes:
- Backend server (FastAPI/Node) crashed or isn’t running.
- Backend is listening on 127.0.0.1:8000, but Nginx is configured to proxy to 10.0.0.5:8000.
- Permission denied on UNIX domain socket (/tmp/uvicorn.sock).

2. `504 Gateway Timeout`

Meaning: Nginx successfully connected to the backend server, but the backend server failed to send a response within Nginx’s configured proxy_read_timeout window.
Common Causes:
- Long-running SQL query or synchronous blocking operation in python code.
- Downstream database deadlock.
- Increase timeout in Nginx if expected: proxy_read_timeout 120s;.

7. What Happens If Nginx Itself Goes Down? (High Availability)

If Nginx is your single entry point and Nginx crashes, your whole application goes down. How do we prevent this single point of failure (SPOF)?

We use High Availability (HA) with Keepalived & VRRP (Virtual Router Redundancy Protocol):

1
2
3
4
5
6
7
8


 [ Virtual IP (VIP): 192.168.1.100 ]
 |
 +---------------------+---------------------+
 | (Active) | (Standby)
 +------------------+ +------------------+
 | Nginx Server #1 | | Nginx Server #2 |
 | (Master) | | (Backup) |
 +------------------+ +------------------+

Two Nginx servers share a single Virtual IP (VIP).
keepalived sends heartbeat packets over the local network.
If Nginx Server #1 fails, Server #2 instantly claims the Virtual IP in under 1 second. The client never notices an interruption!

Classroom Summary & Homework

Today we mastered:

Forward proxies shield clients; Reverse proxies shield servers.
TLS termination simplifies backend code and accelerates handshake performance.
Always sanitize X-Forwarded-For to prevent IP spoofing attacks.
Memorize 502 (Backend down/unreachable) vs 504 (Backend took too long).
Use VRRP/Keepalived to eliminate Nginx as a single point of failure.

Next class: Load Balancing Algorithms, Layer 4 vs Layer 7, and Sticky Sessions. Class dismissed!

Load Balancing Mastery: L4 vs L7, Algorithms, Session Persistence & Distributed Scaling

Sat, 22 Aug 2026 00:00:00 +0000

Welcome back to Systems Architecture. In our previous lecture, we placed Nginx in front of a single backend server as a reverse proxy. But what happens when your application explodes in popularity and a single backend machine can no longer keep up with incoming traffic?

Today, we dive into Load Balancing—the art and science of distributing network traffic across a pool of backend servers to achieve high availability, fault tolerance, and horizontal scalability.

1. The Scaling Dilemma: Vertical vs. Horizontal

When a single server reaches 100% CPU or RAM utilization, you have two options to scale:

1
2
3
4
5
6


VERTICAL SCALING (Scale Up) HORIZONTAL SCALING (Scale Out)
+------------------------+ +--------+ +--------+ +--------+
| BIGGER SERVER | | Node 1 | | Node 2 | | Node 3 |
| 128 Cores, 512GB RAM | | 4 Cores| | 4 Cores| | 4 Cores|
+------------------------+ +--------+ +--------+ +--------+
(Single point of failure, expensive) (Requires a Load Balancer)

Dimension	Vertical Scaling (Scale Up)	Horizontal Scaling (Scale Out)
Strategy	Add more CPU, RAM, or NVMe storage to your existing server.	Add more cheap commodity server instances to a pool.
Cost Curve	Non-linear / Exponentially expensive at high specs.	Linear cost curve (pay only for nodes added).
Hardware Limit	You eventually hit motherboard & CPU socket physical ceilings.	Virtually unlimited (scale to thousands of nodes).
Downtime	Upgrading RAM/CPU requires shutting down the machine.	Zero downtime (dynamically add/remove nodes).
Fault Tolerance	Zero. If the big server hardware fails, everything goes down.	High. If Node 1 crashes, Nodes 2 and 3 take the load.

Key Takeaway: Horizontal scaling is the foundation of modern cloud architecture. However, horizontal scaling requires a Load Balancer to sit in front of the pool and decide which server gets which request.

2. Where Does a Load Balancer Sit?

A Load Balancer (LB) sits between the clients (or reverse proxies) and the pool of worker instances:

1
2
3
4
5
6
7
8
9


[ Public Clients ]
 |
 v
[ External Load Balancer ] (e.g., AWS ALB / HAProxy)
 |
 +-------------------+-------------------+
 | | |
 v v v
 [ Backend App #1 ] [ Backend App #2 ] [ Backend App #3 ]

Is a Reverse Proxy the same as a Load Balancer?

Reverse Proxy: Primarily focused on acting on behalf of servers (TLS termination, header rewrite, caching, path routing).
Load Balancer: Primarily focused on traffic distribution across multiple targets.
Can one tool do both?: Absolutely! Tools like Nginx, HAProxy, and Traefik excel as both reverse proxies AND load balancers simultaneously.

3. Layer 4 vs. Layer 7 Load Balancing

Load balancers operate at different layers of the OSI model. Choosing between Layer 4 (L4) and Layer 7 (L7) is one of the most critical design choices in system architecture.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


OSI LAYER 4 (Transport Layer - TCP/UDP)
Client TCP Packet [SRC: 203.0.113.5:1234 -> DST: 1.2.3.4:443]
 |
 v L4 LB rewrites IP header (NAT / IPVS / eBPF)
Server TCP Packet [SRC: 203.0.113.5:1234 -> DST: 10.0.0.12:8000]
(LB does NOT inspect HTTP path, cookies, or payload)

-------------------------------------------------------------------------------

OSI LAYER 7 (Application Layer - HTTP/HTTPS)
Client Request: GET /api/v1/checkout HTTP/1.1 [Cookie: session=xyz]
 |
 v L7 LB decrypts TLS, parses HTTP headers, cookies, URL path
Forwarded to: Checkout Service Pool (10.0.0.12:8000)

Detailed Comparison: L4 vs. L7

Feature	Layer 4 Load Balancer (Transport)	Layer 7 Load Balancer (Application)
OSI Layer	Layer 4 (TCP / UDP).	Layer 7 (HTTP / HTTPS / gRPC / WebSockets).
Payload Visibility	Blind to application data (no HTTP headers, cookies, or URL paths).	Full visibility into HTTP headers, cookies, paths, and JSON bodies.
Performance / Throughput	Extremely fast (Millions of RPS). Minimal CPU overhead (packet routing in kernel).	Moderate (Hundreds of thousands of RPS). Requires CPU to parse HTTP/TLS.
Routing Capability	IP address & TCP/UDP port only.	Path-based (`/api`), Host-based (`api.com`), Header/Cookie routing.
TLS Handling	SSL Passthrough (no decryption).	TLS Termination (decryption & re-encryption).
Examples	AWS NLB, IPVS, HAProxy (TCP mode), F5 BIG-IP.	AWS ALB, Nginx, HAProxy (HTTP mode), Envoy.

4. Load Balancing Algorithms

How does the load balancer choose which backend server gets the next request?

1
2
3
4


1. ROUND-ROBIN 2. LEAST-CONNECTIONS 3. CONSISTENT HASHING
 Request 1 -> S1 Request -> S2 (Only 2 active) Hash(user_id=42) -> S3
 Request 2 -> S2 (S1 has 15, S3 has 8) (Always routes user 42 to S3)
 Request 3 -> S3

1. Round-Robin & Weighted Round-Robin

Standard Round-Robin: Requests are distributed sequentially across servers ($S_1 \rightarrow S_2 \rightarrow S_3 \rightarrow S_1$). Assumes all servers have equal capacity and processing times.
Weighted Round-Robin: Assigns weights to servers based on hardware spec ($S_1$: Weight 3, $S_2$: Weight 1). $S_1$ gets 3 requests for every 1 sent to $S_2$.

2. Least Connections

Directs incoming requests to the server with the lowest number of active open connections. Ideal for long-running requests (e.g., file processing or WebSocket streams) where connection durations vary wildly.

3. IP-Hash & Consistent Hashing

IP-Hash: Hashes the client IP address (hash(Client_IP) % N_servers). Ensures a specific client consistently hits the same backend.
Consistent Hashing: Hashes request keys onto a virtual ring. When adding or removing a server, only $\frac{1}{N}$ of keys need to be remapped, rather than re-shuffling the entire key space. Essential for distributed caching clusters (Memcached/Redis).

5. Health Checking & Failover Mechanics

What happens if Backend Server 2 crashes or suffers a hardware kernel panic?

1
2
3
4


[ Load Balancer ] --- (1. Active Health Probe GET /health) ---> [ Server 1: 200 OK ]
 --- (2. Active Health Probe GET /health) ---> [ Server 2: TIMEOUT ] ❌
 |
 [ LB Marks Server 2 DOWN & Evicts from Pool ] <------+

Active vs. Passive Health Checks

Active Health Check: The load balancer periodically (e.g., every 5s) sends explicit probe requests (GET /health) to each backend. If a server fails 3 consecutive checks (returns 5xx or times out), it is marked Unhealthy and evicted from the pool.
Passive Health Check: The LB monitors actual user traffic. If a backend fails 3 real client requests in a row, the LB temporarily suspends traffic to that instance.

Connection Draining (Graceful Shutdown)

When deploying new code or terminating an auto-scaled instance, you don’t abruptly kill the server. The LB enables Connection Draining:

LB stops sending new incoming requests to the target server.
LB allows existing active requests/connections a grace period (e.g., 30–60 seconds) to complete.
Once active connections drop to zero, the server is safely shut down.

6. Session Persistence: Sticky Sessions vs. Stateless Architecture

If a user logs in on Server 1, their session data is created. What happens when their next request lands on Server 2?

1
2
3
4
5
6
7
8
9


STATEFUL (Sticky Sessions - Anti-Pattern at Scale)
[ Client ] -> [ Load Balancer ] -> (Cookie says: Server 1) -> [ Server 1 (Holds Session in RAM) ]
Problem: If Server 1 dies, user session is lost! Unbalanced load across pool.

-------------------------------------------------------------------------------

STATELESS ARCHITECTURE (Recommended Production Pattern)
[ Client ] -> [ Load Balancer ] -> (Any Server 1, 2, or 3) -> Reads Session from [ Central Redis Cluster ]
Benefit: Any server can fail at any time without logging users out!

Sticky Sessions (Session Persistence)

The load balancer inspects a cookie or IP hash to bind a client to a specific backend server.

Drawbacks: Causes unequal server load (one heavy user can overload one server), breaks autoscaling, and user sessions are wiped if that specific server crashes.

The Stateless Architecture Solution

Store session state outside the application servers in a fast, distributed memory store like Redis or use self-contained JWT (JSON Web Tokens):

Backend app servers hold zero user state in local RAM.
Any request can hit any backend server interchangeably.
Servers can be added or destroyed dynamically without affecting connected users.

7. Global Load Balancing: Anycast, DNS LB & CDNs

At enterprise scale, load balancing isn’t just inside a single datacenter; it spans the planet.

1
2


[ User in Tokyo ] -------> [ Anycast IP / DNS Geo-Routing ] -------> [ Tokyo Datacenter ]
[ User in Frankfurt ] ---> [ Anycast IP / DNS Geo-Routing ] -------> [ Frankfurt Datacenter ]

DNS Round-Robin / Geo-DNS: DNS server returns different IP addresses based on the client’s geographic origin (resolving example.com to Tokyo IP for Asian users).
Anycast BGP Routing: Multiple datacenters across the globe advertise the exact same public IP address via Border Gateway Protocol (BGP). Internet routers automatically route client packets to the topologically closest datacenter!
CDN vs. Load Balancer: A CDN (Cloudflare, Fastly) sits in front of your load balancer. It caches static content (images, JS, CSS) at edge locations worldwide, absorbing up to 90% of traffic before it ever touches your primary load balancer.

Classroom Summary & Key Takeaways

Let’s review the main concepts:

Scaling: Horizontal scaling out-classes vertical scaling in resilience and cost.
L4 vs L7: L4 is blistering fast packet routing; L7 is smart HTTP path/cookie routing.
Algorithms: Use Least Connections for variable workloads, Consistent Hashing for caching.
Stateless Architecture: Never rely on sticky sessions in production—offload state to Redis or use JWTs.
High Availability: Combine active health checks with connection draining for zero-downtime deploys.

Next lecture: API Gateways, Resilience Patterns (Circuit Breakers & Retries), and Distributed Tracing. Class dismissed!