Featured image of post Demystifying Reverse Proxies & Nginx: Architecture, Configuration & Production Pitfalls
Web Architecture DevOps

Demystifying Reverse Proxies & Nginx: Architecture, Configuration & Production Pitfalls

An in-depth university lecture covering forward vs reverse proxies, TLS termination, Nginx location block routing algorithms, WebSocket upgrades, header security (X-Forwarded-For spoofing), and practical debugging for 502/504 gateway errors.

Welcome back to Systems Architecture. Today, we are taking our foundational knowledge of web servers and diving headfirst into one of the most vital components of modern infrastructure: The Reverse Proxy.

If you look at the infrastructure of Google, Netflix, or any modern startup, you will find reverse proxies standing at the edge of their networks. Today, we’re going to demystify what a reverse proxy is, why it’s called “reverse,” how Nginx implements it, and how to troubleshoot it when things inevitably explode in production.

1. Proxies 101: Forward Proxy vs. Reverse Proxy

Let’s start by clarifying the word proxy. A proxy is simply an intermediary process that acts on behalf of another entity. But which entity is it acting on behalf of?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

FORWARD PROXY (Protects/Hides Clients)
[ Client A ] --\
[ Client B ] ---> [ Forward Proxy ] ---------> [ Public Internet / Web Server ]
[ Client C ] --/  (e.g., Corporate VPN)        (Server sees ONLY Proxy IP)

-------------------------------------------------------------------------------

REVERSE PROXY (Protects/Hides Servers)
[ Public Client ] ---> [ Reverse Proxy ] ---\---> [ Backend Server 1 (FastAPI) ]
(Client sees ONLY      (e.g., Nginx)        |---> [ Backend Server 2 (Node) ]
 Reverse Proxy IP)                          \---> [ Static File Storage ]

Forward Proxy

A Forward Proxy sits in front of clients. When you are on a corporate office network or using a VPN, your laptop sends traffic to the forward proxy first. The proxy forwards the request out to the internet, receives the response, and hands it back to your laptop.

  • Goal: Protects/hides clients, enforces content filtering (blocking social media at school), and circumvents geo-blocks.
  • Identity Awareness: The client knows the proxy exists (it’s configured in browser/network settings). The destination server on the internet has no idea who the real client is—it only sees the proxy’s IP address.

Reverse Proxy

A Reverse Proxy sits in front of backend servers. It intercepts all incoming requests from the public internet and routes them to appropriate internal backend application instances.

  • Why is it called “Reverse”?: Because it acts on behalf of the servers rather than the clients. It sits at the destination network edge rather than the source network edge.
  • Identity Awareness: The client believes it is talking directly to the destination server (e.g., api.example.com resolves to Nginx’s IP). The client usually has no idea that a reverse proxy is secretly sitting in front of 10 internal microservices.

Proxying vs. Redirecting

Do not confuse proxying with HTTP redirecting (301 Moved Permanently or 302 Found):

  • Redirecting: Server responds to the browser with a 30x header and a Location: https://new-site.com URL. The browser’s URL bar changes, and the client opens a brand-new connection to the new URL.
  • Proxying: The reverse proxy receives the request, internally fetches data from an upstream backend server, and returns it to the client. The browser’s URL bar does NOT change.

2. Core Problems Solved by a Reverse Proxy

Why do we put Nginx or HAProxy in front of our applications?

  1. Infrastructure Obfuscation & Security: Hides internal backend IP addresses, network topology, and private ports. Attackers cannot directly port-scan or target internal application servers.
  2. Host & Path Routing: Exposes multiple applications (e.g., Next.js frontend, FastAPI backend, Go microservice) under a single public IP or single domain name (example.com/api vs example.com/).
  3. TLS/SSL Termination: Offloads CPU-heavy RSA/ECC cryptographic handshakes to Nginx, keeping application server code clean and fast.
  4. Compression & Caching: Compresses HTTP responses (gzip/brotli) and caches static assets in memory/disk before requests ever touch application servers.
  5. Rate Limiting & Basic Auth: Intercepts abusive traffic at the network perimeter.

3. TLS Mechanics: Termination vs. Passthrough

When handling HTTPS, a reverse proxy can operate in two distinct modes:

1
2
3
4
5
6
7

TLS TERMINATION
[ Client ] === (Encrypted HTTPS) ===> [ Nginx Proxy ] --- (Plain HTTP) ---> [ Backend App ]
                                      (Decrypts here)

TLS PASSTHROUGH (SSL Passthrough)
[ Client ] === (Encrypted HTTPS) ===> [ Nginx L4 Proxy ] === (Encrypted HTTPS) ===> [ Backend App ]
                                      (Passes raw TCP)                               (Decrypts here)

TLS Termination

The reverse proxy holds the SSL/TLS private key and certificate. It decrypts incoming HTTPS traffic from the client, inspects the HTTP headers/body, and forwards plaintext HTTP (or re-encrypted HTTP) over a secure private network to backend servers.

  • Pros: Centralized certificate management (via Certbot/Let’s Encrypt), enables HTTP header inspection/modification, response caching, and rate limiting.

TLS Passthrough

The proxy operates at Layer 4 (TCP) without decrypting traffic. It routes raw encrypted TLS bytes directly to the backend server, which holds the private key.

  • Pros: Zero-trust security (the proxy cannot read sensitive payload data).
  • Cons: Proxy cannot inspect HTTP headers, route by path (/api), or inject X-Forwarded-For headers.

4. HTTP Headers & The X-Forwarded-For Security Trap

Because a reverse proxy creates a new TCP connection to the backend server, the backend server’s socket naturally sees 127.0.0.1 or the proxy’s private IP as the client address!

To fix this, the proxy injects standardization headers into the request forwarded upstream:

  • X-Forwarded-For: A comma-separated list of client and proxy IPs (X-Forwarded-For: <client-ip>, <proxy1-ip>).
  • X-Forwarded-Proto: Indicates the original protocol (http or https).
  • X-Forwarded-Host: Indicates the original Host header sent by the client.
1
2
3
4
5
6
7

Incoming Request: Client IP 203.0.113.19 -> Nginx (10.0.0.1) -> FastAPI (10.0.0.2)

Nginx injects headers upstream:
  Host: api.example.com
  X-Real-IP: 203.0.113.19
  X-Forwarded-For: 203.0.113.19
  X-Forwarded-Proto: https

⚠️ The Dangerous Vulnerability: Trusting X-Forwarded-For Blindly

Suppose your application checks client IP for rate-limiting or admin access using request.headers.get("X-Forwarded-For").

If an attacker on the internet sends a request with a fake header: X-Forwarded-For: 127.0.0.1

If Nginx simply appends to the incoming header, the header reaching FastAPI becomes: X-Forwarded-For: 127.0.0.1, 203.0.113.19

If your backend code simply reads the first IP in the list, the attacker just bypassed your IP restrictions!

Mitigation: Always configure Nginx to clear or explicitly override X-Forwarded-For with $remote_addr, or configure your backend framework’s proxy middleware to trust only specific proxy IP ranges.

5. Nginx Hands-On Implementation

Let’s look at real Nginx configuration files.

Anatomy of Nginx Location Block Matching

When a request arrives, Nginx selects a location block using this strict precedence order:

  1. location = /exact: Exact string match (Highest priority).
  2. location ^~ /images/: Preferential prefix match (Stops regex searching).
  3. location ~ \.(png|jpg)$: Case-sensitive Regex match.
  4. location ~* \.(png|jpg)$: Case-insensitive Regex match.
  5. location /prefix: Standard prefix match (Longest prefix wins).

proxy_pass Trailing Slash Gotcha!

Pay extremely close attention to the trailing slash in proxy_pass—this trips up developers every single day:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# CASE A: Trailing slash present on proxy_pass
location /api/ {
    proxy_pass http://localhost:8000/;
}
# Request to /api/users/5  -->  forwards to http://localhost:8000/users/5 (Strips /api)

# CASE B: NO Trailing slash on proxy_pass
location /api/ {
    proxy_pass http://localhost:8000;
}
# Request to /api/users/5  -->  forwards to http://localhost:8000/api/users/5 (Preserves /api)

Production Config: Next.js Frontend + FastAPI Backend + WebSockets

Here is a full production-ready Nginx configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://$host$request_uri; # HTTP -> HTTPS Redirect
}

server {
    listen 443 ssl http2;
    server_name example.com www.example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    # 1. Next.js Frontend
    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # 2. FastAPI REST Backend
    location /api/ {
        proxy_pass http://127.0.0.1:8000/; # Strips /api/ prefix
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        client_max_body_size 10M; # Handle large uploads
    }

    # 3. WebSocket Connection Upgrade
    location /ws/ {
        proxy_pass http://127.0.0.1:8000/ws/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_read_timeout 86400s; # Prevent WS idle timeout drop
    }
}

6. Troubleshooting Nginx Errors in Production

When Nginx throws an error, don’t panic. Check your /var/log/nginx/error.log and use this diagnostic guide:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

                         INCOMING REQUEST
                                |
                                v
                       [ Is Nginx Running? ] --- No ---> Connection Refused
                                |
                                Yes
                                |
                   [ Can Nginx reach Backend? ] --- No ---> 502 Bad Gateway
                                |
                                Yes
                                |
            [ Does Backend respond within timeout? ] --- No ---> 504 Gateway Timeout
                                |
                                Yes
                                |
                         200 OK / 400 / 500

1. 502 Bad Gateway

  • Meaning: Nginx contacted the upstream backend socket, but received an invalid response, connection reset, or connection refused.
  • Common Causes:
    • Backend server (FastAPI/Node) crashed or isn’t running.
    • Backend is listening on 127.0.0.1:8000, but Nginx is configured to proxy to 10.0.0.5:8000.
    • Permission denied on UNIX domain socket (/tmp/uvicorn.sock).

2. 504 Gateway Timeout

  • Meaning: Nginx successfully connected to the backend server, but the backend server failed to send a response within Nginx’s configured proxy_read_timeout window.
  • Common Causes:
    • Long-running SQL query or synchronous blocking operation in python code.
    • Downstream database deadlock.
    • Increase timeout in Nginx if expected: proxy_read_timeout 120s;.

7. What Happens If Nginx Itself Goes Down? (High Availability)

If Nginx is your single entry point and Nginx crashes, your whole application goes down. How do we prevent this single point of failure (SPOF)?

We use High Availability (HA) with Keepalived & VRRP (Virtual Router Redundancy Protocol):

1
2
3
4
5
6
7
8

                  [ Virtual IP (VIP): 192.168.1.100 ]
                                 |
           +---------------------+---------------------+
           | (Active)                                  | (Standby)
  +------------------+                        +------------------+
  | Nginx Server #1  |                        | Nginx Server #2  |
  | (Master)         |                        | (Backup)         |
  +------------------+                        +------------------+
  1. Two Nginx servers share a single Virtual IP (VIP).
  2. keepalived sends heartbeat packets over the local network.
  3. If Nginx Server #1 fails, Server #2 instantly claims the Virtual IP in under 1 second. The client never notices an interruption!

Classroom Summary & Homework

Today we mastered:

  • Forward proxies shield clients; Reverse proxies shield servers.
  • TLS termination simplifies backend code and accelerates handshake performance.
  • Always sanitize X-Forwarded-For to prevent IP spoofing attacks.
  • Memorize 502 (Backend down/unreachable) vs 504 (Backend took too long).
  • Use VRRP/Keepalived to eliminate Nginx as a single point of failure.

Next class: Load Balancing Algorithms, Layer 4 vs Layer 7, and Sticky Sessions. Class dismissed!

comments powered by Disqus
© 2020 - 2026 Noman Chowdhury Munna | Blog
Built with Hugo
Theme Stack designed by Jimmy