How to Fix Nginx "stat() failed (13: Permission denied)" & 403 Forbidden

When serving static assets, web applications, or uploaded media through Nginx, requests may suddenly fail with an HTTP 403 Forbidden error. Inspecting /var/log/nginx/error.log reveals one of the following emergency messages:

[error] 1402#1402: *1 stat() "/var/www/html/app/index.html" failed (13: Permission denied), client: 192.0.2.1, server: example.com, request: "GET / HTTP/1.1", host: "example.com"

or:

[error] 1402#1402: *1 open() "/var/www/html/app/index.html" failed (13: Permission denied), client: 192.0.2.1, server: example.com, request: "GET /index.html HTTP/1.1", host: "example.com"

In the Linux kernel, error code 13 corresponds to EACCES (Permission Denied). This failure does not simply mean the target file lacks read permissions—it indicates that the Nginx worker process failed discretionary access controls (DAC) or mandatory access controls (MAC) at some point along the filesystem path.

In this guide, we break down the Linux VFS traversal model, explain how worker process privilege dropping works, and provide exact troubleshooting steps for POSIX permissions, ACLs, SELinux, AppArmor, and Docker user namespaces.

[!TIP] Proactive Gateway & Status Code Audits If your web servers are intermittently returning 403 Forbidden or dropping client requests due to filesystem permission collisions, inspect your HTTP response codes and headers in real time with our free HTTP Status Code Checker, HTTP Header Checker, and Ping Test Tool.

The Multi-Layer Linux Access Model

When an HTTP client requests a static file, Nginx translates the URI into a local filesystem path and calls stat() or open(). The Linux Virtual File System (VFS) evaluates permissions across several independent security boundaries before granting access:

+-----------------------------------------------------------------------------+
|                               HTTP GET Request                              |
+-----------------------------------------------------------------------------+
                                      |
                                      v
+-----------------------------------------------------------------------------+
|               Nginx Master Process (Runs as root: UID 0)                    |
|          Spawns unprivileged Worker Process (Runs as www-data/nginx)        |
+-----------------------------------------------------------------------------+
                                      |
                                      v
+-----------------------------------------------------------------------------+
|                           Linux VFS Permission Checks                       |
|   1. POSIX Traversal: Requires '+x' (0755) on EVERY parent directory        |
|   2. POSIX Read: Requires '+r' (0644) on the target file itself             |
|   3. POSIX Access Control Lists (ACLs) via getfacl                          |
|   4. Mandatory Access Control (MAC): SELinux Contexts (httpd_sys_content_t) |
|   5. AppArmor Security Profiles (/etc/apparmor.d/usr.sbin.nginx)            |
|   6. Docker Container Namespace UID/GID mappings                            |
+-----------------------------------------------------------------------------+
                                      |
                                      v
+-----------------------------------------------------------------------------+
|                         Target File / Document Root                         |
+-----------------------------------------------------------------------------+

A failure at any single layer halts execution and returns EACCES (13: Permission denied).

1. POSIX Directory Traversal: Why +x on Every Parent Matters

The most common misconception in Linux systems administration is that granting 0644 to a file allows any process to read it.

On Linux, directory permissions operate under specific rules:

• Read (r / 4): Allows listing directory contents (e.g., ls).
• Write (w / 2): Allows creating, deleting, or renaming files within the directory.
• Execute (x / 1): Allows traversing through the directory to access downstream files or subdirectories.

The Parent Path Traversal Chain

If Nginx attempts to read /var/www/html/app/index.html, the kernel evaluates traversal sequentially from the root /:

$$\text{Access Granted} \iff \forall d \in {/, /\text{var}, /\text{var}/\text{www}, /\text{var}/\text{www}/\text{html}, /\text{var}/\text{www}/\text{html}/\text{app}}, \text{ExecuteBit}(d) = 1$$

If even one parent directory in this chain (such as /var/www) has permissions 0700 (drwx------ root root), the Nginx worker process cannot cross that boundary, triggering stat() failed (13: Permission denied) even if index.html has permissions 0777.

2. Worker Process Privilege Dropping

Nginx starts its master process as root (UID 0) to bind to privileged network ports (80 and 443). Once bound, Nginx immediately drops privileges and spawns worker processes under an unprivileged user account defined in nginx.conf:

# Ubuntu / Debian default
user www-data;

# RHEL / CentOS / Rocky Linux / Alpine default
user nginx;

Inspect the Running Worker UID

Do not assume the user defined in nginx.conf matches the active runtime process. Verify the effective user and group of all active Nginx processes:

ps -eo user,group,pid,ppid,cmd | grep '[n]ginx'

Typical production output:

USER     GROUP      PID   PPID CMD
root     root      1401      1 nginx: master process /usr/sbin/nginx
www-data www-data  1402   1401 nginx: worker process
www-data www-data  1403   1401 nginx: worker process

3. Step-by-Step Diagnostic Sequence

Follow this diagnostic workflow to pinpoint the exact failure layer without resorting to dangerous chmod 777 commands.

Step 1: Trace the Entire Directory Path with namei

The namei utility lists every parent component in a path along with its owner, group, and permission bits:

namei -l /var/www/html/app/index.html

Example diagnostic output revealing a permission choke point:

f: /var/www/html/app/index.html
drwxr-xr-x root     root     /
drwxr-xr-x root     root     var
drwx------ root     root     www         <-- Traversal blocked here for www-data!
drwxr-xr-x www-data www-data html
drwxr-xr-x www-data www-data app
-rw-r--r-- www-data www-data index.html

In the output above, /var/www has drwx------ (0700), preventing www-data from traversing into /var/www/html.

Step 2: Simulate Worker Access via sudo -u

Never test file readability as root. Test file access using the exact unprivileged worker account:

# Ubuntu / Debian
sudo -u www-data stat /var/www/html/app/index.html
sudo -u www-data test -r /var/www/html/app/index.html && echo "READ: OK" || echo "READ: FAILED"

# RHEL / Rocky Linux / CentOS
sudo -u nginx stat /usr/share/nginx/html/index.html
sudo -u nginx test -r /usr/share/nginx/html/index.html && echo "READ: OK" || echo "READ: FAILED"

If stat or test -r returns an error, the problem is local POSIX permissions or ACLs.

Step 3: Inspect POSIX Access Control Lists (ACLs)

If standard ls -l shows drwxr-xr-x. with a trailing + sign (e.g., drwxr-xr-x+), extended ACLs are active and may override standard mode bits:

getfacl /var/www/html/app/index.html

To grant explicit traversal and read access to www-data without altering global ownership:

sudo setfacl -m u:www-data:rx /var/www/html/app
sudo setfacl -m u:www-data:r /var/www/html/app/index.html

4. Production Remediation for POSIX Permissions

For standard static websites and web applications, apply proper least-privilege ownership and permission masks:

# 1. Set correct ownership (adjust 'www-data' to 'nginx' on RHEL/CentOS)
sudo chown -R www-data:www-data /var/www/html

# 2. Set directory traversal permissions (0755: rwxr-xr-x)
sudo find /var/www/html -type d -exec chmod 755 {} \;

# 3. Set file read permissions (0644: rw-r--r--)
sudo find /var/www/html -type f -exec chmod 644 {} \;

# 4. Verify traversal on all parent directories
sudo chmod 755 /var /var/www

5. SELinux Remediation (RHEL, CentOS, Rocky Linux, Fedora)

On SELinux-enabled distributions, standard POSIX permissions are insufficient. Even if a file is 0777 and owned by nginx, SELinux will block access if the security context does not match httpd_sys_content_t.

1. Check SELinux Status and Contexts

# Check if SELinux is enforcing
getenforce

# Inspect file security labels
ls -Z /var/www/html/app/index.html

If the context shows unconfined_u:object_r:user_home_t:s0 or default_t, Nginx will be denied access.

2. Restore Standard SELinux Contexts

If your web root is in /var/www/ or /usr/share/nginx/html/, restore the default system contexts:

sudo restorecon -Rv /var/www/html

3. Configure Persistent Contexts for Custom Web Roots

If your application resides in a custom directory (e.g., /srv/myapp/public):

# Define persistent context rule in SELinux policy database
sudo semanage fcontext -a -t httpd_sys_content_t "/srv/myapp/public(/.*)?"

# Apply the policy recursively
sudo restorecon -Rv /srv/myapp/public

4. Inspect SELinux Audit Logs

If denials persist, query the audit logs using ausearch and audit2why:

sudo ausearch -m avc -ts recent | audit2why

6. Docker Container Volume & UID Mapping Issues

In containerized deployments, permission errors frequently stem from numeric UID mismatches across host bind mounts.

+---------------------------------------+       +---------------------------------------+
|              Host System              |       |           Docker Container            |
+---------------------------------------+       +---------------------------------------+
|  File: ./public/index.html            | ----> |  Mount: /usr/share/nginx/html         |
|  Owner: deploy (UID 1000, GID 1000)   |       |  Worker Process: nginx (UID 101)      |
|  Mode: 0600 (rw-------)               |       |  Result: 403 Forbidden (EACCES)       |
+---------------------------------------+       +---------------------------------------+

1. Identify Numeric UIDs

Inspect the container worker process UID:

docker exec nginx-container sh -c 'ps -eo user,uid,gid,cmd | grep nginx'

Inspect mounted file ownership numerically from inside the container:

docker exec nginx-container ls -ln /usr/share/nginx/html

2. Solutions for Docker Environments

• Option A (Relax Mode Bits on Host): Ensure host files have 0644 and directories have 0755.
• Option B (Docker SELinux Flag): On SELinux hosts, append :ro,z (shared) or :ro,Z (private) to bind mounts in docker-compose.yml:

  volumes:
    - ./public:/usr/share/nginx/html:ro,z
  

• Option C (Match Container User): Specify the matching host UID in docker-compose.yml:

  services:
    nginx:
      image: nginx:alpine
      user: "1000:1000"
  

Permission & Security Context Matrix

Edge Cases: Serving from /home and Immutable Attributes

1. Serving from User Home Directories (/home/user/public_html)

User home directories are typically created with 0700 (drwx------) for security. To allow Nginx to traverse /home/user without making the user's entire home directory world-readable, use an explicit POSIX ACL:

sudo setfacl -m u:www-data:x /home/user
sudo setfacl -R -m u:www-data:rx /home/user/public_html

On SELinux systems, enable the httpd_read_user_content boolean:

sudo setsebool -P httpd_read_user_content 1

2. Immutable Filesystem Attributes

If files cannot be read or modified despite correct permissions, check for the Linux immutable attribute (+i):

lsattr /var/www/html/app/index.html

If the i attribute is set (----i---------), remove it using chattr:

sudo chattr -i /var/www/html/app/index.html

Production Synthetic Monitoring with Pingzo

Filesystem permission regressions frequently occur during CI/CD deployments, automated cache purges, or server provisioning scripts. Synthetic monitoring ensures that when a deployment accidentally applies incorrect permission masks or breaks SELinux labels, your team is notified before users experience widespread 403 Forbidden outages.

+-----------------------------------------------------------------------------+
|                         Pingzo Global Edge Network                          |
+-----------------------------------------------------------------------------+
               |                                   |
    [ Multi-Region HTTP Status Probes ]    [ Header & Payload Integrity Audits ]
               |                                   |
               v                                   v
+-----------------------------------------------------------------------------+
|               Your Web Infrastructure & Application Fleet                   |
|               (Immediate Alerting on 403, 500, 502, or 404 Drops)           |
+-----------------------------------------------------------------------------+
               |
               v
+-----------------------------------------------------------------------------+
|              Instant Multi-Channel Alerts: WhatsApp, Slack, & Telegram      |
+-----------------------------------------------------------------------------+

Pingzo Monitoring Plans

Ensure your web servers and static assets remain highly available. Start synthetic monitoring in seconds with Pingzo.