Fixing “Permission Denied” Errors in Nginx Reverse Proxy Setups with SELinux

Running Nginx as a reverse proxy on a system with SELinux enabled can sometimes lead to frustrating errors like:

[crit] connect() to 172.16.5.32:32400 failed (13: Permission denied) while connecting to upstream, client: 172.16.0.1, server: rplex.adminz.in, request: "GET /web/index.html HTTP/2.0", upstream: "http://172.16.5.32:32400/web/index.html", host: "rplex.adminz.in:8443"

If you’re seeing this, SELinux is likely blocking Nginx from making outbound network connections to your upstream servers. Here’s how you can diagnose and fix the issue.

Understanding the Problem

When SELinux is in enforcing mode, it restricts what processes can do—even if you’re running as root. By default, Nginx (and other web servers running under the httpd_t SELinux context) cannot make arbitrary outbound network connections. This is a security feature, but it can block legitimate reverse proxy setups.

Typical log entries look like this:

[crit] connect() to <backend-ip>:<port> failed (13: Permission denied) while connecting to upstream, ...

Diagnosing SELinux Denials

To confirm SELinux is the culprit:

Check your Nginx error logs for “(13: Permission denied)” messages.

Inspect the SELinux audit logs:

sudo grep nginx /var/log/audit/audit.log | grep denied

If you see denials related to name_connect on a TCP socket, SELinux is blocking the connection.

The Solution: Allow Nginx Network Connections

SELinux controls network permissions for web servers using Boolean flags. The most relevant for Nginx reverse proxies is httpd_can_network_connect.

What does httpd_can_network_connect do?

Enabling this Boolean allows Nginx (and other httpd processes) to make outgoing network connections to any port.

This is required for Nginx to proxy requests to other backend servers, especially if they’re not on standard HTTP/HTTPS ports.

How to Enable It

Make the change persistent with:

setsebool -P httpd_can_network_connect true

The -P flag makes the change survive reboots.

After running this command, restart Nginx:

systemctl restart nginx

This should resolve the “permission denied” errors when connecting to upstream servers.

Securing Your Linux System with SELinux: A Step-by-Step Installation Guide

Security-Enhanced Linux (SELinux) is a powerful security mechanism built into the Linux kernel. It provides an additional layer of protection beyond standard user permissions, helping to prevent unauthorized access and malicious activity. If you're serious about Linux security, understanding and using SELinux is a must.

In this guide, we'll walk you through the process of installing and configuring SELinux on your system.

Step 1: Install the SELinux Packages

Open your terminal and run the following command as the root user:

yum install -y selinux-policy-targeted selinux-policy libselinux libselinux-python libselinux-utils policycoreutils policycoreutils-python setroubleshoot setroubleshoot-server setroubleshoot-plugins

Verify that the packages are installed correctly:

rpm -qa | grep selinux
rpm -q policycoreutils
rpm -qa | grep setroubleshoot

Step 2: Prepare for Labeling

Before enabling SELinux, you need to label every file on your system with an SELinux context. To ensure a smooth boot, set SELinux to permissive mode in the /etc/selinux/config file:

SELINUX=permissive SELINUXTYPE=targeted

Step 3: Reboot and Label

Reboot your system. During the boot process, watch for a message indicating that files are being labeled with an SELinux context:

*** Warning -- SELinux targeted policy relabel is required. *** Relabeling could take a very long time, depending on file *** system size and speed of hard drives. ****

Step 4: Check for Denials (Permissive Mode)

While in permissive mode, SELinux doesn't enforce policies but logs any actions that would be denied in enforcing mode. Run the following command to check the logs:

grep "SELinux is preventing" /var/log/messages

If you see no output, it means there were no denied actions.

Step 5: Enable Enforcing Mode

If everything looks good, switch SELinux to enforcing mode in /etc/selinux/config:

SELINUX=enforcing SELINUXTYPE=targeted

Reboot again.

Step 6: Verify SELinux Status

After the reboot, verify that SELinux is running in enforcing mode:

getenforce

You should see the output "Enforcing."

Step 7: Check User Mappings

Finally, run this command to view the mapping between SELinux and Linux users:

semanage login -l

If the mappings aren't correct, follow the instructions in the content you provided to fix them.

The output should look like this:

Login Name      SELinux User         MLS/MCS Range

__default__     unconfined_u         s0-s0:c0.c1023
root            unconfined_u         s0-s0:c0.c1023
system_u        system_u            s0-s0:c0.c1023

Fixing Incorrect User Mappings:

If your output doesn't match the above, run the following commands as the root user. These commands ensure the correct mapping between Linux user accounts and their SELinux roles. If you see warnings about "SELinux-user username is already defined," you can safely ignore them.

semanage user -a -S targeted -P user -R "unconfined_r system_r" -r s0-s0:c0.c1023 unconfined_u
semanage login -m -S targeted -s "unconfined_u" -r s0-s0:c0.c1023 __default__
semanage login -m -S targeted -s "unconfined_u" -r s0-s0:c0.c1023 root
semanage user -a -S targeted -P user -R guest_r guest_u
semanage user -a -S targeted -P user -R xguest_r xguest_u
 

Important Considerations:

• Permissive Mode vs. Enforcing Mode: Start with permissive mode to identify potential issues before switching to enforcing mode, where SELinux actively blocks unauthorized actions.
• Troubleshooting: SELinux denials can be cryptic. To resolve issues, familiarize yourself with SELinux logs and troubleshooting tools like troubleshoot.
• Customization: SELinux policies are highly customizable. Learn how to create custom policies to tailor SELinux to your specific environment.

By following these steps, you can effectively leverage SELinux to enhance the security of your Linux system.