Pages

Wednesday, July 31, 2013

Understanding Linux Resource Limits with limits.conf

The limits.conf file, along with files in the /etc/security/limits.d directory, controls how many system resources users can consume on a Linux system. This is handled by the pam_limits.so module and helps prevent a single user or process from monopolizing resources and affecting system stability.


WHAT ARE RESOURCE LIMITS?

Resource limits define how much of a system's resources a user or a group of users can use. These resources include things like:

  • CPU time: How long a process can use the processor.

  • Memory: How much RAM a process can occupy.

  • Open files: The maximum number of files a user can have open simultaneously.

  • Processes: The maximum number of programs or tasks a user can run.

  • Login sessions: The total number of times a user or group can be logged in.


HOW LIMITS.CONF WORKS

The limits.conf file uses a simple, four-column structure for each rule:

<domain> <type> <item> <value>

Let's break down each part:


DOMAIN: WHO THE LIMIT APPLIES TO

This specifies who the rule affects. It can be:

  • A specific username: e.g., john

  • A group: Use @groupname, e.g., @students.

  • Everyone: Use the wildcard *.

  • For login limits only: Use % for all system logins or %groupname for total logins for a specific group.

  • User ID (UID) ranges: e.g., 1000:2000 for users with UIDs between 1000 and 2000.

  • Group ID (GID) ranges: e.g., @100:200 for groups with GIDs between 100 and 200.

  • Specific GID for maxlogins: e.g., %:500 for users in the group with GID 500.


TYPE: HARD VS. SOFT LIMITS

This defines how strictly the limit is enforced:

  • hard: These are strict limits set by the system administrator. Users cannot exceed these limits.

  • soft: These are flexible limits that users can adjust downwards, but not above the hard limit. Think of them as default recommendations.

  • - (hyphen): Applies both soft and hard limits at the same time.


ITEM: WHAT RESOURCE IS BEING LIMITED

This specifies the resource you are limiting. Some common examples include:

  • core: Size of core dump files.

  • data: Maximum data segment size.

  • fsize: Maximum file size.

  • nofile: Maximum number of open files.

  • nproc: Maximum number of processes.

  • cpu: Maximum CPU time (in minutes).

  • maxlogins: Maximum number of simultaneous logins for a user.

  • maxsyslogins: Maximum number of simultaneous logins on the entire system.

  • priority: The "nice" priority of processes.

  • stack: Maximum stack size.

Most items support -1, unlimited, or infinity to mean no limit.


VALUE: THE LIMIT ITSELF

This is the numerical value for the limit you are setting, corresponding to the item. The units are usually specified in the item's description (e.g., KB for memory sizes, minutes for CPU time).


IMPORTANT CONSIDERATIONS

  • Per Login Session: Limits are applied when a user logs in and last only for that specific session. They are not system-wide permanent settings.

  • Individual Over Group: If a user has an individual limit set, it will override any group limits they are a part of.

  • Comments: Lines starting with # are comments and are ignored.

  • Error Reporting: The pam_limits module logs any configuration issues to syslog.


EXAMPLES

Here are a few common examples of how you might set limits:

  • * soft core 0

    • This sets the soft limit for core file size to 0 for all users. This prevents core dump files from being created by default.

  • * hard nofile 512

    • This sets a hard limit of 512 for the number of open files for all users. No user can open more than 512 files.

  • @student hard nproc 20

    • Users in the student group are limited to a hard maximum of 20 processes.

  • @faculty soft nproc 20

    • Users in the faculty group have a soft limit of 20 processes.

  • @faculty hard nproc 50

    • Users in the faculty group have a hard limit of 50 processes.

  • ftp hard nproc 0

    • The ftp user cannot run any processes.

  • @student - maxlogins 4

    • Users in the student group are limited to a maximum of 4 simultaneous logins (both soft and hard).

Understanding and configuring limits.conf is crucial for maintaining a stable and fair multi-user Linux environment.

Friday, July 26, 2013

How To Extract a Single File / Directory from Tarball Archive

A. tar command allows to extract a single file or directory using the following format. It works under UNIX, Linux, and BSD operating systems.

tar xvf /dev/st0 filename
tar xvf /dev/st0 directory-name
tar xvf mytar.ball.tar filename
tar -zxvf mytar.ball.tar.gz directory-name


Extract file to /tmp directory
tar -zxvf mytar.ball.tar.gz -C /tmp filename
tar -zxvf mytar.ball.tar.gz -C /tmp dir-name


Read tar man page for more information:
man tar

Monday, July 8, 2013

Cpanel intro

cPanel is a popular control panel that simplifies website and server management. It provides a user-friendly interface to handle tasks like creating email accounts, managing files, and setting up databases without needing to use complex command-line tools. Think of it as a dashboard for your web hosting.


IMPORTANT CPANEL DIRECTORIES

cPanel relies on specific directories to store its files, configurations, and scripts. Knowing these can help understand how cPanel works behind the scenes.

  • /usr/local/cpanel: This is the main directory for cPanel's core files, including its main services, binaries, logs, and user interface elements.

  • /var/cpanel: This directory holds important configuration data for cPanel, user settings, reseller information, logs, and bandwidth data.

  • /scripts: This directory contains many scripts that perform various cPanel and WHM (Web Host Manager) functions, like updates, backups, and account management.


DEEPER DIVE INTO /USR/LOCAL/CPANEL

This directory is the heart of your cPanel installation.

  • Core Services: You'll find executables like cpsrvd (the main cPanel process), cpsrvd-ssl (for secure connections), and cpkeyclt (for license management).

  • Binaries and Scripts (/usr/local/cpanel/bin): This sub-directory contains scripts for installing and configuring cPanel services, such as eximstats (for email statistics) and checkperlmodules (for checking Perl modules).

  • Logs (/usr/local/cpanel/logs): Important activity logs are stored here, including access_log and error_log for cpsrvd, stats_log for cpanellogd, and license_log for license updates.

  • User Interface Files (/usr/local/cpanel/base): This is where the different cPanel and Webmail themes (like x, x2, monsoon) and third-party applications (like squirrelmail, phpMyAdmin) are located.

  • Configuration Templates (/usr/local/cpanel/etc): This directory holds templates for various services like Exim (mail server), FTP, Apache (web server), and DNS zones.

  • Third-Party Tools (/usr/local/cpanel/3rdparty): This contains various third-party binaries and configuration files used by cPanel, such as PHP, stunnel, and website analytics tools like Analog and Webalizer.


DEEPER DIVE INTO /VAR/CPANEL

This directory stores crucial configuration and data files.

  • Primary cPanel Configuration (cpanel.conf): This file defines how cPanel behaves, with each setting on a new line. If it's missing, cPanel uses default values.

  • Reseller Information (resellers): This file lists all resellers and the WHM resources they can access.

  • Accounting Logs (accounting.log): Records actions like account creation and removal performed through WHM.

  • Bandwidth Data (bandwidth): Contains files tracking bandwidth usage for each account, named after the user.

  • Feature Lists (features): These files determine which cPanel resources are available to users, based on their assigned feature list name.

  • Packages (packages): Defines different hosting packages. If a package belongs to a reseller, its file name will start with the reseller's name.

  • User Configurations (users): These files store individual cPanel user settings, including their account resources, themes, and domains.

  • Other Notable Subdirectories:

    • LOGS: Stores logs from account copies and transfers.

    • UPDATELOGS: Contains logs from every cPanel update.

    • MAINIPS: Stores the main shared IP address for each reseller.

    • ZONETEMPLATES: Holds customized DNS zone templates created in WHM.


THE /SCRIPTS DIRECTORY

This directory is a powerhouse of utility scripts. These scripts are "building blocks" for many cPanel/WHM features and can be used to:

  • Update cPanel and its managed services.

  • Automate account creation.

  • Perform backups of cPanel accounts.

  • Install and update cPanel-managed services.


KEY CPANEL SERVICES

Several services work together to make cPanel function.

  • CPSRVD: The "master" process for cPanel. It handles all requests from the cPanel, WHM, and Webmail interfaces, logging its activity to access_log and error_log. It communicates securely using stunnel for SSL connections.

  • CHKSERVD: A service monitoring tool that checks the status of various services (like CPU, memory, and disk usage) every eight minutes. It logs to /var/log/chkservd.log and can dispatch alerts.

    • Configuration: Monitored services are defined in /etc/chkserv.d/chkservd.conf.

  • CPANELLOGD: Responsible for processing bandwidth logs and running statistics generators for each account. It's configured through WHM and stores statistics in /home/{username}/tmp.

  • CPBACKUP: Handles cPanel backups, typically configured to run daily at 1:00 AM via a cron job. Backups can be standard, incremental (using rsync), or remote (to an FTP server).

  • EXIMSTATS: This daemon collects bandwidth information from Exim (mail server) transactions, storing it in the eximstats database. It monitors exim_mainlog for data.


CPANEL STARTUP AND LICENSING

  • Startup: Services like cpsrvd, cPanel POP, and Log Services are controlled by the cPanel init script. You can check if ports are in use using netstat -lnp | egrep '20(8|9)'.

  • SSL Certificates: Default certificates are in /usr/local/cpanel/etc/cpanel.pem. User-installed certificates are in /usr/local/cpanel/etc/mycpanel.pem.

  • Troubleshooting Startup (SSL): If SSL services aren't available, try /usr/local/cpanel/startstunnel and check /usr/local/cpanel/3rdparty/bin/stunnel.log. For cpsrvd issues, run it directly and check its error log.

  • Licensing: License requests are handled by /usr/local/cpanel/cpkeyclt and sent to auth.cpanel.net over port 2089. The license key is stored in /usr/local/cpanel/cpanel.lisc.

  • Troubleshooting License Issues: Verify your license at http://verify.cpanel.net, check server connectivity to auth.cpanel.net on port 2089, and review license_log for errors.


CPANEL AND WHM REQUESTS

  • cPanel Requests: Logins use system password files. The document root for cPanel is /usr/local/cpanel/base. User themes and resources are defined in their configuration files and feature lists.

  • WHM Requests: The root password authenticates reseller users. The document root for WHM is /usr/local/cpanel/whostmgr/docroot/. Reseller resources are controlled by Access Control Lists, stored in /var/cpanel/resellers.


CPANEL MAINTENANCE AND UPDATES

cPanel automatically applies nightly updates at 2:13 AM using the /scripts/upcp script.

  • Update Process:

    1. /scripts/updatenow: Synchronizes the /scripts directory.

    2. /scripts/sysup: Updates cPanel-managed RPMs.

    3. /scripts/rpmup: Updates other system packages using your distribution's package manager (e.g., yum for CentOS).

  • Logging: Updates are logged to timestamped files in /var/cpanel/updatelogs.

  • Configuration (/etc/cpupdate.conf): This file controls update behavior, including the cPanel update branch (e.g., stable, release, current, edge) and whether system and RPM updates are applied.

  • Manual Updates: You can force a cPanel update by executing /scripts/upcp or /scripts/upcp --force if components are missing.

  • cpanelsync: This script, called by updatenow and upcp, uses MD5 checksums to synchronize files with cPanel update servers.


USEFUL CPANEL SCRIPTS

The /scripts directory contains many command-line utilities for managing your server.

  • Account Management:

    • /scripts/wwwacct: Creates new cPanel accounts.

    • /scripts/killacct: Terminates existing accounts.

    • /scripts/suspendacct//scripts/unsuspendacct: Suspends and unsuspends accounts.

    • /scripts/addpop: Creates new email accounts.

    • /scripts/updateuserdomains: Updates user and domain tables.

  • Package Management:

    • /scripts/ensurerpm//scripts/ensurepkg: Installs or updates RPMs/packages.

    • /scripts/realperlinstaller: Installs Perl modules via CPAN.

  • Service Updates and Configuration:

    • /scripts/mysqlup//scripts/cleanupmysqlprivs//scripts/mysqlconnectioncheck//scripts/restartsrv_mysql: For MySQL management.

    • /scripts/eximup//scripts/buildeximconf//scripts/restartsrv_exim: For Exim (mail server) management.

    • /scripts/rebuildnamedconf//scripts/restartsrv_bind: For BIND (DNS server) management.

    • /scripts/easyapache//scripts/rebuildhttpdconf//scripts/restartsrv_httpd: For Apache (web server) management.

  • General cPanel and System Scripts:

    • /scripts/restartsrv_{servicename}: Restarts most cPanel-managed services.

    • /scripts/makecpphp: Rebuilds cPanel's internal PHP interpreter.

    • /usr/local/cpanel/bin/checkperlmodules: Scans and installs required Perl modules.

    • /scripts/fixquotas: Attempts to rebuild quota databases.

FATAL error while starting VPS

FATAL error while starting VPS

=========
Problem:

Error "FATAL: kernel too old" while creating VPS with Fedora Core 5.

Solution:

Fedora Core 5 is compiled to require kernel 2.6.9. Still, it works fine with OpenVZ stable kernel based on 2.6.8. You can solve this problem by using following command:

echo 2.6.9 > /proc/sys/kernel/virt_osrelease

Increase max emails per hour for a single domain in cPanel

You can change the maximum number of emails allowed for a specific domain from the system default by editing the backend file.

vi /var/cpanel/maxemails

Simply add the entry “domain.com = 100". This sets the limit to 100 emails per hour for domain.com.

Remember to run the following script after updating /var/cpanel/maxemails:

/scripts/build_maxemails_config

myisamchk “A super tool to view and repair corrupt databases”

myisamchk “A super tool to view and repair corrupt databases”

The myisamchk utility gets information about your database tables or checks, repairs, or optimizes them. myisamchk works with MyISAM tables (tables that have .MYD and .MYI files for storing data and indexes).
Caution
It is best to make a backup of a table before performing a table repair operation; under some circumstances the operation might cause data loss. Possible causes include but are not limited to file system errors.
To find the tables that are corrupt and to output it to a file run the following script

find -name “*.MYI” -exec myisamchk -c {} \; > /root/tbl_chk

To view the number of tables corrupt

cat /root/tbl_chk | grep MyISAM | wc -l

To repair the corrupt tables

find / -name “*.MYI” -exec myisamchk -r {} \;

Thursday, May 16, 2013

Atomic mod security rules

ConfigServer ModSecurity Control provides an easy way of monitoring which rules are being triggered on the server in real time but more importantly, you can whitelist certain rules either globally accross the entire server or on a per account/domain basis if some of the rules conflict with a particular script or functionality (e.g. FrontPage). To install CMC, run the following:

rm -fv cmc.tgz

wget http://www.configserver.com/free/cmc.tgz
tar -xzf cmc.tgz
cd cmc
sh install.sh
cd ..
rm -Rfv cmc/ cmc.tgz
If you log in to WHM you will now see “ConfigServer ModSec Control” under “Plugins”. It’s important that you click on it because when it’s run the first time, it will create the file “modsec2.whitelist.conf” if it doesn’t already exist. If that file doesn’t exist then you’ll find Apache won’t start when we come to the end of this guide. Also while you’re here, click on “Disable modsecparse.pl”. This will disable the cPanel cron job that processes and empties the mod_security log, allowing you to use the log watching tool built in to CMC.

As to help our VPS and Dedicated Server customers who might also be effected by this we have designed the following guide to make installing Atmoic Mod Security into cPanel with little to no fuss.

Stage 1: Run the following commands at command line:

mkdir /var/asl
mkdir /var/asl/tmp
mkdir /var/asl/data
mkdir /var/asl/data/msa
mkdir /var/asl/data/audit
mkdir /var/asl/data/suspicious
chown nobody.nobody /var/asl/data/msa
chown nobody.nobody /var/asl/data/audit
chown nobody.nobody /var/asl/data/suspicious
chmod o-rx -R /var/asl/data/*
chmod ug+rwx -R /var/asl/data/*
mkdir /var/asl/updates
mkdir /var/asl/rules/
mkdir /var/asl/rules/clamav
mkdir /etc/asl/
touch /etc/asl/whitelist
cd /usr/local/src/
wget http://updates.atomicorp.com/channels/rules/delayed/modsec-2.7-free-latest.tar.gz
tar zxvf modsec-2.7-free-latest.tar.gz
mkdir /usr/local/apache/conf/modsec_rules/
cp modsec/* /usr/local/apache/conf/modsec_rules/
These command will create the required directory’s and download the latest free version of the Atomic Mod Security rules. It will also directly install them into the location of Apache designed for cPanel and configure the permission.

Stage 2: Configure cPanel to use the Mod Security Rules

In this stage, you can do everything from WHM as long as you have Mod Security already installed as part of your EasyApache build. If you do not, you will need to rebuild apache with Mod Security.

In go to: WHM -> Plugins -> Mod Security and then click: Edit Config

In this section, delete all the current content and then paste in the following configuration:

SecRequestBodyAccess On
SecAuditLogType Concurrent
SecResponseBodyAccess On
SecResponseBodyMimeType (null) text/html text/plain text/xml
SecResponseBodyLimit 2621440
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecServerSignature Apache
SecUploadDir /var/asl/data/suspicious
SecUploadKeepFiles Off
SecAuditLogParts ABIFHZ
SecArgumentSeparator "&"
SecCookieFormat 0
SecRequestBodyInMemoryLimit 131072
SecDataDir /var/asl/data/msa
SecTmpDir /tmp
SecAuditLogStorageDir /var/asl/data/audit
SecResponseBodyLimitAction ProcessPartial

Include /usr/local/apache/conf/modsec_rules/10_asl_antimalware.conf
Include /usr/local/apache/conf/modsec_rules/10_asl_rules.conf
Include /usr/local/apache/conf/modsec_rules/20_asl_useragents.conf
Include /usr/local/apache/conf/modsec_rules/30_asl_antispam.conf
Include /usr/local/apache/conf/modsec_rules/50_asl_rootkits.conf
Include /usr/local/apache/conf/modsec_rules/60_asl_recons.conf
Include /usr/local/apache/conf/modsec_rules/99_asl_jitp.conf
Include /usr/local/apache/conf/modsec2.whitelist.conf
Save this and restart Apache.

This should now have successfully installed the Atomic mod security rules into cPanel which are a much more secure rule base and include extra protection which is important for the latest hacks.

Testing

http://YOUR_HOST/foo.php?foo=http://www.example.com

should give 403