LVM and RAID

When it comes to managing storage on your computer, especially in professional settings, traditional methods can sometimes be limiting. This is where technologies like Logical Volume Management (LVM) and Redundant Array of Inexpensive Disks (RAID) come in handy. They offer flexible and powerful ways to handle your disk space.

---

LOGICAL VOLUME MANAGEMENT (LVM)

LVM allows you to manage your disk space in a more flexible way than traditional disk partitions. Instead of being stuck with fixed-size partitions on a single disk, LVM lets you create "logical" partitions that can span across multiple physical hard drives and be easily resized.

---

HOW LVM WORKS

LVM builds a layer of abstraction between your operating system and the physical storage.

• Physical Volumes (PVs): Your physical disks or partitions are first converted into PVs. Think of these as the basic building blocks.

• Volume Groups (VGs): PVs are then combined to form VGs. A VG can include PVs from different physical disks, creating a large pool of storage.

• Logical Volumes (LVs): From these VGs, you create LVs, which are the "logical" partitions that your operating system sees and uses.

---

KEY ADVANTAGES OF LVM

LVM offers significant benefits over traditional disk partitioning:

• Flexible Resizing: You can resize logical volumes (make them bigger or smaller) even while your system is running and applications are using them. This means less downtime for your services.

• Data Migration: Easily move data from a failing or older physical disk to a newer one without taking your system offline.

• Performance and Redundancy: Combine physical devices to improve performance (disk striping) or add redundancy for data protection (disk mirroring).

• Snapshots: Create point-in-time "snapshots" of your logical volumes. This is incredibly useful for taking backups without interrupting your system's operations.

---

BASIC LVM COMMANDS

Here's a quick look at common LVM commands:

• pvcreate: Initializes a disk or partition as a Physical Volume (PV).

  • Example: pvcreate /dev/hda

• vgcreate: Creates a Volume Group (VG) from one or more PVs.

  • Example: vgcreate my_volume_group /dev/hda /dev/hdb

• vgextend: Adds a PV to an existing VG.

  • Example: vgextend my_volume_group /dev/hdc

• vgreduce: Removes a PV from a VG.

  • Example: vgreduce my_volume_group /dev/hdc (Note: This will also remove any LVs using space from that PV.)

• lvcreate: Creates a Logical Volume (LV) from free space in a VG.

  • Example: lvcreate -n my_logical_volume --size 100G my_volume_group

• lvextend: Extends the size of an LV.

  • Example: lvextend -L+20G /dev/my_volume_group/my_logical_volume

Once an LV is extended, you often need to extend the filesystem on it to utilize the new space. For example, with an ext3 filesystem, you might use ext2online.

---

LVM VERSIONS

There are different versions of LVM, with LVM2 being the current standard, offering more features and scalability compared to LVM1.

• LVM1: Older, limited in size and number of volumes.

• LVM2: Newer, supports much larger volumes (Exabytes) and more PVs/LVs, includes features like transactional metadata for faster recovery and volume mirroring.

---

REDUNDANT ARRAY OF INEXPENSIVE DISKS (RAID)

RAID combines multiple physical hard disks into a single logical unit. This is done by a "RAID controller," which can be hardware-based (transparent to the OS) or software-based (managed by the OS).

---

PURPOSE OF RAID

RAID aims to achieve one or more of the following:

• Increased Storage Capacity: Combine smaller disks into one larger logical disk.

• Improved Performance: Speed up read/write operations by distributing data across multiple disks.

• Data Redundancy: Protect against data loss if one or more hard disks fail.

---

ADVANTAGES OF RAID

• Cost-Effective: Often a more affordable way to achieve performance or redundancy compared to single, very expensive disks.

• Flexibility: Easily adapt to changing storage needs by reconfiguring software RAID without major hardware changes.

• Performance Boost: Certain RAID levels significantly improve data access speeds.

• Data Protection: Other RAID levels provide excellent data redundancy, ensuring your data survives disk failures.

---

DISADVANTAGES OF RAID

• Increased Complexity: RAID setups are more complex than using single disks, which can lead to issues with certain software or system recovery.

• Trade-offs: Gains in performance often come at the cost of reduced redundancy, and vice versa.

• Increased Wear: Spreading data across multiple disks can lead to more wear and tear, potentially increasing failure rates.

• Backup Challenges: Backing up and restoring data on RAID arrays can sometimes be more difficult.

---

COMMON RAID LEVELS

Different RAID levels offer varying balances of performance, redundancy, and capacity.

• RAID 0 (Striping):

  • How it works: Data is broken into chunks and written across all disks simultaneously.

  • Benefits: Excellent read/write performance.

  • Drawbacks: No redundancy. If one disk fails, all data is lost.

  • Best for: Non-critical data where speed is paramount (e.g., temporary files).

• RAID 1 (Mirroring):

  • How it works: Data is duplicated (mirrored) across all disks. Each disk contains an identical copy.

  • Benefits: High redundancy. System continues to operate as long as one disk is functional.

  • Drawbacks: Reduced usable capacity (only the size of the smallest disk). Slower write performance due to data being written multiple times.

  • Best for: Critical data storage where data integrity is essential.

• RAID 5:

  • How it works: Requires at least three disks. Data is striped across disks, and "parity" information (used for recovery) is distributed among them.

  • Benefits: Good balance of performance and redundancy. Can survive the loss of one disk.

  • Drawbacks: Slight CPU overhead for parity calculations.

  • Best for: Mission-critical scenarios requiring both good throughput and data integrity.

• Linear RAID:

  • How it works: Disks are simply concatenated (joined end-to-end) to form one large volume. Data is written sequentially.

  • Benefits: Increases overall storage size beyond individual disk limits.

  • Drawbacks: No performance improvement, no redundancy. Loss of any disk means loss of the entire array.

  • Best for: Creating a single large volume from smaller, disparate disks, where neither performance nor redundancy is a primary concern.

• Other Levels: More advanced levels like RAID 6 (dual parity) and nested levels (e.g., RAID 0+1, combining striping and mirroring) exist for highly specific and demanding environments.

Extending cPanel's Home Directory to a New Partition

Is your cPanel server's /home directory running out of space? This can stop you from creating new accounts or cause other issues. A common solution is to add a new hard drive and configure cPanel to use it for new user accounts. This guide explains how to do that simply.

---

PREPARING THE NEW PARTITION

Before cPanel can use the new space, you need to add and set up a new hard drive on your server.

Steps:

1. Install New Hard Drive: Physically install the new hard drive into your server.

2. Format the Drive: Format the new drive with a suitable file system (like ext4).

3. Mount as /home2: Create a new directory, for example, /home2, and mount the new hard drive to this location. This makes the new storage accessible.

---

TELLING CPANEL TO USE THE NEW PARTITION

Once /home2 is ready, you have two main ways to tell cPanel to use it for new accounts. Both methods involve editing the cPanel configuration file: /etc/wwwacct.conf.

---

OPTION 1: DEDICATE NEW ACCOUNTS TO /home2

This option makes sure all future cPanel accounts are created directly in the new /home2 partition.

How to Configure:

• Edit the /etc/wwwacct.conf file.

• Find the line that says HOMEDIR.

• Change its value to /home2.

Example:

HOMEDIR=/home2

Result:

After this change, any new cPanel accounts you create will automatically be placed in the /home2 directory. Existing accounts in /home will remain there.

---

OPTION 2: AUTO-SELECT BASED ON FREE SPACE

This is a more flexible option. cPanel will check both /home and /home2 and create new accounts on the partition with the most free space.

How to Configure:

• Edit the /etc/wwwacct.conf file.

• Find the line that says HOMEMATCH.

• Set its value to /home*.

Example:

HOMEMATCH=/home*

Result:

With this setting, when a new account is created, cPanel will intelligently decide whether to put it in /home or /home2 based on which directory has more available disk space. This helps distribute user data more evenly across your storage.

NCftp - get multiple Folders with ftp

The NcFTP client is a powerful tool for transferring files over FTP, especially useful for system administrators and developers. It offers advanced features like recursive directory downloads, making it a great alternative to standard FTP clients. This guide will walk you through installing NcFTP and using its recursive download capabilities.

---

INSTALLING NCFTP

NcFTP is compatible with various Unix-like operating systems, including FreeBSD, Solaris, and most Linux distributions.

On Debian/Ubuntu Systems: You can easily install NcFTP using the apt-get package manager. Open your terminal and run the following command:

sudo apt-get install ncftp

This command will download and install the NcFTP client on your system.

---

RECURSIVE DIRECTORY DOWNLOADS WITH NCFTPGET

ncftpget is a command-line tool within the NcFTP suite designed for efficient file transfers, particularly for scripting and advanced usage. It allows you to download entire directories and their contents recursively.

Basic Recursive Download Command: To download a remote directory and all its subdirectories and files, use the following format:

ncftpget -R -v -u "USERNAME" ftp.server.com /local/directory /remote/directory

Let's break down the options:

• -R: This crucial option tells ncftpget to copy all subdirectories and files recursively from the remote server.

• -v: Stands for "verbose." This displays detailed download activity and progress in your terminal, which is helpful for monitoring transfers.

• -u "USERNAME": Specifies the username for logging into the FTP server. If you omit this, ncftpget will attempt to log in anonymously.

• ftp.server.com: Replace this with the actual hostname or IP address of your FTP server.

• /local/directory: This is the path on your local machine where you want to save the downloaded files.

• /remote/directory: This is the path to the directory on the remote FTP server that you wish to copy.

Example: If you wanted to download the /www-data directory from ftp.nixcraft.net to your local /home/vivek/backup directory, using the username ftpuser, the command would be:

ncftpget -R -v -u "ftpuser" ftp.nixcraft.net /home/vivek/backup /www-data

---

TROUBLESHOOTING: "TAR MODE" ERROR

Sometimes, when performing recursive downloads, you might encounter an error message similar to this:

tar: End of archive volume 1 reached
tar: Sorry, unable to determine archive format.
Could not read directory listing data: Connection reset by peer

This error often indicates that ncftpget is attempting to use "TAR mode" for the recursive transfer, which might not be supported or configured correctly on the FTP server, or it's encountering issues with directory listings.

Solution: Disable TAR Mode To resolve this, you can add the -T option to your ncftpget command. The -T option explicitly tells ncftpget not to try using TAR mode with recursive mode.

Revised Command for Error Resolution:

ncftpget -T -R -v -u "ftpuser" ftp.nixcraft.net /home/vivek/backup /www-data

By adding -T, you ensure that ncftpget uses a different method for recursive downloads, bypassing the TAR mode issue.