Automatic deletion of files on QNAP drive via SSHFS

Automation of Disk Space Management in a Linux Environment

In today’s digital world, where data is being accumulated in ever-increasing amounts, managing disk space has become a key aspect of maintaining operational efficiency in systems. In this article, I will present a script that automates the process of managing space on a remote disk mounted via SSHFS, particularly useful for system administrators who regularly deal with filling storage media.

Prerequisites

Before starting, ensure that SSHFS and all necessary packages enabling its proper operation are installed on your system. SSHFS allows remote file systems to be mounted via SSH, which is crucial for our script’s operation. To install SSHFS and the necessary tools, including a package that enables password forwarding (sshpass), use the following command:

apt-get update
apt-get install sshfs fuse sshpass -y

1 2	apt-get update apt-get install sshfs fuse sshpass -y

Bash Script for Disk Space Management

Our Bash script focuses on monitoring and maintaining a defined percentage of free disk space on a remote disk mounted via SSHFS. Here are the script’s main functions:

Goal Definition:

TARGET_USAGE=70 – the percentage of disk space we want to maintain as occupied. The script will work to keep at least 30% of the disk space free.

Mount Point and Paths:

MOUNT_POINT=”/mnt/qnapskorupki” – the local directory where the remote disk is mounted. TARGET_DIRS=”$MOUNT_POINT/up*.soban.pl” – the directories where the script will look for files to delete if needed.

Function check_qnap: This function checks whether the disk is mounted and whether the mount directory is not empty. If there are issues, the script attempts to unmount and remount the disk using sshfs with a password forwarded through sshpass.

File Deletion: The script monitors disk usage and, if TARGET_USAGE is exceeded, it finds and deletes the oldest files in specified directories until the target level of free space is achieved.

#!/bin/bash

TARGET_USAGE=70
MOUNT_POINT="/mnt/qnapskorupki"
TARGET_DIRS="$MOUNT_POINT/up*.soban.pl"

# Function to check and mount SSHFS
function check_qnap {
    local remote_path="/share/MD0_DATA/backup_proxmox/"
    local user_remote="remote_user"
    local remote_host="192.168.1.XX"
    local port=22
    local password='XXXXXXXXXXXXXXXXXXXXXXX'

    # Check if the mounting directory exists and is empty
    if [ ! -d "$MOUNT_POINT" ] || [ -z "$(ls -A $MOUNT_POINT)" ]; then
        echo "Problem: The directory $MOUNT_POINT is missing or empty. Attempting to remount..."

        # Unmount if anything is currently mounted
        if mountpoint -q $MOUNT_POINT; then
            echo "Unmounting $MOUNT_POINT..."
            fusermount -u $MOUNT_POINT
            sleep 5
        fi

        # Remount
        echo "Mounting SSHFS: $user_remote@$remote_host:$remote_path to $MOUNT_POINT..."
        echo $password | sshfs $user_remote@$remote_host:$remote_path $MOUNT_POINT -o password_stdin

        # Check if the mounting was successful
        if mountpoint -q $MOUNT_POINT; then
            echo "QNAP successfully mounted."
        else
            echo "Error: Failed to mount $remote_host:$remote_path to $MOUNT_POINT."
            exit 1
        fi
    fi
}

# Check and mount the disk
check_qnap

# Begin deleting files
current_usage=$(df --output=pcent "$MOUNT_POINT" | tail -n 1 | tr -d '%')
echo "Current usage: $current_usage%"
target_free_space=$((100 - $TARGET_USAGE))

# Continue deleting files until the required free space is achieved
while [[ $current_usage -gt $TARGET_USAGE ]]; do
    # Find the oldest file in the target directories
    oldest_file=$(find $TARGET_DIRS -type f -printf '%T+ %p\n' | sort | head -n 1 | cut -d' ' -f2)

    if [ -z "$oldest_file" ]; then
        echo "No files to delete."
        break
    fi

    # Delete the oldest file
    echo "Deleting file $oldest_file..."
    rm -f "$oldest_file"

    # Check usage again
    current_usage=$(df --output=pcent "$MOUNT_POINT" | tail -n 1 | tr -d '%')
    echo "New usage after deletion: $current_usage%"
done

echo "Target disk usage achieved."

#!/bin/bash

TARGET_USAGE=70

MOUNT_POINT="/mnt/qnapskorupki"

TARGET_DIRS="$MOUNT_POINT/up*.soban.pl"

# Function to check and mount SSHFS

function check_qnap {

local remote_path="/share/MD0_DATA/backup_proxmox/"

local user_remote="remote_user"

local remote_host="192.168.1.XX"

local port=22

local password='XXXXXXXXXXXXXXXXXXXXXXX'

# Check if the mounting directory exists and is empty

if [ ! -d "$MOUNT_POINT" ] || [ -z "$(ls -A $MOUNT_POINT)" ]; then

echo "Problem: The directory $MOUNT_POINT is missing or empty. Attempting to remount..."

# Unmount if anything is currently mounted

if mountpoint -q $MOUNT_POINT; then

echo "Unmounting $MOUNT_POINT..."

fusermount -u $MOUNT_POINT

sleep 5

# Remount

echo "Mounting SSHFS: $user_remote@$remote_host:$remote_path to $MOUNT_POINT..."

echo $password | sshfs $user_remote@$remote_host:$remote_path $MOUNT_POINT -o password_stdin

# Check if the mounting was successful

if mountpoint -q $MOUNT_POINT; then

echo "QNAP successfully mounted."

else

echo "Error: Failed to mount $remote_host:$remote_path to $MOUNT_POINT."

exit 1

}

# Check and mount the disk

check_qnap

# Begin deleting files

current_usage=$(df --output=pcent "$MOUNT_POINT" | tail -n 1 | tr -d '%')

echo "Current usage: $current_usage%"

target_free_space=$((100 - $TARGET_USAGE))

# Continue deleting files until the required free space is achieved

while [[ $current_usage -gt $TARGET_USAGE ]]; do

# Find the oldest file in the target directories

oldest_file=$(find $TARGET_DIRS -type f -printf '%T+ %p\n' | sort | head -n 1 | cut -d' ' -f2)

if [ -z "$oldest_file" ]; then

echo "No files to delete."

break

# Delete the oldest file

echo "Deleting file $oldest_file..."

rm -f "$oldest_file"

# Check usage again

current_usage=$(df --output=pcent "$MOUNT_POINT" | tail -n 1 | tr -d '%')

echo "New usage after deletion: $current_usage%"

done

echo "Target disk usage achieved."

Example Script Execution:

script starts working and gradually deletes files

The script will run until it reaches 70% usage as planned:

Downloading the script and adding it to crontab

Of course, the script should be adjusted to meet your specific needs. However, if you want to download it and add it to crontab, follow these steps:

wget soban.pl/bash/remove_old_files.sh
chmod +x remove_old_files.sh

1 2	wget soban.pl/bash/remove_old_files.sh chmod +x remove_old_files.sh

If you want to automate the file removal process, for example, at the end of the day, add the following entry to crontab:

crontab -e

1	crontab -e

In this case, the script will run every day at 11:55 PM:

45 23 * * * /root/scripts/remove_old_files.sh > /dev/null 2>&1

1	45 23 * * * /root/scripts/remove_old_files.sh > /dev/null 2>&1

Make sure to use the correct path to the script.

Security and Optimization

The script uses a password directly in the command line, which can pose a security risk. In practical applications, it is recommended to use more advanced authentication methods, such as SSH keys, which are more secure and do not require a plaintext password in the script. However, in the case of QNAP, we used a password when writing this script.

Conclusion

The presented script is an example of how daily administrative tasks, such as disk space management, can be automated, thus increasing efficiency and reliability. Its implementation in real IT environments can significantly streamline data management processes, especially in situations where quick response to changes in disk usage is critical.