If you run a Linux server with Nginx, SSH, or WordPress, you probably already know Fail2Ban. It is a solid tool, but it works locally — it blocks only the IP addresses that attacked your server.

CrowdSec works in a completely different way. It is a protection system based on shared IP reputation. If thousands of servers worldwide detect a malicious IP address, your server can block it before an attack even happens.

How Does CrowdSec Work?

• analyzes system logs (nginx, ssh, wordpress)
• detects suspicious behavior
• shares attacker IP intelligence with other servers
• blocks traffic at the firewall level

The result? Most bots and internet scanners never even reach your Nginx server.

Installing CrowdSec on Debian / Ubuntu

CrowdSec installation is very simple and available directly from Debian repositories.

apt update
apt install crowdsec

During installation, CrowdSec automatically:

• creates a local API (LAPI)
• registers the server in CrowdSec Central API
• downloads default security scenarios

Installing Firewall Bouncer

CrowdSec detects threats, but it requires an enforcement component — called a bouncer — which blocks traffic at the firewall level.

apt install crowdsec-firewall-bouncer

By default, the bouncer uses nftables and automatically adds blocking rules for malicious IP addresses.

Installing Security Collections

Collections include log parsers and attack detection scenarios.

cscli collections install crowdsecurity/nginx
cscli collections install crowdsecurity/wordpress
cscli collections install crowdsecurity/base-http-scenarios
cscli collections install crowdsecurity/sshd

Reload configuration after installation:

systemctl reload crowdsec

Configuring Nginx Logs

To allow CrowdSec to analyze HTTP traffic, you must specify Nginx log files.

Edit the file:

nano /etc/crowdsec/acquis.yaml

Add the following configuration:

filenames:
  - /var/log/nginx/access*.log
  - /var/log/nginx/error*.log
labels:
  type: nginx

Then restart the service:

systemctl restart crowdsec

Verifying CrowdSec Operation

Service status:

systemctl status crowdsec

Active bans list:

cscli decisions list

Operational metrics:

cscli metrics

Final Result

After proper CrowdSec installation:

• the server automatically blocks known botnets
• WordPress attacks and SSH brute-force attempts are stopped at the firewall
• Nginx handles significantly less malicious traffic
• server CPU and IO usage are noticeably reduced

CrowdSec can be considered an evolution of Fail2Ban — a system that not only reacts locally but also benefits from global threat intelligence.

Artykuł CrowdSec – Intelligent Linux Server Protection Against Botnets, Brute-Force Attacks and Internet Scanning pochodzi z serwisu soban.

This guide shows a complete Fail2Ban installation and configuration for Nginx, designed to:

• block real scanners and exploit attempts (e.g. requests to /.env, /.git, /phpmyadmin, etc.),
• avoid blocking administrators by accident (common issue when banning only by HTTP errors),
• ban IP addresses after repeated suspicious activity,
• use a short ban time (5 minutes) to reduce the risk of locking yourself out.

Why banning by HTTP errors alone can backfire

Many guides suggest banning by HTTP status codes (4xx/499) only. In real life this often causes self-bans, because modern apps generate bursts of requests (AJAX, admin panels, cache rebuilds), and you can hit error statuses during normal work.

This setup uses a safer approach:

• exploit paths are always suspicious,
• HTTP errors are counted only when the request has no Referer (typical for scanners),
• known bad User-Agents are counted.

Step 1: Install Fail2Ban

Install Fail2Ban:

apt update
apt install -y fail2ban

Enable and start the service:

systemctl enable fail2ban
systemctl start fail2ban

Verify that it is running:

fail2ban-client ping
systemctl status fail2ban

Step 2: Create the nginx-secure filter

Create the filter file:

nano /etc/fail2ban/filter.d/nginx-secure.conf

Paste the following configuration:

[Definition]

failregex =
    # exploit paths
    ^<HOST> - .* "(?:GET|POST|HEAD|PUT|DELETE|OPTIONS|PATCH|PROPFIND|CONNECT) (?:/\.env|/wp-config\.php|/phpinfo\.php|/(?:phpmyadmin|adminer)(?:/|$)|/(?:\.git|\.svn|\.hg)(?:/|$)|/vendor/phpunit/|/cgi-bin/).*" \d{3} .*

    # errors without referer (real scanners)
    ^<HOST> - .* "(?:GET|POST|HEAD|PUT|DELETE|OPTIONS|PATCH|PROPFIND|CONNECT) [^"]*" (?:400|403|404|405|408|413|414|429|444) [^"]* "-" ".*"

    # bad user agents
    ^<HOST> - .* "(?:GET|POST|HEAD|PUT|DELETE|OPTIONS|PATCH|PROPFIND|CONNECT).*" \d{3} .* "(?:[^"]*)" "(?:[^"]*(?:sqlmap|nikto|masscan|zgrab|nmap|acunetix|wpscan|dirbuster|gobuster)[^"]*)"

ignoreregex =

Step 3: Create the nginx-secure jail

Create the jail configuration file:

nano /etc/fail2ban/jail.d/nginx-secure.conf

Paste the following configuration:

[nginx-secure]
enabled  = true
port     = http,https
filter   = nginx-secure

logpath  = /var/log/nginx/access.log
           /var/log/nginx/access-*.log

findtime = 600
maxretry = 20
bantime  = 300

action   = iptables-multiport[name=nginx-secure, port="http,https"]

ignoreip = 127.0.0.1/8 ::1

Step 4: Restart Fail2Ban

fail2ban-server -t
systemctl restart fail2ban

Step 5: Verify firewall integration

Check that the Fail2Ban chain exists:

iptables -S | grep f2b-nginx-secure
iptables -L f2b-nginx-secure -n -v

External test

Run from another machine (exploit path test):

for i in 1 2 3 4 5; do
  curl -I https://soban.pl/.env
done

Test “errors without Referer” logic (scanner-like request – no referer header):

for i in 1 2 3 4 5; do
  curl -sS -o /dev/null -w "%{http_code}\n" https://soban.pl/this-path-should-not-exist-$i
done

After repeated suspicious requests, the IP address will be banned for 5 minutes.

Check banned IPs

fail2ban-client status nginx-secure

Unban IP address

Unban an IP manually:

fail2ban-client set nginx-secure unbanip YOUR_IP

Summary

• blocks exploit paths and real scanners,
• reduces self-bans by counting HTTP errors only when the Referer is missing,
• uses a short 5-minute ban duration,
• works with iptables-nft,
• easy to test and easy to unban IP addresses.

Artykuł Fail2Ban Nginx Setup – Block Scanners and Exploits Without Blocking WordPress Admin pochodzi z serwisu soban.

Upgrading Debian from version 12 (bookworm) to 13 (trixie) is an operation that should be performed in a repeatable and predictable way, especially on servers and containers (for example Proxmox LXC or virtual machines). Below you will find a simple guide and ready-to-use commands to download and run the upgrade script.

Before running the upgrade: create a backup or snapshot. In Proxmox, the best option is vzdump or a snapshot. On bare metal, at minimum back up /etc, applications, and databases.

• Proxmox LXC / VM: backup using vzdump or create a snapshot.
• Server: backup /etc, /var/www, databases (MySQL/PostgreSQL), and SSL certificates.

Script download:

https://soban.pl/bash/upgrade_to_debian13.sh

1) Backup before upgrade (examples)

Example backup in Proxmox (run on Proxmox host, replace CTID/VMID):

vzdump 101 --mode snapshot --compress zstd --storage local

Example simple filesystem backup on a server (this does not replace a full snapshot, but it’s better than nothing):

tar czf /root/backup_before_upgrade.tar.gz /etc /var/www /root

2) Download the script (wget / curl)

The easiest way is to use wget. If the wget command does not work even though the package is installed, use the full path /usr/bin/wget.

Variant A (standard wget):

apt update
apt install -y wget
cd /root
wget -O upgrade_to_debian13.sh https://soban.pl/bash/upgrade_to_debian13.sh
chmod +x upgrade_to_debian13.sh

Variant B (wget with full path – useful if PATH is broken):

apt update
apt install -y wget
cd /root
/usr/bin/wget -O upgrade_to_debian13.sh https://soban.pl/bash/upgrade_to_debian13.sh
chmod +x upgrade_to_debian13.sh

Variant C (curl):

apt update
apt install -y curl
cd /root
curl -fsSL -o upgrade_to_debian13.sh https://soban.pl/bash/upgrade_to_debian13.sh
chmod +x upgrade_to_debian13.sh

3) Script help (parameters)

Before running the upgrade, display available parameters and usage examples:

cd /root
./upgrade_to_debian13.sh --help

4) Upgrade Debian 12 → Debian 13 (system only)

If you are currently running Debian 12 (bookworm) and want to perform a system upgrade:

cd /root
./upgrade_to_debian13.sh

The script will create a backup of /etc/apt/sources.list, switch repositories to trixie, run apt update and apt full-upgrade, and finally execute autoremove and autoclean.

5) Auto-detect PHP/nginx and update if needed

If the container or VM is running a web stack and you want the script to automatically detect PHP usage (nginx + fastcgi_pass) and upgrade PHP and nginx if necessary:

cd /root
./upgrade_to_debian13.sh --auto

6) Force PHP and nginx upgrade (PHP-FPM socket fix)

If you want to force installation or upgrade of PHP and automatically fix nginx configuration to use the correct PHP-FPM socket:

cd /root
./upgrade_to_debian13.sh --with-php --with-nginx --php-version 8.2

This command installs PHP 8.2 (php-fpm and common modules) and replaces old PHP-FPM socket paths in nginx configuration with /run/php/php8.2-fpm.sock. Then it runs nginx -t and reloads or restarts services.

7) Already running Debian 13? PHP/nginx only mode

If the system is already running Debian 13 (trixie) and you only want to upgrade PHP and nginx without modifying system repositories:

cd /root
./upgrade_to_debian13.sh --php-nginx-only --with-php --with-nginx --php-version 8.2

8) Dry-run mode (test mode)

If you want to see what the script will do without making any changes:

cd /root
./upgrade_to_debian13.sh --auto --dry-run

9) Troubleshooting: wget installed but not working

If apt reports wget is installed but the shell shows command not found, it is usually a PATH issue. The easiest workaround is to use the full path: /usr/bin/wget.

echo "$PATH"
command -v wget || true
ls -l /usr/bin/wget || true
/usr/bin/wget --version || true

Summary

This solution provides a convenient way to upgrade Debian 12 → Debian 13 and optionally fix common PHP/nginx issues after upgrade (PHP-FPM socket paths, nginx config testing, and service restart). Always create a backup before upgrading and start by running --help to review available options.

Script: https://soban.pl/bash/upgrade_to_debian13.sh

Artykuł Automatic upgrade Debian 12 → Debian 13 with optional PHP and nginx update pochodzi z serwisu soban.

If you’re trying to update the firmware of a Dell WD19/WD19S docking station on Proxmox (or Debian) using fwupdmgr, you may hit the classic Operation timed out error during the Erasing… stage. In practice, the update fails due to USB power management (autosuspend). Below you’ll find a simple, effective fix and a ready-to-run script you can use on a server or laptop.

Symptoms

Most commonly, during a dock firmware update (WD19/WD19S), you’ll see an error similar to:

failed to write-firmware: failed to erase bank: failed after 5 retries: failed to SetReport: USB error: Operation timed out [-7]

At this point, fwupdmgr may show the WD19S device as Update State: Failed or keep reporting pending activation, but the flash process never completes.

Why does this happen?

During flashing, the dock performs long-running operations. If the system tries to save power on USB (autosuspend), control transfers can fail. The result is a timeout exactly during the flash bank erase stage (erase bank).

Quick fix: disable USB autosuspend during the update

The simplest solution is to temporarily set autosuspend to -1 (disabled). This setting lasts until reboot (unless you make it permanent in the kernel cmdline), but it’s more than enough for the update process.

echo -1 > /sys/module/usbcore/parameters/autosuspend

Then run the firmware update:

fwupdmgr refresh --force
fwupdmgr update

After the update: “pending activation” and unplug requirement

After a successful firmware installation for WD19/WD19S, fwupdmgr often prints the following message:

The update will continue when the device USB cable has been unplugged.
WD19S is pending activation; use fwupdmgr activate to complete the update.

Do the following in this exact order:

• Unplug the USB-C cable from the dock (from the laptop).
• (Optional) Unplug the dock power for 10–15 seconds and plug it back in.
• Plug the USB-C cable back in.
• Run the activation step:

fwupdmgr activate

Finally, verify the status:

fwupdmgr get-updates
fwupdmgr get-devices | less

Ready-to-use script: install + update + autosuspend disable (with restore)

Below is a ready-to-run script that:

• installs fwupd
• refreshes LVFS metadata
• temporarily disables USB autosuspend (so WD19S won’t fail with timeouts)
• runs firmware updates
• restores the previous autosuspend value afterwards (even if the update fails)

You can copy the script directly from this article, but if you prefer a faster way, you can download it from:

https://soban.pl/bash/dell_updage.sh

Example download and run:

cd /root/scripts
curl -fsSL https://soban.pl/bash/dell_updage.sh -o dell_updage.sh
chmod +x dell_updage.sh
./dell_updage.sh

If you want to preview the script before running it:

curl -fsSL https://soban.pl/bash/dell_updage.sh | less

If you don’t have less installed:

apt update
apt install -y less

Script (full content)

#!/bin/bash
set -euo pipefail

# dell upgrade:
# - installs fwupd
# - refreshes metadata
# - runs fwupdmgr update
# - TEMPORARILY disables USB autosuspend to avoid WD19/WD19S timeouts
# - restores autosuspend setting after update

echo "[INFO] Installing fwupd..."
apt update
apt install -y fwupd

echo "[INFO] Refreshing firmware metadata..."
fwupdmgr refresh --force
fwupdmgr get-updates || true

# Save current autosuspend value (if exists)
AUTOSUSPEND_PATH="/sys/module/usbcore/parameters/autosuspend"
OLD_AUTOSUSPEND=""

if [[ -f "$AUTOSUSPEND_PATH" ]]; then
  OLD_AUTOSUSPEND="$(cat "$AUTOSUSPEND_PATH" || true)"
  echo "[INFO] Current usbcore autosuspend: ${OLD_AUTOSUSPEND}"
else
  echo "[WARN] autosuspend sysfs file not found: $AUTOSUSPEND_PATH"
fi

restore_autosuspend() {
  if [[ -f "$AUTOSUSPEND_PATH" && -n "${OLD_AUTOSUSPEND}" ]]; then
    echo "[INFO] Restoring usbcore autosuspend back to: ${OLD_AUTOSUSPEND}"
    echo "${OLD_AUTOSUSPEND}" > "$AUTOSUSPEND_PATH" || true
  fi
}
trap restore_autosuspend EXIT

# Disable autosuspend to prevent USB timeout during dock firmware erase/write
if [[ -f "$AUTOSUSPEND_PATH" ]]; then
  echo "[INFO] Disabling USB autosuspend (set to -1) to avoid dock update timeouts..."
  echo -1 > "$AUTOSUSPEND_PATH"
fi

read -p "Do you want to update the entire device? (y/n): " answer
if [[ "$answer" == "y" || "$answer" == "Y" ]]; then
  echo "[INFO] Running fwupdmgr update..."
  fwupdmgr update || {
    echo "[ERROR] fwupdmgr update failed."
    echo "[HINT] If this is Dell dock (WD19/WD19S), try: unplug USB-C, replug, then run: fwupdmgr activate"
    exit 1
  }

  echo "[INFO] Update finished."
  echo "[INFO] If you updated a Dell dock and it says 'pending activation':"
  echo "       1) Unplug USB-C cable from the dock"
  echo "       2) (Optional) Unplug dock power for 10 seconds, plug back"
  echo "       3) Run: fwupdmgr activate"
else
  echo "[INFO] Skipping firmware update."
fi

echo "[INFO] Done."

Running the script

The simplest way:

chmod +x dell_updage.sh
./dell_updage.sh

FAQ & tips

• The update still fails? Disconnect all peripherals from the dock (monitors/LAN/USB), leave only power and USB-C, do a hard reset of the dock (unplug power for 30s), and try again.
• “pending activation” after update – this is normal for WD19/WD19S. You must unplug USB-C, plug it back in, then run fwupdmgr activate.
• Does this update the laptop BIOS? Not always. fwupdmgr shows “System Firmware” (BIOS/UEFI) separately from the dock. This article focuses on the dock and the USB timeout issue.

Summary

If Dell WD19/WD19S firmware updates fail on Proxmox/Debian during the Erasing… stage, in most cases it’s enough to disable USB autosuspend temporarily. The script above does that automatically and restores the previous setting afterwards, so your system keeps working normally.

Artykuł Dell WD19/WD19S: firmware update on Proxmox/Debian without USB timeouts (fwupd + autosuspend fix) pochodzi z serwisu soban.

Want your Tmux window titles to automatically show the hostname you connect to via SSH — and restore the local name after you disconnect? This guide will walk you through setting that up step by step.

1. Tmux configuration – ~/.tmux.conf

In your Tmux configuration file, set the default shell command to a custom Bash init file (.bash_tmux) that will handle window title updates.

set-option -g default-command 'bash --rcfile ~/.bash_tmux'

2. Bash init script – ~/.bash_tmux

This file runs at shell startup inside Tmux. It sets the window name to your local hostname and overrides the ssh command to change the window title dynamically.

#!/bin/bash
# ~/.bash_tmux
# Custom Bash init file for Tmux sessions with dynamic tab naming

# Override the default ssh command to dynamically rename the Tmux window
ssh() {
  if [ -n "$TMUX" ]; then
    # Loop through arguments to find the target host (first non-flag argument)
    for arg in "$@"; do
      if [[ "$arg" != -* ]]; then
        if [[ "$arg" == *@* ]]; then
          host="${arg#*@}"  # extract host from user@host
        else
          host="$arg"
        fi
        break
      fi
    done

    # Strip any trailing junk or whitespace
    host="$(echo "$host" | cut -d' ' -f1)"

    # Rename the Tmux window to the SSH target
    [ -n "$host" ] && tmux rename-window "$host"

    # Save the local hostname to restore later
    local_host="$(hostname -s)"

    # Run the actual SSH command
    command ssh "$@"

    # Restore the original window name after logout
    tmux rename-window "$local_host"
  else
    # Not in Tmux — just run ssh normally
    command ssh "$@"
  fi
}

# Load the user's regular bash configuration
source ~/.bashrc

# On shell start, if in Tmux, set the window name to the current hostname
if [ -n "$TMUX" ]; then
  tmux rename-window "$(hostname -s)"
fi

3. Reload Tmux config without restarting

To apply the changes without restarting the Tmux session, run the following:

tmux source-file ~/.tmux.conf

To manually reload the current shell with the new .bash_tmux configuration:

exec bash --rcfile ~/.bash_tmux

4. Final result

• New Tmux windows automatically show the local hostname
• When connecting via SSH, the window is renamed to the remote host
• When disconnecting, the original local name is restored

Clean, readable, and perfect for sysadmins, devops engineers, and Tmux lovers who appreciate automatic order in their terminal tabs 💪

Artykuł Dynamic Tmux Window Titles with SSH and Hostname pochodzi z serwisu soban.

Automation of Disk Space Management in a Linux Environment

In today’s digital world, where data is being accumulated in ever-increasing amounts, managing disk space has become a key aspect of maintaining operational efficiency in systems. In this article, I will present a script that automates the process of managing space on a remote disk mounted via SSHFS, particularly useful for system administrators who regularly deal with filling storage media.

Prerequisites

Before starting, ensure that SSHFS and all necessary packages enabling its proper operation are installed on your system. SSHFS allows remote file systems to be mounted via SSH, which is crucial for our script’s operation. To install SSHFS and the necessary tools, including a package that enables password forwarding (sshpass), use the following command:

apt-get update
apt-get install sshfs fuse sshpass -y

Bash Script for Disk Space Management

Our Bash script focuses on monitoring and maintaining a defined percentage of free disk space on a remote disk mounted via SSHFS. Here are the script’s main functions:

Goal Definition:

TARGET_USAGE=70 – the percentage of disk space we want to maintain as occupied. The script will work to keep at least 30% of the disk space free.

Mount Point and Paths:

MOUNT_POINT=”/mnt/qnapskorupki” – the local directory where the remote disk is mounted. TARGET_DIRS=”$MOUNT_POINT/up*.soban.pl” – the directories where the script will look for files to delete if needed.

Function check_qnap: This function checks whether the disk is mounted and whether the mount directory is not empty. If there are issues, the script attempts to unmount and remount the disk using sshfs with a password forwarded through sshpass.

File Deletion: The script monitors disk usage and, if TARGET_USAGE is exceeded, it finds and deletes the oldest files in specified directories until the target level of free space is achieved.

#!/bin/bash

TARGET_USAGE=70
MOUNT_POINT="/mnt/qnapskorupki"
TARGET_DIRS="$MOUNT_POINT/up*.soban.pl"

# Function to check and mount SSHFS
function check_qnap {
    local remote_path="/share/MD0_DATA/backup_proxmox/"
    local user_remote="remote_user"
    local remote_host="192.168.1.XX"
    local port=22
    local password='XXXXXXXXXXXXXXXXXXXXXXX'

    # Check if the mounting directory exists and is empty
    if [ ! -d "$MOUNT_POINT" ] || [ -z "$(ls -A $MOUNT_POINT)" ]; then
        echo "Problem: The directory $MOUNT_POINT is missing or empty. Attempting to remount..."

        # Unmount if anything is currently mounted
        if mountpoint -q $MOUNT_POINT; then
            echo "Unmounting $MOUNT_POINT..."
            fusermount -u $MOUNT_POINT
            sleep 5
        fi

        # Remount
        echo "Mounting SSHFS: $user_remote@$remote_host:$remote_path to $MOUNT_POINT..."
        echo $password | sshfs $user_remote@$remote_host:$remote_path $MOUNT_POINT -o password_stdin

        # Check if the mounting was successful
        if mountpoint -q $MOUNT_POINT; then
            echo "QNAP successfully mounted."
        else
            echo "Error: Failed to mount $remote_host:$remote_path to $MOUNT_POINT."
            exit 1
        fi
    fi
}

# Check and mount the disk
check_qnap

# Begin deleting files
current_usage=$(df --output=pcent "$MOUNT_POINT" | tail -n 1 | tr -d '%')
echo "Current usage: $current_usage%"
target_free_space=$((100 - $TARGET_USAGE))

# Continue deleting files until the required free space is achieved
while [[ $current_usage -gt $TARGET_USAGE ]]; do
    # Find the oldest file in the target directories
    oldest_file=$(find $TARGET_DIRS -type f -printf '%T+ %p\n' | sort | head -n 1 | cut -d' ' -f2)

    if [ -z "$oldest_file" ]; then
        echo "No files to delete."
        break
    fi

    # Delete the oldest file
    echo "Deleting file $oldest_file..."
    rm -f "$oldest_file"

    # Check usage again
    current_usage=$(df --output=pcent "$MOUNT_POINT" | tail -n 1 | tr -d '%')
    echo "New usage after deletion: $current_usage%"
done

echo "Target disk usage achieved."

Example Script Execution:

The script will run until it reaches 70% usage as planned:

Downloading the script and adding it to crontab

Of course, the script should be adjusted to meet your specific needs. However, if you want to download it and add it to crontab, follow these steps:

wget soban.pl/bash/remove_old_files.sh
chmod +x remove_old_files.sh

If you want to automate the file removal process, for example, at the end of the day, add the following entry to crontab:

crontab -e

In this case, the script will run every day at 11:55 PM:

45 23 * * * /root/scripts/remove_old_files.sh > /dev/null 2>&1

Make sure to use the correct path to the script.

Security and Optimization

The script uses a password directly in the command line, which can pose a security risk. In practical applications, it is recommended to use more advanced authentication methods, such as SSH keys, which are more secure and do not require a plaintext password in the script. However, in the case of QNAP, we used a password when writing this script.

Conclusion

The presented script is an example of how daily administrative tasks, such as disk space management, can be automated, thus increasing efficiency and reliability. Its implementation in real IT environments can significantly streamline data management processes, especially in situations where quick response to changes in disk usage is critical.

Artykuł Automatic deletion of files on QNAP drive via SSHFS pochodzi z serwisu soban.

Expanding disk space in Linux virtual machines is a key aspect of server system management. In this article, we show how to effectively increase disk space using LVM and fdisk tools, based on real system data.

Preliminary Preparations

Before making changes to partitions and volumes, it is important to check the current state of the disks in the system. We will use the lsblk command to identify available disks and partitions.

lsblk

Here is an example of the lsblk command output on a machine:

NAME                      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0                       7:0    0 55.7M  1 loop /snap/core18/2829
sda                         8:0    0   42G  0 disk
├─sda1                      8:1    0  512M  0 part /boot/efi
├─sda2                      8:2    0    1G  0 part /boot
└─sda3                      8:3    0 40.5G  0 part
  └─ubuntu--vg-ubuntu--lv 253:0    0 78.5G  0 lvm  /
sdb                         8:16   0  350G  0 disk
└─sdb1                      8:17   0   60G  0 part
  └─ubuntu--vg-ubuntu--lv 253:0    0 78.5G  0 lvm  /
sr0                        11:0    1 1024M  0 rom

Creating Snapshots

Before making changes to disk configurations, it is advisable to create a snapshot of the LVM volumes to ensure data can be restored in case of unexpected issues.

lvcreate -L 20G -s -n my_snapshot /dev/ubuntu-vg/ubuntu-lv

Modifying Partitions

Next, we proceed to modify the partitions using fdisk. We remove the existing partition and then create a new one that utilizes the entire available space on disk sdb.

fdisk /dev/sdb

Saving Changes

After properly configuring the partitions, we use the w command in fdisk to save the changes and update the partition table.

Command (m for help): w

Executing pvscan

After modifying the partitions, we execute the pvscan command so the system can update information about available physical volumes.

pvscan

Configuring LVM

After saving changes to the partition table, we need to update the LVM configuration to include the new disk space. We use the lvextend command with automatic file system resizing.

lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv -r

Summary

Expanding disk space on a Linux virtual machine enhances performance and the availability of storage space. Thanks to the steps described, managing disk space in systems using LVM becomes simpler and more efficient.

Artykuł Expanding Storage Space in Linux: Step-by-Step Guide using LVM and fdisk pochodzi z serwisu soban.

In today’s world, where data is becoming increasingly valuable, proper backup management is crucial for the security of information systems. In this article, I present an effective way to automate the backup of key configuration files in Proxmox-based systems using a simple bash script and Crontab configuration.

Bash Script for Backup of the /etc Directory

The /etc file contains critical system configuration files that are essential for the proper functioning of the operating system and various applications. Loss or damage to these files can lead to serious problems. Below, I present an effective script, backup-etc.sh, that allows for the automated backup of this directory:

#!/bin/bash
date_now=$(date +%Y_%m_%d-%H_%M_%S)
tar --use-compress-program zstd -cf /var/lib/vz/dump/vz-etc-$(hostname)-$date_now.tar.zst /etc/
find /var/lib/vz/dump/ -name vzup1-etc-* -type f -mtime +100 | xargs rm -f

This script performs the following operations:

1. Generates the current date and time, which are added to the name of the archive to easily identify individual copies.
2. Uses the tar program with zstd compression to create an archived and compressed copy of the /etc directory.
3. Removes archives older than 100 days from the /var/lib/vz/dump/ location, thus ensuring optimal disk space management.

Adding Script to Crontab

To automate the backup process, the script should be added to crontab. Below is a sample configuration that runs the script daily at 2:40 AM:

# crontab -e
40 2 * * * /root/backup-etc.sh > /dev/null 2>&1

Redirecting output to /dev/null ensures that operations are performed quietly without generating additional output to standard output.

Download the Script from soban.pl

The backup-etc.sh script is also available for download from the soban.pl website. You can download it using the following wget command and immediately save it as /root/backup-etc.sh:

# wget -O /root/backup-etc.sh https://soban.pl/bash/backup-etc.sh && chmod +x /root/backup-etc.sh

With this simple command, the script is downloaded from the server and granted appropriate executable permissions.

Benefits and Modifications

The backup-etc.sh script is flexible and can be easily modified to suit different systems. It is default placed in the /var/lib/vz/dump/ folder, which is a standard backup storage location in Proxmox environments. This simplifies backup management and can be easily integrated with existing backup solutions.

By keeping backups for 100 days, we ensure a balance between availability and disk space management. Old copies are automatically deleted, minimizing the risk of disk overflow and reducing data storage costs.

Summary

Automating backups using a bash script and Crontab is an effective method to secure critical system data. The backup-etc.sh script provides simplicity, flexibility, and efficiency, making it an excellent solution for Proxmox system administrators. I encourage you to adapt and modify this script according to your own needs to provide even better protection for your IT environment.

Artykuł Automating the Backup Process in Proxmox: Practical Crontab Script and Configuration pochodzi z serwisu soban.

Upgrading Apache Cassandra to a newer version is a significant task that database administrators undertake to ensure their systems benefit from new features, enhanced security measures, and improved performance. This guide provides a detailed walkthrough for upgrading Apache Cassandra from version 3.1.15 and higher to the latest 4.1.x version, specifically on Ubuntu 20.04.5 LTS, with an emphasis on pre-upgrade cleaning operations to manage disk space effectively.

Pre-upgrade Preparation

Backup Configuration Directory:

Before initiating the upgrade, it’s crucial to back up the Cassandra configuration directory. This precaution allows for a swift restoration of the configuration should any issues arise during the upgrade process. Utilize the following command to create a backup, incorporating the current date into the folder name for easy identification:

# cp -r /etc/cassandra/ /root/cassandra-conf-bkp-$(date +%Y%m%d)

Pre-Cleanup Operations

Preparation is key to a smooth upgrade. Begin with maintenance commands to guarantee data integrity and optimize space usage, especially important for systems with limited disk space.

Scrub Data:

Execute nodetool scrub to clean and reorganize data on disk. Given that this operation may be time-consuming, particularly for databases with large amounts of data or limited disk space, it’s a critical step for a healthy upgrade process.

Clear Snapshots:

To further manage disk space, use nodetool clearsnapshot to remove existing snapshots, freeing up space for the upgrade process. To delete all snapshots on the node, simply use this method if you’re running out of space:

# nodetool clearsnapshot --all

Cleanup Data:

Perform a nodetool cleanup to purge unnecessary data. In scenarios where disk space is a premium, it’s advisable to execute a scrub operation without generating a snapshot to conserve space:

# nodetool scrub --no-snapshot

Draining and Stopping Cassandra

Drain the Node:

Prior to halting the Cassandra service, ensure all data in memory is flushed to disk with nodetool drain.

# nodetool drain

Stop the Cassandra Service:

Cease the running Cassandra services to proceed with the upgrade safely:

# systemctl stop cassandra.service

Upgrading Cassandra

Update Source List:

Edit the repository sources to point to the new version of Cassandra by adjusting the cassandra.sources.list file:

# echo "deb https://debian.cassandra.apache.org 41x main" > /etc/apt/sources.list.d/cassandra.sources.list

Upgrade Packages:

With the repository sources updated, refresh the package list and upgrade the packages. When executing the apt upgrade command, you can keep pressing Enter as the default option is ‘N’ (No):

# apt update && apt upgrade

Modify Configuration:

Adjust the Cassandra configuration for version 4.1.x by commenting out or deleting deprecated options:

# for var in thrift_prepared_statements_cache_size_mb start_rpc rpc_port rpc_server_type thrift_framed_transport_size_in_mb request_scheduler; do sed -i "/$var:/s/^/#/" /etc/cassandra/cassandra.yaml; done

Update JAMM Library:

Ensure the Java Agent Memory Manager (JAMM) library is updated to enhance performance:

# sed -i 's|jamm-0.3.0.jar|jamm-0.3.2.jar|g' /etc/cassandra/cassandra-env.sh

Backup and update the JVM options file:

It’s a good practice to back up configuration files before making changes. This step renames the existing jvm-server.options file to jvm-server.options.orig as a backup. Then, it copies the jvm.options file to jvm-server.options to apply the standard JVM options for Cassandra servers.

# cd /etc/cassandra/
# mv jvm-server.options jvm-server.options.orig && cp -p jvm.options jvm-server.options

Optimization and Verification

Optimize Memory Usage:

Post-upgrade, it’s beneficial to evaluate and optimize memory usage and swap space to ensure efficient Cassandra operation:

# swapoff -a && swapon -a

Restart the Cassandra Service:

Apply the new version by restarting the Cassandra service:

# systemctl start cassandra.service

Verify Upgrade:

Confirm the success of the upgrade by inspecting the cluster’s topology and state, ensuring all nodes are functional:

# nodetool describecluster
# nodetool status

By adhering to this comprehensive guide, database administrators can effectively upgrade Apache Cassandra to version 4.1.x, capitalizing on the latest advancements and optimizations the platform has to offer, while ensuring data integrity and system performance through careful pre-upgrade preparations.

Optimization and Verification

After successfully upgrading Apache Cassandra to version 4.1.x and ensuring the cluster is fully operational, it’s crucial to conduct post-upgrade maintenance to optimize the performance and security of your database system. This section outlines essential steps and considerations to maintain a healthy and efficient Cassandra environment.

Monitor Performance and Logs

In the immediate aftermath of the upgrade, closely monitor the system’s performance, including CPU, memory usage, and disk I/O, to identify any unexpected behavior or bottlenecks. Additionally, review the Cassandra system logs for warnings or errors that may indicate potential issues requiring attention.

Tune and Optimize

Based on the performance monitoring insights, you may need to adjust Cassandra’s configuration settings for optimal performance. Consider tuning parameters related to JVM options, compaction, and read/write performance, keeping in mind the specific workload and data patterns of your application.

Run nodetool upgradesstables

To ensure that all SSTables are updated to the latest format, execute nodetool upgradesstables on each node in the cluster. This operation will rewrite SSTables that are not already in the current format, which is essential for taking full advantage of the improvements and features in Cassandra 4.1.x (Check the space, and if required, delete all snapshots as shown above.):

# time nodetool upgradesstables

This process can be resource-intensive and should be scheduled during off-peak hours to minimize impact on live traffic.

Implement Security Enhancements

Cassandra 4.1.x includes several security enhancements. Review the latest security features and best practices, such as enabling client-to-node encryption, node-to-node encryption, and advanced authentication mechanisms, to enhance the security posture of your Cassandra cluster.

Review and Update Backup Strategies

With the new version in place, reassess your backup strategies to ensure they are still effective and meet your recovery objectives. Verify that your backup and restore procedures are compatible with Cassandra 4.1.x and consider leveraging new tools or features that may have been introduced in this release for more efficient data management.

Artykuł Upgrading Apache Cassandra from Version 3.1.15 and Higher to 4.1.x on Ubuntu 20.04.5 LTS: A Comprehensive Guide pochodzi z serwisu soban.

As of this writing, the addressing is as follows:
new_machine IP: 10.10.14.100
old_machine IP: 10.10.14.101
Traffic will be routed on port 443 from new_machine to old_machine.

Before we go to proxy configuration, please make sure there are network transitions from new_machine (10.10.14.100) to old_machine (10.10.14.101) to port 443. You can read how to verify network connections here: check network connection and open tcp port via netcat.

We go to the installation of apache and mod_proxy:

# yum install httpd mod_proxy

After installing apache, go to the edition:

# vi /etc/httpd/conf.d/ssl.conf

Below are the news on the check level, what are the updates, and ip on the next service update:

<VirtualHost _default_:443>
SSLProtocol -all +TLSv1.2
SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:!NULL:!RC4:!RC2:!DES:!3DES:!SHA:!SHA256:!SHA384:!MD5+HIGH:+MEDIUM:!KRB5
#…
<IfModule mod_proxy.c>
SSLProxyProtocol all
SSLProxyEngine on
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
ProxyRequests On
    <Proxy *>
        Require all granted
    </Proxy>
    # backend server and forwarded path
    ProxyPass / https://10.10.14.101/
    ProxyPassReverse / https://10.10.14.101/
</IfModule>
</VirtualHost>

In order to verify the correctness of apache configuration, you can issue a command that will check it:

# service httpd configtest

If the apache configuration is correct, we can proceed to reloading apache:

# service httpd restart

At this point, we have a configured proxy connection. Before we move on to limiting traffic with iptables, I suggest you go to the site – with the new mod_proxy configured and test if everything is working properly and if there are any problems with the application.

Once everything is working fine, the network transitions are there, we can go to the iptables configuration for red hat 5. Let’s start by checking the system version:

# cat /etc/redhat-release

Now we are going to prepare iptables so that the network traffic is available on port 443 from the new_machine (10.10.14.100). To do this, edit the file /etc/sysconfig/iptables:

# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp --dport 443 -s 10.10.14.100 -j ACCEPT
-A INPUT -p tcp --dport 443 -j DROP
-A INPUT -p tcp --dport 80 -j DROP
-A INPUT -j ACCEPT
COMMIT

After iptables settings are correct, we can reload the service:

# /etc/init.d/iptables restart

In this way, we managed to cover up the weak encryption by proxying and diverting traffic to the new machine. This is not a recommended solution and you should try to transfer the application to a new environment compatible with the new system. However, in crisis situations, we can use this solution. Network traffic is not allowed by other IP addresses, so scanners will not be able to detect weak encryption on the old machine, and users using the old environment will not be able to use it. This does not change the fact that weak encryption is still set in the old environment and needs to be corrected. The example I gave is for the old red hat 5 and the new oracle linux 7, but it can be assumed that a similar solution and configuration is possible for other versions of the system.

Artykuł Improving encryption on old red hat 5 by new Oracle Linux 7 using apache mod_proxy pochodzi z serwisu soban.