Archiwa Nginx - soban

Proxy through nginx frontend to the second virtual server wordpress

soban — Sun, 07 Nov 2021 11:48:33 +0000

In a situation where we have one public IP address and we have many domains directed to that IP address, it is worth considering spreading the traffic to other servers. Proxmox, which allows you to create a pair of virtual machines, is perfect in such a situation. In my case, each virtual machine is separated and the traffic is broken down by nginx, which distributes the traffic to other servers. The virtual machine on my website will redirect traffic, I have the IP address for wordpress: 10.10.11.105 on port 80. In this case, no encryption is required, but the frontend itself, which manages the traffic, will present itself with encryption and security on port 443.

Two machines with the following configuration will participate throughout the process:
up-page IP: 10.10.14.200
soban-pl IP: 10.10.11.105

So let’s move on to the frontend that distributes traffic to other machines.
The frontend is done by linux debian 11 (bullseye), in addition, I have the following entry in the repository (/etc/apt/sources.list):

#...
deb http://nginx.org/packages/debian/ bullseye nginx
deb-src http://nginx.org/packages/debian/ bullseye nginx

To install nginx, run the following commands:

# apt update
# apt install nginx

You should make sure that the traffic from the frontend has the appropriate port 80 transitions. You can read how to check the network transitions here: Check network connection and open TCP port via netcat.

The configuration of the frontend that distributes the traffic is as follows (/etc/nginx/conf.d/soban.pl.ssl.conf):

upstream soban-pl-webservers {
    server 10.10.11.105:80;
}

server {
    if ($host = www.soban.pl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = soban.pl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80;
        server_name soban.pl www.soban.pl;
        return 301 https://soban.pl$request_uri;
}

server {
    listen 443 ssl http2;
    server_name  www.soban.pl;
    ssl_certificate /etc/letsencrypt/live/www.soban.pl/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/www.soban.pl/privkey.pem; # managed by Certbot
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    return 301 https://soban.pl$request_uri;
}

server {
    listen 443 ssl http2;
    server_name  soban.pl _;
    ssl_certificate /etc/letsencrypt/live/soban.pl/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/soban.pl/privkey.pem; # managed by Certbot
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
}

    location / {
        access_log /var/log/nginx/access-soban.pl.log;
        error_log /var/log/nginx/error-soban.pl.log;
        proxy_pass http://soban-pl-webservers;
        proxy_redirect https://soban-pl-webservers http://soban-pl-webservers;
        expires off;

        proxy_read_timeout       3500;
        proxy_connect_timeout    3250;

        proxy_set_header   X-Real-IP          $remote_addr;
        proxy_set_header   Host               $host;
        proxy_set_header   X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto  https;
        proxy_set_header   SSL_PROTOCOL $ssl_protocol;
        proxy_set_header   SSL_CLIENT_CERT $ssl_client_cert;
        proxy_set_header   SSL_CLIENT_VERIFY $ssl_client_verify;
        proxy_set_header   SSL_SERVER_S_DN $ssl_client_s_dn;

                proxy_set_header X-Scheme $scheme;
                proxy_ssl_session_reuse off;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
    }
      location ~ ^/(wp-admin|wp-login\.php) {
            auth_basic "Restricted";
            auth_basic_user_file /etc/nginx/conf.d/htpasswd;
           proxy_pass http://soban-pl-webservers;
           proxy_redirect https://soban-pl-webservers http://soban-pl-webservers;
           expires off;
           proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;
           proxy_set_header        Host            $host;
           proxy_set_header        X-Real-IP       $remote_addr;
           proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
      }

}

Configuration of the above-mentioned wordpress, additional authorization is also set when you try to log in to wp-admin, you can read about it here: More security wp-admin in nginx.

In the next step, check if the nginx configuration is correct by:

# service nginx configtest

If everything is fine, restart nginx:

# service nginx restart

In a virtual machine with nginx it should also be installed. This is the same as debian linux 11 (bullseye), so the respository should look like this:

#...
deb http://nginx.org/packages/debian/ bullseye nginx
deb-src http://nginx.org/packages/debian/ bullseye nginx

Just installing nginx looks the same as on a machine that acts as a proxy.

# apt update
# apt install nginx

All configuration is in /etc/nginx/conf.d/soban.pl.conf:

server {
    listen   80;

   client_max_body_size 20M;

    server_name soban.pl www.soban.pl;
    access_log /var/log/nginx/access-soban.pl.log; #access logi
    error_log /var/log/nginx/error-soban.log; # error logi
    port_in_redirect off;
    set_real_ip_from  10.10.11.105;
    real_ip_header    X-Forwarded-For;
    real_ip_recursive on;
       root /home/produkcja/wordpress/;
       index index.html index.php;

if ($host ~* ^www\.(.*))
{
    set $host_without_www $1;
    rewrite ^/(.*)$ $scheme://$host_without_www/$1 permanent;
}

    error_page 404 /index.php;


        location ~ \.php$ {
                root /home/produkcja/wordpress/; # dir where is wordpress
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass unix:/var/run/php/php-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

        }

        location / {
                try_files $uri $uri/ /index.php?$args;
        }
        location = /sitemap.xml {
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.xml$ "/index.php?xml_sitemap=params=$2" last;
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.xml\.gz$ "/index.php?xml_sitemap=params=$2;zip=true" last;
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.html$ "/index.php?xml_sitemap=params=$2;html=true" last;
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.html.gz$ "/index.php?xml_sitemap=params=$2;html=true;zip=true" last;
       }

location = /favicon.ico {
  return 204;
  access_log     off;
  log_not_found  off;
}

location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
expires 365d;
        }location ~* \.(pdf)$ {
expires 30d;
}

}

Also in this case, check the correctness of the nginx service configuration:

# service nginx configtest

Everything looks fine, so let’s move on to restarting the service:

# service nginx restart

If the whole configuration was done correctly, the page should be directed without encrypted traffic to the virtual machine with wordpress. A wordpress service with nginx is not the only one that can be hosted or proxied. We can direct traffic from nginx to e.g. jboss, apacha and all other web services. Of course, this requires a corresponding modification of the configuration presented above, but the general outline of the concept as an nginx proxy has been presented. You should also remember about the appropriate configuration of keys and certificates. In my case let’s encrypt works perfectly for this.

Artykuł Proxy through nginx frontend to the second virtual server wordpress pochodzi z serwisu soban.

More security wp-admin in nginx

soban — Thu, 21 Oct 2021 16:44:39 +0000

Some time ago I noticed that my wordpress hacks are being hacked by logging into the backend of the website. A bot or a hacker is trying to do this using a set of passwords. I decided to secure the website’s backend by requiring additional authentication. In nginxe we can set this up by:

location ~ ^/(wp-admin|wp-login\.php) {
           auth_basic "Restricted";
           auth_basic_user_file /etc/nginx/.htpasswd;
           proxy_pass http://upstream-webservers;
           proxy_redirect https://upstream-webservers http://upstream-webservers;
           expires off;
           proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;
           proxy_set_header        Host            $host;
           proxy_set_header        X-Real-IP       $remote_addr;
           proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
      }

We still need to provide the username for authorization and save to the file (/etc/nginx/.htpasswd) as we entered in the nginx configuration file. In “my_user_name”, replace the login of the user with which we will be authorized.:

# echo -n 'my_user_name:' >> /etc/nginx/.htpasswd

And the encrypted password has been set by openssl:

# openssl passwd -apr1 >> /etc/nginx/.htpasswd

Openssl will ask you to come up with a password and enter it twice:

As a result, we will get a file with an encrypted password:

Before reloading nginx, we do a configuration verification:

# service nginx configtest

If everything is set correctly, we should receive the following message:

Now we can restart the service nginx:

# service nginx restart

The final verification will be to log in to the backend (e.g. www.example-page-wordpress.pl/wp-admin/), as a result, we should be asked for the login and password that we created above:

This is a simple trick to protect your wordpress from bot attacks. However, it should be remembered that we do not share passwords with anyone and setting default usernames and simple passwords is asking for a problem.

Artykuł More security wp-admin in nginx pochodzi z serwisu soban.

Useful tricks to view and search logs

soban — Thu, 21 Oct 2021 13:12:36 +0000

It often happens that we have to catch a given message, e.g. “error” while browsing the logs. Alternatively, we look for the occurrence of a given phrase in the old files. Both “tail” and “grep” are very useful for this. Especially if the logs are set in verbal mode, where there is a lot of messages in the log. We can also exclude certain phrases after parsing the information set. It is enough to use grep properly.

Let’s start by looking at all nginx logs.

In this case, sorting from oldest to newest is very useful as we know where to find the newest log entries:

# cd /var/log/nginx/
# ls -ltr *

If we are interested in the latest data, we will focus on the access-soban.pl.log file.

I know that my website is monitored by uptimerobot.com and I would like to find out, for example, from what IP address the website gets a query, e.g. to add it to the firewall as trusted:

# grep -i uptime access-soban.pl.log

As you can see, in this case, the bot that is querying the server has the IP address: 208.115.191.21. If I wanted to see all calls from this IP address, I could view them this way:

# grep 208.115.199.21 access-soban.pl.log* | less

If I press (shift + g) I’ll go to the bottom of the log:

It is worth noting that in this case the file in which the query is located is also given.

Now suppose I would like the logs, but without the “uptimerobot“:

# grep -v uptimerobot access-soban.pl.log* | less

This way all queries containing the word “uptimerobots” were cut. We can of course diminish the output from the console more by adding “| grep -v” possibly. Let’s cut out “sitemap“:

# grep -v uptimerobot access-soban.pl.log* | grep -v sitemap | less

One handy thing is to direct the stream from the console output to a file. We do this as follows “/tmp/file.log“:

# grep -v uptimerobot access-soban.pl.log | grep -v sitemap >> /tmp/file.log

Additionally, we can pack the file:

# tar -zcf /tmp/file.log.tar.gz /tmp/file.log

After packing the file, we can send it to another person. Sensitive data, such as inquiries or logins, can be cut using grep, as we did above.

Now let’s move on to one of the most useful tools for watching live what happens when someone enters a page:

# tail -f *.log

At this point it is worth noting that we “caught” the logs from the files: “access-soban.pl.log” and “error-soban.pl.log”. However, the “error-soban.pl.log” log is empty, so its content is not shown below. However, if something came up, we would see the contents of the updated file on the console.

Useful at this point is to combine grep and tail. We’re assuming we don’t want uptimerobots to bump into our consoles while observing the logs, so we’re going to cut them like this:

# tail -f *.log | grep -v uptimerobots

The given examples can be modified in any way. I encourage you to use it in various combinations of tail and grep, especially in situations where erros/warning are repeated. Of course, not only in nginx logs you can use these commands. In all logs where we operate on text, be it system or application. Passing the text mentioned above is very helpful.

Artykuł Useful tricks to view and search logs pochodzi z serwisu soban.

Upgrade Debian GNU/Linux 10 (buster) to 11 (bullseye)

soban — Fri, 01 Oct 2021 13:26:37 +0000

Before we start upgrading the Debian system version, please make a snapshot or a possible backup of the system. Such a change entails significant changes that can damage the system. Of course, before you upgrade the system in production, it is best to upgrade to the testing environment first. The upgrade process affects the entire system. Services may not be available at this time. If the system is doing the hosting for your website then it may not be available!

Always bear in mind that the production environment is different from test environment, so I recommend that you do it carefully. A good practice is to keep a time interval between the upgrade of test and production environment, in my case it is a week. Remember not to make changes to production on Friday!

This is how we check the version of the system:

And now, we going to update OS, but before we do that – we will make copy of:

cp /etc/apt/sources.list /root/sources.list_buster

There is sorces setup for buster:

# cat /etc/apt/sources.list
deb http://ftp.debian.org/debian buster main contrib
deb http://ftp.debian.org/debian buster-updates main contrib
deb http://security.debian.org buster/updates main contrib
deb http://nginx.org/packages/debian/ buster nginx
deb-src http://nginx.org/packages/debian/ buster nginx

Before we upgrade the debian system version to 11, we need to do a full upgrade:

# apt update && apt upgrade -q -y && apt dist-upgrade -q -y && apt autoremove -y -q

Then we use sed to replace “buster” on “bullseye”:

# sed -i 's/buster/bullseye/g' /etc/apt/sources.list

There here’s the effect:

# cat /etc/apt/sources.list

To be sure, we will do a comparison of the backed up file:

And then we can do update:

# apt update && apt upgrade -q -y && apt dist-upgrade -q -y && apt autoremove -y -q

So that it is not too easy, to start the task in the form of:

Specifically about this error:

Err:5 http://security.debian.org bullseye/updates Release
  404  Not Found [IP: 199.232.150.132 80]
Reading package lists... Done
E: The repository 'http://security.debian.org bullseye/updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

To fix it, use any editor on the file (vi in my case):

# vi /etc/apt/sources.list

We remove line 4 – “deb http://security.debian.org buster/updates main contrib” as you can see, we go out and save :

Of course, any other editor like nano is also good for this case.
The contents of the /etc/apt/sources.list file now looks like this:

# cat /etc/apt/sources.list
deb http://ftp.debian.org/debian bullseye main contrib
deb http://ftp.debian.org/debian bullseye-updates main contrib
deb http://nginx.org/packages/debian/ bullseye nginx
deb-src http://nginx.org/packages/debian/ bullseye nginx

We can try to update the system again:

# apt update && apt upgrade -q -y && apt dist-upgrade -q -y && apt autoremove -y -q

Now you need get some the system upgrade process has started. You can go for a coffee, or not 😉
There will be questions.
And more information, albout apt listchanges:

Just press ‘q’ and enter.

In this case we press enter.

Services to restart:

Enter agien.

This is question, about ssh deamon configuration:

In my case, I press enter because I don’t want to make changes to its configuration.

If all went well, we can reboot the system:

# reboot

Now check the system:

Congratulations, we are on the new version of the system!
At this point, we can verify all services, for example whether the website is working properly. If it is OK, upgrade the production environment.

In my cases, I have problem PHP new version.
PHP have no persmision on nginx to user – when I try enter to page:

# tail -f /var/log/nginx/error-*.log 
unix:/var/run/php/php-fpm.sock failed (13: Permission denied)

So we need to fix that in this way:

# service php7.4-fpm status
* php7.4-fpm.service - The PHP 7.4 FastCGI Process Manager
     Loaded: loaded (/lib/systemd/system/php7.4-fpm.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-10-01 14:59:14 CEST; 3min 28s ago
       Docs: man:php-fpm7.4(8)
    Process: 289 ExecStartPost=/usr/lib/php/php-fpm-socket-helper install /run/php/php-fpm.sock /etc/php/7.4/fpm/pool.d/www.conf 74 (code=exited, status=0/S>
   Main PID: 116 (php-fpm7.4)
     Status: "Processes active: 0, idle: 2, Requests: 0, slow: 0, Traffic: 0req/sec"
        CPU: 364ms
     CGroup: /system.slice/php7.4-fpm.service
             |-116 php-fpm: master process (/etc/php/7.4/fpm/php-fpm.conf)
             |-287 php-fpm: pool www
             `-288 php-fpm: pool www

Oct 01 14:59:11 soban-pl systemd[1]: Starting The PHP 7.4 FastCGI Process Manager...
Oct 01 14:59:14 soban-pl systemd[1]: Started The PHP 7.4 FastCGI Process Manager.

There is something wrong with the permissions:

# grep www-data /etc/php/7.4/fpm/pool.d/www.conf
user = www-data
group = www-data
listen.owner = www-data
listen.group = www-data

Let’s make some backup:

# cp /etc/php/7.4/fpm/pool.d/www.conf /etc/php/7.4/fpm/pool.d/www.conf.backup.$(date +%F)

I will replace www-data to nginx:

# sed -i 's/www-data/nginx/g' /etc/php/7.4/fpm/pool.d/www.conf

Now it’s looks better:

# grep nginx /etc/php/7.4/fpm/pool.d/www.conf
user = nginx
group = nginx
listen.owner = nginx
listen.group = nginx

And restart services:

# service php7.4-fpm restart

Also checking status:

Another problem is that, mariadb was removed, so I install it agien:

# apt-get install mariadb-server mariadb-client -y

and that resolved all my problems.

To be 100% sure, I reloaded the entire machine.

Artykuł Upgrade Debian GNU/Linux 10 (buster) to 11 (bullseye) pochodzi z serwisu soban.

Expired domain nginx

soban — Mon, 27 Sep 2021 17:41:00 +0000

Script to help check if the domain is expiring (don’t forget change e-mail in script – now is soban@soban.pl):

$ cat /root/check_domain.sh
#!/bin/bash
# Domain name list - add your domainname here
email=soban@soban.pl
DOM=`grep server_name /etc/nginx/conf.d/* | awk '{print $3}' | sed 's/\.$//'`
DOMAIN=`ls /etc/nginx/conf.d/ | sed -e 's/\//g' | sed -e 's/\//g' | sed 's/\.$//' | sed 's/\.$//' && echo -e $DOM | sed 's/\;/''/g'`
DOM=`echo -e $DOMAIN | tr ' ' '\n' | sed 's/\;/''/g'| sort -u`
for d in $DOM
do
  date_exp=`whois $d  | egrep [0-9] | egrep -i 'Expiry|renewal|Expiration' | tr '.' "-" | head -1 | sed 's/^[^:]*://g' | awk '{print $1}'`
  if [ ! -z "$date_exp" ]; then
    left=`echo $(expr '(' $(date -d $date_exp +%s) - $(date +%s) + 86399 ')' / 86400)`
    echo "$d - `date -d $date_exp +%F` - $left days until expiration"
        if (( $left < 30 )); then
          echo "$d - `date -d $date_exp +%F` - $left days until expiration" |  mail -s "$d - DOMAIN EXPIRES!!" $email
        fi
  fi
  sleep 300
done

This way you can download the script and give it permission to run:

$ cd /root/
$ wget https://soban.pl/bash/check_domain.sh
$ chmod +x check_domain.sh

And now we can add it to crontab, on monday for example:

$ crontab -e
# checking domains every Monday
00 08 * * 1 date >> /root/expiring_domain && /root/check_domain.sh >> /root/expiring_domain

Artykuł Expired domain nginx pochodzi z serwisu soban.