Archive

Archives pour la catégorie ‘Tutoriel’

HowTo: The Ultimate Logrotate Command Tutorial with 10 Examples

19/01/2021 Aucun commentaire

Managing log files effectively is an essential task for Linux sysadmin.

In this article, let us discuss how to perform following log file operations using UNIX logrotateutility.

  • Rotate the log file when file size reaches a specific size
  • Continue to write the log information to the newly created file after rotating the old log file
  • Compress the rotated log files
  • Specify compression option for the rotated log files
  • Rotate the old log files with the date in the filename
  • Execute custom shell scripts immediately after log rotation
  • Remove older rotated log files

1. Logrotate Configuration files

Following are the key files that you should be aware of for logrotate to work properly.

/usr/sbin/logrotate – The logrotate command itself.

/etc/cron.daily/logrotate – This shell script executes the logrotate command everyday.

$ cat /etc/cron.daily/logrotate
#!/bin/sh

/usr/sbin/logrotate /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0

/etc/logrotate.conf – Log rotation configuration for all the log files are specified in this file.

 

$ cat /etc/logrotate.conf
weekly
rotate 4
create
include /etc/logrotate.d
/var/log/wtmp {
    monthly
    minsize 1M
    create 0664 root utmp
    rotate 1
}

/etc/logrotate.d – When individual packages are installed on the system, they drop the log rotation configuration information in this directory. For example, yum log rotate configuration information is shown below.

$ cat /etc/logrotate.d/yum
/var/log/yum.log {
    missingok
    notifempty
    size 30k
    yearly
    create 0600 root root
}

2. Logrotate size option: Rotate the log file when file size reaches a specific limit

If you want to rotate a log file (for example, /tmp/output.log) for every 1KB, create the logrotate.conf as shown below.

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        create 700 bala bala
        rotate 4
}

This logrotate configuration has following three options:

  • size 1k – logrotate runs only if the filesize is equal to (or greater than) this size.
  • create – rotate the original file and create the new file with specified permission, user and group.
  • rotate – limits the number of log file rotation. So, this would keep only the recent 4 rotated log files.

Before the logrotation, following is the size of the output.log:

$ ls -l /tmp/output.log
-rw-r--r-- 1 bala bala 25868 2010-06-09 21:19 /tmp/output.log

Now, run the logrotate command as shown below. Option -s specifies the filename to write the logrotate status.

$ logrotate -s /var/log/logstatus logrotate.conf

Note : whenever you need of log rotation for some files, prepare the logrotate configuration and run the logroate command manually.
After the logrotation, following is the size of the output.log:

$ ls -l /tmp/output*
-rw-r--r--  1 bala bala 25868 2010-06-09 21:20 output.log.1
-rwx------ 1 bala bala        0 2010-06-09 21:20 output.log

Eventually this will keep following setup of rotated log files.

  • output.log.4.
  • output.log.3
  • output.log.2
  • output.log.1
  • output.log

Please remember that after the log rotation, the log file corresponds to the service would still point to rotated file (output.log.1) and keeps on writing in it. You can use the above method, if you want to rotate the apache access_log or error_log every 5 MB.

Ideally, you should modify the /etc/logrotate.conf to specify the logrotate information for a specific log file.

Also, if you are having huge log files, you can use: 10 Awesome Examples for Viewing Huge Log Files in Unix

3. Logrotate copytruncate option: Continue to write the log information in the newly created file after rotating the old log file.

$ cat logrotate.conf
/tmp/output.log {
         size 1k
         copytruncate
         rotate 4
}

copytruncate instruct logrotate to creates the copy of the original file (i.e rotate the original log file) and truncates the original file to zero byte size. This helps the respective service that belongs to that log file can write to the proper file.

While manipulating log files, you might find the sed substitutesed delete tips helpful.

4. Logrotate compress option: Compress the rotated log files

If you use the compress option as shown below, the rotated files will be compressed with gzip utility.

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        copytruncate
        create 700 bala bala
        rotate 4
        compress
}

Output of compressed log file:

$ ls /tmp/output*
output.log.1.gz output.log

5. Logrotate dateext option: Rotate the old log file with date in the log filename

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        copytruncate
        create 700 bala bala
        dateext
        rotate 4
        compress
}

After the above configuration, you’ll notice the date in the rotated log file as shown below.

$ ls -lrt /tmp/output*
-rw-r--r--  1 bala bala 8980 2010-06-09 22:10 output.log-20100609.gz
-rwxrwxrwx 1 bala bala     0 2010-06-09 22:11 output.log

This would work only once in a day. Because when it tries to rotate next time on the same day, earlier rotated file will be having the same filename. So, the logrotate wont be successful after the first run on the same day.

Typically you might use tail -f to view the output of the log file in realtime. You can even combine multiple tail -f output and display it on single terminal.

6. Logrotate monthly, daily, weekly option: Rotate the log file weekly/daily/monthly

For doing the rotation monthly once,

$ cat logrotate.conf
/tmp/output.log {
        monthly
        copytruncate
        rotate 4
        compress
}

Add the weekly keyword as shown below for weekly log rotation.

$ cat logrotate.conf
/tmp/output.log {
        weekly
        copytruncate
        rotate 4
        compress
}

Add the daily keyword as shown below for every day log rotation. You can also rotate logs hourly.

$ cat logrotate.conf
/tmp/output.log {
        daily
        copytruncate
        rotate 4
        compress
}

7. Logrotate postrotate endscript option: Run custom shell scripts immediately after log rotation

Logrotate allows you to run your own custom shell scripts after it completes the log file rotation. The following configuration indicates that it will execute myscript.sh after the logrotation.

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        copytruncate
        rotate 4
        compress
        postrotate
               /home/bala/myscript.sh
        endscript
}

8. Logrotate maxage option: Remove older rotated log files

Logrotate automatically removes the rotated files after a specific number of days.  The following example indicates that the rotated log files would be removed after 100 days.

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        copytruncate
        rotate 4
        compress
        maxage 100
}

9. Logrotate missingok option: Dont return error if the log file is missing

You can ignore the error message when the actual file is not available by using this option as shown below.

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        copytruncate
        rotate 4
        compress
        missingok
}

10. Logrotate compresscmd and compressext option: Sspecify compression command for the log file rotation

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        copytruncate
        create
        compress
        compresscmd /bin/bzip2
        compressext .bz2
        rotate 4
}

Following compression options are specified above:

  • compress – Indicates that compression should be done.
  • compresscmd – Specify what type of compression command should be used. For example: /bin/bzip2
  • compressext – Specify the extension on the rotated log file. Without this option, the rotated file would have the default extension as .gz. So, if you use bzip2 compressioncmd, specify the extension as .bz2 as shown in the above example.
 
Categories: Système, Tutoriel Tags:

Comment configurer un pare-feu avec UFW sur Ubuntu 16.0

15/01/2021 Aucun commentaire

Introduction

UFW, ou Pare-feu simple, est une interface iptablesqui vise à simplifier le processus de configuration d’un pare-feu. Bien iptablesqu’il s’agisse d’un outil solide et flexible, il peut être difficile pour les débutants d’apprendre à l’utiliser pour configurer correctement un pare-feu. Si vous cherchez à commencer à sécuriser votre réseau et que vous ne savez pas quel outil utiliser, UFW peut être le bon choix pour vous.

Ce tutoriel vous montrera comment configurer un pare-feu avec UFW sur Ubuntu 16.04.

Conditions préalables

Pour suivre ce tutoriel, vous aurez besoin de:

UFW est installé par défaut sur Ubuntu. S’il a été désinstallé pour une raison quelconque, vous pouvez l’installer avec sudo apt-get install ufw.

Étape 1 – Utilisation d’IPv6 avec UFW (facultatif)

Ce didacticiel est écrit avec IPv4 à l’esprit, mais fonctionnera pour IPv6 aussi longtemps que vous l’activez. Si votre serveur Ubuntu a activé IPv6, assurez-vous que UFW est configuré pour prendre en charge IPv6 afin qu’il gère les règles de pare-feu pour IPv6 en plus d’IPv4. Pour ce faire, ouvrez la configuration UFW avec nanoou votre éditeur préféré.

sudo nano /etc/default/ufw

Assurez-vous ensuite que la valeur de IPV6est yes. Ça devrait ressembler à ça:

/ etc / default / ufw extrait
...
IPV6=yes
...

Enregistrez et fermez le fichier. Désormais, lorsque UFW est activé, il sera configuré pour écrire les règles de pare-feu IPv4 et IPv6. Cependant, avant d’activer UFW, nous voulons nous assurer que votre pare-feu est configuré pour vous permettre de vous connecter via SSH. Commençons par définir les politiques par défaut.

Lire la suite…
Categories: Réseau, Sécurité, Tutoriel Tags:

Apache Web Server Hardening & Security Guide

02/01/2021 Comments off

apache security best practicesSecure Apache Web Server – Practical Guide

1       Introduction

The Web Server is a crucial part of web-based applications. Apache Web Server is often placed at the edge of the network hence it becomes one of the most vulnerable services to attack. Having default configuration supply many sensitive information which may help hacker to prepare for an attack the web server.

The majority of web application attacks are through XSS, Info Leakage, Session Management and PHP Injection attacks which is due to weak programming code and failure to sanitize web application infrastructure. According to the security vendor Cenzic, 96% of tested applications have vulnerabilities. Below chart from Cenzic shows the vulnerability trend report of 2013.

This practical guide provides you the necessary skill set to secure Apache Web Server.  In this course, we will talk about how to Harden & Secure Apache Web Server on Unix platform. Following are tested on Apache 2.4.x and I don’t see any reason it won’t work with Apache 2.2.x.

  1. This assumes you have installed Apache on UNIX platform. If not, you can go through Installation guide. You can also refer very free video about how to Install Apache, MySQL & PHP.
  2. We will call Apache installation directory /opt/apache as $Web_Server throughout this course.
  3. You are advised to take a backup of existing configuration file before any modification.

1.1  Audience

This is designed for Middleware Administrator, Application Support, System Analyst, or anyone working or eager to learn Hardening & Security guidelines. Fair knowledge of Apache Web Server & UNIX command is mandatory. This is seven page guide, click on Next to proceed. You may navigate through table of contents at right hand side.

 

BONUS (Download in PDF Format): Apache HTTP Security & Hardening Guide

Lire la suite…

Tutorial: Using VMWare ESXi and PFsense as a network firewall/router

01/01/2021 Comments off

vmware esxi

Using VMWare ESXi and PFsense as a network firewall/router

In most networks, you will have dedicated hardware to function as your “edge” (firewall/router). This is typically for the best, but there are always cases where you can’t put out that dedicated hardware. Sometimes it’s for cost reasons and sometimes it’s for complexity. In my particular case, I was installing an ESXi server in a datacenter and only had 2 amps of power to work with, of which my server took up ~1.8amps at peak load. So cost came into play and we simply couldn’t afford to put in dedicated hardware that could push enough bits. In such cases, it is possible the setup ESXi on the network edge, in a reasonably secure fashion, with PFSense acting as a firewall.

vmware_vsphereThe most important requirement to this project is that your VMWare ESXi server has at least two network ports on it. One will be the WAN port, one will be the LAN port. Also throughout this tutorial I will use PFSense as my firewall/router OS of choice, however it is just an example that can be easily swapped out with any other virtualized firewall product. Some options include Palo Alto Networks, Fortinet, and even generic *NIX operating systems with the right forwarding/firewall setup.

Section 1 – VMWare Setup

Step 1 – Install & Connect to ESXi

  • You should already have ESXi setup and connected via the VSphere client on Windows.
  • It’s recommended that you static the IP address of the VMWare Management interface, if you’ve not done so already.
  • Go to Configuration > Networking
  • Rename the vSwitch interface you’re using to “LAN”
2015-08-25-18_23_50-esxi1
Step 2 – Add new interface
You want “Virtual Machine” type
2015-08-25-18_24_15-Add-Network-Wizard
Step 3 – Select NIC
You want to select your unused NIC (assuming you only have two)
2015-08-25-18_25_11-Add-Network-Wizard
Step 4 – Name it
This is your “WAN” interface
2015-08-25-18_25_35-Add-Network-Wizard
Step 5 – Confirm you’ve got two networks
You’ll notice that we’ve got two vSwitches now. The “LAN” switch has the Management network and is connected currently. The “WAN” switch has nothing, and the adapter is disconnected.
2015-08-25-18_26_06-VMware

Section 2 – Virtual Machine Setup

Step 1 – New VM2015-08-25-18_29_17-New-VM
Step 2 – Typical Setup2015-08-25-18_29_31-Create-New-Virtual-Machine
Step 3 – Name your VM2015-08-25-18_29_39-Create-New-Virtual-Machine
Step 4 – Select Datastore2015-08-25-18_29_46-Create-New-Virtual-Machine
Step 5 – OS Type
If you’re using PFSense, select “Other” and “FreeBSD 64bit”
2015-08-25-18_29_57-Create-New-Virtual-Machine
Step 6 – Two NICs
Unlike most VMs with 1 NIC, add 2 NICs to this VM.
Make sure one adapter is on “WAN” network and one adapter is on “LAN” network.
2015-08-25-18_30_18-Create-New-Virtual-Machine
Step 7 – Allocated HD
PFSense doesn’t need much space, but it should be allocated a 2:1 for swap (e.g. 4096 MB swap file for 2048 MB of RAM), plus some extra space for packages and logs may be useful.
2015-08-25-18_30_38-Create-New-Virtual-Machine
Step 8 – Edit before completion2015-08-25-18_30_46-Create-New-Virtual-Machine
Step 9 – Final settings
As this is my firewall, I want to make sure it is plenty fast. So I opted for 4 cores and 2 GB RAM. Also attach the CD drive to PFSense installer (be it datastore ISO or real USB/Optical drive).
2015-08-25-18_31_54-pfsense-Virtual-Machine-Properties
Step 8 – Verify Network
Hop back to Configuration > Networking and you should see something like this. Note: various VMs are all attached to the LAN vSwitch, however only PFsense VM is attached to both WAN & LAN (just like a real firewall).
2015-08-25-18_33_31-VMWare-Verify
Step 9 – VM Startup
Go to Configuration > VM Startup/Shutdown
Click Properties
2015-08-31-12_30_32-Store
Step 10 – Set PFSense to first boot order
You may have other VMs that you want to auto-start, but as this is your firewall, it should be the first to start.
2015-08-31-12_31_05-Virtual-Machine-Startup-and-Shutdown

Section 3 – PFSense

Step 1 – Install PFSense
Once you’ve installed PFSense, it will automatically configure its local interface to 192.168.1.1
pfsense-install1
Step 2 (Optional) – Change local network
You can reconfigure the local network either via web interface (at the aforementioned IP: http://192.168.1.1) or command line
pfsense-installer
Step 3 – Configure WAN
Again, this can be configured either via the web, or command line.
2015-08-31-12_19_39-pfSense-Interfaces_-WAN
Step 4 – Plug in WAN cable2015-08-19-13.59.53
Step 5 – Test
If you’ve got the ports configured properly (i.e. WAN hardware is WAN in VMWare and WAN in PFSense), you should be able to connect to the internet.
2015-08-31-12_27_35-pfSense-Status_-Dashboard

There are two big questions after building a setup like this, the first is security. Since PFSense is the host to provide an interface on the WAN, it should be the only method of ingress into your network. With no VMware management interface on the WAN, there should be no way for an outside party to access ESXi directly. I’ve used this setup successfully (and safely) before, as have others. However, you always need to balance your particular security concerns with the cost of dedicated devices.

The second question is remote management/maintenance/failure. Managing ESXi remotely is easy, if you setup a VPN on your PFSense VM. Without that (or similar) you will not be able to remotely manage the box (by design). But what happens if there is a failure either in the VMWare hardware or the PFSense virtual machine? That’s the big failing point of this setup – you’re down. If, for whatever reason, PFsense dies – your network is offline and you cannot remotely manage it. If this hardware is installed in a dateacenter, you’d need to either get in there yourself or remote hands reboot. Something to keep in mind when balancing the cost issue. OF course, if it’s local (say you use this at home), then it’s not such a big deal.
IMG_07121I will note that this is the setup I use in my home network, which doubles as my homelab. Having a VM for a firewall gives me a lot of flexibility, like adding an entirely separate vSwitched network for experimental VMs. I can also swap out the firewall VM for another one with next to no downtime. It also allows me to skip one more piece of hardware at home which would add to my otherwise hefty powerbill.

Source: obviate.io

Categories: Réseau, Sécurité, Tutoriel Tags:

MySQL database replication with Linux

01/12/2020 Comments off

MySQL database replication with Linux

Database replication is a technique where a given database is copied to one or more locations, so that the reliability, fault-tolerance or accessibility of the database can be improved. Replication can be snapshot-based (where entire data is simply copied over to another location), merge-based (where two or more databases are merged into one), or transaction-based (where data updates are periodically applied from master to slaves).

MySQL replication is considered as transactional replication. To implement MySQL replication, the master keeps a log of all database updates that have been performed. The slave(s) then connect to the master, read individual log entries, and perform recorded updates. Besides maintaining a transaction log, the master performs various housekeeping tasks, such as log rotation and access control. When new transactions occur and get logged on the master server, the slaves commit the same transactions on their copy of the master database, and update their position in the master server’s transaction log. This master-to-slave replication process is done asynchronously, which means that the master server doesn’t have to wait for the slaves to catch up. If the slaves are unable to connect to the master for a period of time, they will download and execute all pending transactions when connectivity is re-established.

Database replication allows one to have an exact copy of a live database of a master server at another remote server (slave server) without taking the master server offline. In case the master server is down or having any trouble, one can temporarily point database clients or DNS resolver to the slave server’s IP address, achieving transparent failover. It is must be noted that MySQL replication is not a backup solution. For example, if an unintended DELETE command gets executed in the master server by accident, the same transaction will mess up all slave servers.

In this article, we will demonstrate master-slave based MySQL replication on two Linux computers. Let’s assume that the IP addresses of master/slave servers are 192.168.2.1 and 192.168.2.2, respectively.

Setting up a Master MySQL Server

This part will explain the steps needed on the master server.

First, log in to MySQL, and create test_repl database.

$ mysql -u root -p
mysql> CREATE DATABASE test_repl;

Next, create a table inside test_repl database, and insert three sample records.

mysql> USE test_repl;
mysql> CREATE TABLE employee (EmployeeID int, LastName varchar(255), FirstName varchar(255), Address varchar(255), City varchar(255));
mysql> INSERT INTO employee VALUES(1,"LastName1","FirstName1","Address1","City1"),(2,"Lastname2","FirstName2","Address2","City2"),(3,"LastName3","FirstName3","Address3","City4");

After exiting the MySQL server, edit my.cnf file using your favorite text editor. my.cnf is found under /etc, or /etc/mysql directory.

# nano /etc/my.cnf

Add the following lines under [mysqld] section.

[mysqld]
id=1
log-bin=master-bin.log
do-db=test_repl
innodb_flush_log_at_trx_commit=1
sync_binlog=1

The server-id option assigns an integer ID (ranging from 1 to 2^23) to the master server. For simplicity, ID 1 and 2 are assigned to the master server and the slave server, respectively. The master server must enable binary logging (with log-bin option), which will activate the replication. Set the binlog-do-db option to the name of a database which will be replicated to the slave server. The innodb_flush_log_at_trx_commit=1 and sync_binlog=1options must be enabled for the best possible durability and consistency in replication.

After saving the changes in my.cnf, restart mysqld daemon.

# systemctl restart mysqld

or:

# /etc/init.d/mysql restart

Log in to the master MySQL server, and create a new user for a slave server. Then grant replication privileges to the new user.

mysql> CREATE USER repl_user@192.168.2.2;
mysql> GRANT REPLICATION SLAVE ON *.* TO repl_user@192.168.2.2 IDENTIFY BY 'repl_user_password';
mysql> FLUSH PRIVILEGES;

A new user for the slave server is repl_user, and its password is repl_user_password. Note that the master MySQL server must not bind to the loopback interface since a remote slave server needs to log in to the master server as repl_user. Check this tutorial to change MySQL server’s binding interface.

Finally, check the master server status by executing the following command on the server.

mysql> SHOW MASTER STATUS;

18157192466_b3cc2d5ced_o

Please note that the first and second columns (e.g., master-bin.000002 and 107) will be used by the slave server to perform master-to-slave replication.

Lire la suite…