Databases, Systems & Networks

source: Paul’s blog

yum install fail2ban

logtarget = /var/log/fail2ban.log

Protecting Apache HTTP server with fail2ban

Fail2ban: An Apache HTTP server in production environments can be under attack in various different ways. Attackers may attempt to gain access to unauthorized or forbidden directories by using brute-force attacks or executing evil scripts. Some malicious bots may scan your websites for any security vulnerability, or collect email addresses or web forms to send spams to.

Apache HTTP server comes with comprehensive logging capabilities capturing various abnormal events indicative of such attacks. However, it is still non-trivial to systematically parse detailed Apache logs and react to potential attacks quickly (e.g., ban/unban offending IP addresses) as they are perpetrated in the wild. That is when fail2ban comes to the rescue, making a sysadmin‘s life easier.

fail2ban is an open-source intrusion prevention tool which detects various attacks based on system logs and automatically initiates prevention actions e.g., banning IP addresses with iptables, blocking connections via /etc/hosts.deny, or notifying the events via emails. fail2ban comes with a set of predefined “jails” which use application-specific log filters to detect common attacks. You can also write custom jails to deter any specific attack on an arbitrary application.

In this tutorial, I am going to demonstrate how you can configure fail2ban to protect your Apache HTTP server. I assume that you have Apache HTTP server and fail2ban already installed. Refer to another tutorial for fail2ban installation.

What is a Fail2ban Jail

Let me go over more detail on fail2ban jails. A jail defines an application-specific policy under which fail2ban triggers an action to protect a given application. fail2ban comes with several jails pre-defined in /etc/fail2ban/jail.conf, for popular applications such as Apache, Dovecot, Lighttpd, MySQL, Postfix, SSH, etc. Each jail relies on application-specific log filters (found in /etc/fail2ban/fileter.d) to detect common attacks. Let’s check out one example jail: SSH jail.

[ssh]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 6
banaction = iptables-multiport

This SSH jail configuration is defined with several parameters:

• [ssh]: the name of a jail with square brackets.
• enabled: whether the jail is activated or not.
• port: a port number to protect (either numeric number of well-known name).
• filter: a log parsing rule to detect attacks with.
• logpath: a log file to examine.
• maxretry: maximum number of failures before banning.
• banaction: a banning action.

Any parameter defined in a jail configuration will override a corresponding fail2ban-wide default parameter. Conversely, any parameter missing will be assgined a default value defined in [DEFAULT] section.

Predefined log filters are found in /etc/fail2ban/filter.d, and available actions are in /etc/fail2ban/action.d.

If you want to overwrite fail2ban defaults or define any custom jail, you can do so by creating /etc/fail2ban/jail.local file. In this tutorial, I am going to use /etc/fail2ban/jail.local.

Lire la suite…

In an operating system, logs are all about keeping track of events, be it critical system errors, resource usage warnings, transaction history, application status, or user activities. These logs, which are stored as (text or binary) files in the system, are useful for system auditing, debugging and maintenance. However, with so many different system entities generating log files, and even at growing rate, the challenge as a system admin is to how to “consume” these log files effectively.

That’s when log monitoring tools come into the picture, which streamline the often laborious process of collecting, parsing and analyzing log files, as well as alerting system admins for any interesting events. These tools are designed from ground up focused on log monitoring, so they offer many attractive features, such as scalable log aggregation and filtering, human-readable display, event correlation, visual or email notification, flexible log retention policy, and so on.

In this post, I am going to introduce a list of popular open-source log monitoring software for Linux, ranging from simple log file viewers to full-blown log monitoring frameworks.

Log Aggregation and Filtering

Log monitoring would not be possible without efficient and scalable mechanisms to collect and pre-process log files. Tools in this category focus on shipping, collecting, filtering, indexing and storing log files, so that they can be further analyzed and visualized in subsequent monitoring pipelines.

1. rsyslog: an open-source log collector server which can filter and consolidate log data (based on syslog protocol) from different hosts and devices in the network. rsyslog can be configured as a server or a client, where the former plays the role of a log collector and the latter runs as a log sender.

2. syslog-ng: another open-source implementation of the syslog protocol with more advanced and user-friendly features such as content-based filtering, easier-to-understand config format, and real-time event correlation.

3. systemd journal: systemd journal can be configured for remote journal logging, where locally logged events are forwarded to a remote server over HTTP. In this setup, systemd-journal-upload on a client host serializes and forwards journal messages to systemd-journal-remote running on a remote collector server.

4. logstash: an open-source tool that collects, parses, and stores log files for offline search and analysis. logstash can run in various pipelines due to many plugins supporting different input/output interfaces, decoding/encoding, and filtering rules. Input plugins allows logstash to gather log files from different sources and protocols (e.g., files, S3, RabbitMQ, syslog, collectd, TCP/UDP sockets). Filter/codec plugins are used to parse, convert, modify and add metadata to log files. Output plugins pass processed log files to various target storages (e.g., file, Google cloud storage, Nagios, S3, Zabbix).

5. collectd: a daemon service which gathers various system-level statistics, and stores them for historical analysis or real-time graphing. Similar to logstash, collectd is an extensible architecture, where you can enable various input/output plugins to change its collection behaviors. For log collection, collectd can leverage LogFile and Network plugins to aggregate remote log files.

6. Logster: an open-source utility for parsing log files for any interesting data, and aggregating extracted data into metrics for subsequent reporting and graphing pipelines.

7. Fluentd: a unified log aggregation layer which allows in-stream processing for a variety of streaming data and log files. It comes with a huge plugin ecosystem with more than 300 plugins to support various input sources and output interfaces.

8. Nxlog: a unified log collector and forwarder which supports a variety of log sources, formats and protocols. Advanced features include multi-threaded log collection and processing, message buffering and prioritization, built-in log rotation, and TLS/SSL transport.

9. Scribe:: a scalable log collector server developed by Facebook. Scribe can aggregate log data which is streamed in real time from a large number of clients. It uses Apache Thrift for protocol encoding, so its interface is compatible with pretty much any languages. While a proven solution, Scribe is not something you can deploy quickly as a turnkey. Also, note that Scribe is no longer updated and maintained.

10. Flume: a highly scalable and reliable service to transport and collect large volumes of streaming log data from any clients, and store them in backend storage such as Apache Hadoops’ HDFS.

Lire la suite…

One of the most daunting prospects of administering your own server on a public network is dealing with your server’s security. While security threats in a networked world are real and it is always important to be mindful of security issues, protecting against possible attacks is often a matter of exercising basic common sense and adhering to some general best practices.

This guide takes a broad overview of common security concerns and provides a number of possible solutions to common security problems. You are encouraged to consider deploying some of these measures to “harden” your server against possible attacks.

It’s important to remember that all of the solutions we present in this document are targeted at specific kinds of attacks, which themselves may be relevant only in specific configurations. Security solutions need to be tailored to the kind of services that you’re providing and the software you’re running, and the decision whether or not to deploy a specific security solution is often a matter of personal discretion and cost-benefit analysis.

Perhaps most importantly, it should be understood that security is a process, not a product (credit to Bruce Schneier.) There is no “magic bullet” set of guidelines that can be followed to ensure the security of any system. Threats are constantly evolving, so vigilance is required on the part of network administrators to prevent unauthorized access to systems.

Keep Systems and Software Up To Date

One of the most significant sources of security vulnerabilities are systems running out of date software with known security holes. Make a point of using your system’s package management tools to keep your software up to date; this will greatly assist in avoiding easily preventable security intrusions.

Running system updates with the package management tool, using apt-get update && apt-get upgrade (for Debian and Ubuntu Systems) or yum update (for CentOS and Fedora systems) is simple and straightforward. This practice ensures that if your distribution maintains active security updates, your system will be guarded against many security holes in commonly used software packages.

System update tools will, however, not keep software up to date that you’ve installed outside of package management. This includes software that you’ve compiled and installed “by hand” (e.g. with ./configure && make && make install) and web-based applications that you’ve installed from a software developer’s site, as is often the case with applications like WordPress and Drupal. Also excluded from protection will be libraries and packages you’ve installed with supplementary package management tools like Ruby’s Gems, Perl’s CPAN tool, Python easy_install, and Haskell Cabal. You will have to manage the process of keeping these files up to date yourself.

The method you use to make sure that your entire system is kept up to date is a matter of personal preference, and depends on the nature of your workflow. We would recommend trying very hard to use the versions of software provided by your operating system or other programming platform-specific package management tools. If you must install from “source,” we would recommend that you save the tarballs and source files for all such software in /src/ or ~/src/ so that you can keep track of what software you’ve installed in this manner. Often, you can remove a manually compiled application by issuing make uninstall in the source repository (directory). Additionally, it may be helpful to maintain a list of manually installed software, with version numbers and download locations. You may also want to investigate packaging your own software so that you can install it with apt, yum or pacman.

Because of the complexity of maintaining software outside of the system’s package management tools we strongly recommend avoiding manually installing software unless absolutely necessary. Your choice in a Linux distribution should be heavily biased by the availability of software in that distro’s repositories for the systems you need to run on your server.

Lire la suite…

Appropriate firewall rules heavily depend on the services being run. Below are iptables rulesets to secure your Linode if you’re running a web server. These are given as an example! A real production web server may want or require more or less configuration and these rules would not be appropriate for a file or database server, Minecraft or VPN server, etc.

iptables rules can always be modified or reset later, but these basic rulesets serve only as a beginning demonstration.

IPv4

/tmp/v4

*filter

# Allow all loopback (lo0) traffic and reject traffic
# to localhost that does not originate from lo0.
-A INPUT -i lo -j ACCEPT
-A INPUT ! -i lo -s 127.0.0.0/8 -j REJECT

# Allow ping and traceroute.
-A INPUT -p icmp --icmp-type 3 -j ACCEPT
-A INPUT -p icmp --icmp-type 8 -j ACCEPT
-A INPUT -p icmp --icmp-type 11 -j ACCEPT

# Allow SSH connections.
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT

# Allow HTTP and HTTPS connections from anywhere
# (the normal ports for web servers).
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT

# Accept inbound traffic from established connections.
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Log what was incoming but denied (optional but useful).
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7

# Reject all other inbound.
-A INPUT -j REJECT

# Log any traffic which was sent to you
# for forwarding (optional but useful).
-A FORWARD -m limit --limit 5/min -j LOG --log-prefix "iptables_FORWARD_denied: " --log-level 7

# Reject all traffic forwarding.
-A FORWARD -j REJECT

COMMIT


Optional: If you plan to use Linode Longview, add this additional rule below the section for allowing HTTP and HTTPS connections:

# Allow incoming Longview connections 
-A INPUT -s longview.linode.com -m state --state NEW -j ACCEPT

 

Lire la suite…