Archive

Articles taggués ‘shell’

How to sniff HTTP traffic from the command line on Linux

01/12/2021 Aucun commentaire

sniff HTTP trafficSuppose you want to sniff live HTTP web traffic (i.e., HTTP requests and responses) on the wire for some reason. For example, you may be testing experimental features of a web server. Or you may be debugging a web application or a RESTful service. Or you may be trying to troubleshoot PAC (proxy auto config) or check for any malware files surreptitiously downloaded from a website. Whatever the reason is, there are cases where HTTP traffic sniffing is helpful, for system admins, developers, or even end users.

While packet sniffing tools such as tcpdump are popularly used for live packet dump, you need to set up proper filtering to capture only HTTP traffic, and even then, their raw output typically cannot be interpreted at the HTTP protocol level so easily. Real-time web server log parsers such as ngxtopprovide human-readable real-time web traffic traces, but only applicable with a full access to live web server logs.

What will be nice is to have tcpdump-like sniffing tool, but targeting HTTP traffic only. In fact, httpryis extactly that: HTTP packet sniffing tool. httpry captures live HTTP packets on the wire, and displays their content at the HTTP protocol level in a human-readable format. In this tutorial, let’s see how we can sniff HTTP traffic with httpry.

Install httpry on Linux

On Debian-based systems (Ubuntu or Linux Mint), httpry is not available in base repositories. So build it from the source:

$ sudo apt-get install gcc make git libpcap0.8-dev
$ git clone https://github.com/jbittel/httpry.git
$ cd httpry
$ make
$ sudo make install

On Fedora, CentOS or RHEL, you can install httpry with yum as follows. On CentOS/RHEL, enableEPEL repo before running yum.

$ sudo yum install httpry

If you still want to build httpry from the source on RPM-based systems, you can easily do that by:

$ sudo yum install gcc make git libpcap-devel
$ git clone https://github.com/jbittel/httpry.git
$ cd httpry
$ make
$ sudo make install

Basic Usage of httpry

The basic use case of httpry is as follows.

$ sudo httpry -i <network-interface>

httpry then listens on a specified network interface, and displays captured HTTP requests/responses in real time.

In most cases, however, you will be swamped with the fast scrolling output as packets are coming in and out. So you want to save captured HTTP packets for offline analysis. For that, use either ‘-b’ or ‘-o’ options. The ‘-b’ option allows you to save raw HTTP packets into a binary file as is, which then can be replayed with httpry later. On the other hand, ‘-o’ option saves human-readable output ofhttpry into a text file.

To save raw HTTP packets into a binary file:

$ sudo httpry -i eth0 -b output.dump

To replay saved HTTP packets:

$ httpry -r output.dump

Note that when you read a dump file with ‘-r’ option, you don’t need root privilege.

To save httpry‘s output to a text file:

$ sudo httpry -i eth0 -o output.txt

Advanced Usage of httpry

If you want to monitor only specific HTTP methods (e.g., GET, POST, PUT, HEAD, CONNECT, etc), use ‘-m’ option:

$ sudo httpry -i eth0 -m get,head

If you downloaded httpry‘s source code, you will notice that the source code comes with a collection of Perl scripts which aid in analyzing httpry‘s output. These scripts are found in httpry/scripts/plugins directory. If you want to write a custom parser for httpry‘s output, these scripts can be good examples to start from. Some of their capabilities are:

  • hostnames: Display a list of unique host names with counts.
  • find_proxies: Detect web proxies.
  • search_terms: Find and count search terms entered in search services.
  • content_analysis: Find URIs which contain specific keywords.
  • xml_output: Convert output into XML format.
  • log_summary: Generate a summary of log.
  • db_dump: Dump log file data into a MySQL database.

Before using these scripts, first run httpry with ‘-o’ option for some time. Once you obtained the output file, run the scripts on it at once by using this command:

$ cd httpry/scripts
$ perl parse_log.pl -d ./plugins <httpry-output-file>

You may encounter warnings with several plugins. For example, db_dump plugin may fail if you haven’t set up a MySQL database with DBI interface. If a plugin fails to initialize, it will automatically be disabled. So you can ignore those warnings.

After parse_log.pl is completed, you will see a number of analysis results (*.txt/xml) in httpry/scripts directory. For example, log_summary.txt looks like the following.

To conclude, httpry can be a life saver if you are in a situation where you need to interpret live HTTP packets. That might not be so common for average Linux users, but it never hurts to be prepared. What do you think of this tool?

Source: Xmodulo

Categories: Système Tags: , , ,

How to access a Linux server behind NAT via reverse SSH tunnel

01/12/2021 Aucun commentaire

reverse sshYou are running a Linux server at home, which is behind a NAT router or restrictive firewall. Now you want to SSH to the home server while you are away from home. How would you set that up? SSH port forwarding will certainly be an option. However, port forwarding can become tricky if you are dealing with multiple nested NAT environment. Besides, it can be interfered with under various ISP-specific conditions, such as restrictive ISP firewalls which block forwarded ports, or carrier-grade NAT which shares IPv4 addresses among users.

What is Reverse SSH Tunneling?

One alternative to SSH port forwarding is reverse SSH tunneling. The concept of reverse SSH tunneling is simple. For this, you will need another host (so-called “relay host”) outside your restrictive home network, which you can connect to via SSH from where you are. You could set up a relay host using a VPS instance with a public IP address. What you do then is to set up a persistent SSH tunnel from the server in your home network to the public relay host. With that, you can connect “back” to the home server from the relay host (which is why it’s called a “reverse” tunnel). As long as the relay host is reachable to you, you can connect to your home server wherever you are, or however restrictive your NAT or firewall is in your home network.

Reverse SSH Tunneling

 

Set up a Reverse SSH Tunnel on Linux

Let’s see how we can create and use a reverse SSH tunnel. We assume the following. We will be setting up a reverse SSH tunnel from homeserver to relayserver, so that we can SSH to homeserver via relayserver from another computer called clientcomputer. The public IP address of relayserver is 1.1.1.1.

On homeserver, open an SSH connection to relayserver as follows.

homeserver~$ ssh -fN -R 10022:localhost:22 relayserver_user@1.1.1.1

Here the port 10022 is any arbitrary port number you can choose. Just make sure that this port is not used by other programs on relayserver.

The “-R 10022:localhost:22” option defines a reverse tunnel. It forwards traffic on port 10022 of relayserver to port 22 of homeserver.

With “-fN” option, SSH will go right into the background once you successfully authenticate with an SSH server. This option is useful when you do not want to execute any command on a remote SSH server, and just want to forward ports, like in our case.

After running the above command, you will be right back to the command prompt of homeserver.

Log in to relayserver, and verify that 127.0.0.1:10022 is bound to sshd. If so, that means a reverse tunnel is set up correctly.

relayserver~$ sudo netstat -nap | grep 10022
tcp      0    0 127.0.0.1:10022          0.0.0.0:*               LISTEN      8493/sshd           

Now from any other computer (e.g., clientcomputer), log in to relayserver. Then access homeserver as follows.

relayserver~$ ssh -p 10022 homeserver_user@localhost

One thing to take note is that the SSH login/password you type for localhost should be for homeserver, not for relayserver, since you are logging in to homeserver via the tunnel’s local endpoint. So do not type login/password for relayserver. After successful login, you will be on homeserver.

Lire la suite…

Categories: Sécurité, Système Tags: , ,

25 Useful Basic Commands of APT-GET and APT-CACHE for Package Management

27/11/2021 Aucun commentaire

This article explains how quickly you can learn to install, remove, update and search software packages using apt-get and apt-cache commands from the command line. This article provides some useful commands that will help you to handle package management in Debian/Ubuntu based systems.

APT-GET and APT-CACHE Commands

What is apt-get?

The apt-get utility is a powerful and free package management command line program, that is used to work with Ubuntu’s APT (Advanced Packaging Tool) library to perform installation of new software packages, removing existing software packages, upgrading of existing software packages and even used to upgrading the entire operating system.

What is apt-cache?

The apt-cache command line tool is used for searching apt software package cache. In simple words, this tool is used to search software packages, collects information of packages and also used to search for what available packages are ready for installation on Debian or Ubuntu based systems.

APT-CACHE – 5 Useful Basic Commands

1. How Do I List All Available Packages?

To list all the available packages, type the following command.

$ apt-cache pkgnames
esseract-ocr-epo
pipenightdreams
mumudvb
tbb-examples
libsvm-java
libmrpt-hmtslam0.9
libboost-timer1.50-dev
kcm-touchpad
g++-4.5-multilib
...

2. How Do I Find Out Package Name and Description of Software?

To find out the package name and with it description before installing, use the ‘search‘ flag. Using “search” with apt-cache will display a list of matched packages with short description. Let’s say you would like to find out description of package ‘vsftpd‘, then command would be.

$ apt-cache search vsftpd
vsftpd - lightweight, efficient FTP server written for security
ccze - A robust, modular log coloriser
ftpd - File Transfer Protocol (FTP) server
yasat - simple stupid audit tool

To find and list down all the packages starting with ‘vsftpd‘, you could use the following command.

$ apt-cache pkgnames vsftpd
vsttpd

3. How Do I Check Package Information?

For example, if you would like to check information of package along with it short description say (version number, check sums, size, installed size, category etc). Use ‘show‘ sub command as shown below.

$ apt-cache show netcat
Package: netcat
Priority: optional
Section: universe/net
Installed-Size: 30
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Ruben Molina <rmolina@udea.edu.co>
Architecture: all
Version: 1.10-40
Depends: netcat-traditional (>= 1.10-39)
Filename: pool/universe/n/netcat/netcat_1.10-40_all.deb
Size: 3340
MD5sum: 37c303f02b260481fa4fc9fb8b2c1004
SHA1: 0371a3950d6967480985aa014fbb6fb898bcea3a
SHA256: eeecb4c93f03f455d2c3f57b0a1e83b54dbeced0918ae563784e86a37bcc16c9
Description-en: TCP/IP swiss army knife -- transitional package
 This is a "dummy" package that depends on lenny's default version of
 netcat, to ease upgrades. It may be safely removed.
Description-md5: 1353f8c1d079348417c2180319bdde09
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu

4. How Do I Check Dependencies for Specific Packages?

Use the ‘showpkg‘ sub command to check the dependencies for particular software packages. whether those dependencies packages are installed or not. For example, use the ‘showpkg‘ command along with package-name.

$ apt-cache showpkg vsftpd
Package: vsftpd
Versions: 
2.3.5-3ubuntu1 (/var/lib/apt/lists/in.archive.ubuntu.com_ubuntu_dists_quantal_main_binary-i386_Packages)
 Description Language: 
                 File: /var/lib/apt/lists/in.archive.ubuntu.com_ubuntu_dists_quantal_main_binary-i386_Packages
                  MD5: 81386f72ac91a5ea48f8db0b023f3f9b
 Description Language: en
                 File: /var/lib/apt/lists/in.archive.ubuntu.com_ubuntu_dists_quantal_main_i18n_Translation-en
                  MD5: 81386f72ac91a5ea48f8db0b023f3f9b

Reverse Depends: 
  ubumirror,vsftpd
  harden-servers,vsftpd
Dependencies: 
2.3.5-3ubuntu1 - debconf (18 0.5) debconf-2.0 (0 (null)) upstart-job (0 (null)) libc6 (2 2.15) libcap2 (2 2.10) libpam0g (2 0.99.7.1) libssl1.0.0 (2 1.0.0) libwrap0 (2 7.6-4~) adduser (0 (null)) libpam-modules (0 (null)) netbase (0 (null)) logrotate (0 (null)) ftp-server (0 (null)) ftp-server (0 (null)) 
Provides: 
2.3.5-3ubuntu1 - ftp-server 
Reverse Provides:

Lire la suite…

Categories: Système Tags: , ,

Email notifications using Gmail SMTP server on Linux

18/11/2021 Comments off

Send email notifications using Gmail SMTP

email notifications gmailUsing Gmail SMTP: Suppose you want to configure a Linux app to send out email messages from your server or desktop. The email messages can be part of email newsletters, status updates (e.g., Cachet), monitoring alerts (e.g., Monit), disk events (e.g., RAID mdadm), and so on. While you can set up your own outgoing mail server to deliver messages, you can alternatively rely on a freely available public SMTP server as a maintenance-free option.

One of the most reliable free SMTP servers is from Google’s Gmail service. All you have to do to send email notifications within your app is to add Gmail’s SMTP server address and your credentials to the app, and you are good to go.

One catch with using Gmail’s SMTP server is that there are various restrictions in place, mainly to combat spammers and email marketers who often abuse the server. For example, you can send messages to no more than 100 addresses at once, and no more than 500 recipients per day. Also, if you don’t want to be flagged as a spammer, you cannot send a large number of undeliverable messages. When any of these limitations is reached, your Gmail account will temporarily be locked out for a day. In short, Gmail’s SMTP server is perfectly fine for your personal use, but not meant for commercial bulk emails.

With that being said, let me demonstrate how to use Gmail’s SMTP server in Linux environment.

Google Gmail SMTP Server Setting

If you want to send emails from your app using Gmail’s SMTP server, you need to adjust the security setting of the Gmail account to be used. Go to the Google account settings, and enable the option to allow less secure apps, which is off by default.

Then you will need to provide your app with the following details.

  • Outgoing mail server (SMTP server): smtp.gmail.com
  • Use authentication: yes
  • Use secure connection: yes
  • Username: your Gmail account ID (e.g., “alice” if your email is alice@gmail.com)
  • Password: your Gmail password
  • Port: 587 (TLS) or 465 (SSL)

Exact configuration syntax may vary depending on apps. In the rest of this tutorial, I will show you several useful examples of using Gmail SMTP server in Linux.

Send Emails from the Command Line

As the first example, let’s try the most basic email functionality: send an email from the command line using Gmail SMTP server. For this, I am going to use a command-line email client called Mutt.

First, install Mutt as follows.

For Debian-based system:

$ sudo apt-get install mutt

For Red Hat based system:

$ sudo yum install mutt

Create a mutt configuration file (~/.muttrc) and specify in the file Gmail SMTP server information as follows. Replace <gmail-id> with your own Gmail ID. Note that this configuration is for sending emails only (not receiving emails).

$ vi ~/.muttrc

set from = "<gmail-id>@gmail.com"
set realname = "Dan Nanni"
set smtp_url = "smtp://<gmail-id>@smtp.gmail.com:587/"
set smtp_pass = "<gmail-password>"

Now you are ready to send out an email using mutt:

$ echo "This is an email body." | mutt -s "This is an email subject" alice@yahoo.com

To attach a file in an email, use “-a” option:

$ echo "This is an email body." | mutt -s "This is an email subject" alice@yahoo.com -a ~/test_attachment.jpg

22239850784_5fb0988075_c

Using Gmail SMTP server means that the emails appear as sent from your Gmail account. In other words, a recipient will see your Gmail address as the sender’s address. If you want to use your domain as the email sender, you need to use Gmail SMTP relay service instead.

Another note is that if you are concerned about placing your SMTP password in the plain-text .muttrc file, you can encrypt the password using GnuPG.

Lire la suite…

Categories: Logiciel Tags: , ,

Append TimeStamp to file name

16/11/2021 Comments off

I need to create a shell script that appends a timestamp to existing file. I mainly use Mac OS X for development. Wanted to create the same on Mac Terminal.

Here are some basics on date command.

NAME

date -- display or set date and time

SYNOPSIS
date [-ju] [-r seconds] [-v [+|-]val[ymwdHMS]] ... [+output_fmt]
date [-jnu] [[[mm]dd]HH]MM[[cc]yy][.ss]
date [-jnu] -f input_fmt new_date [+output_fmt]
date [-d dst] [-t minutes_west]

Samples:

Script to append date stamp to file:

Categories: Système Tags: ,