Troubleshooting iptables
Source: microhowto.info
Content
Objective
To ensure that iptables has been correctly configured.
Background
iptables is a component of the Linux kernel that allows IPv4 traffic to be manipulated as it traverses the network stack. Its two main uses are:
- packet filtering (firewalling) and
- network address translation (NAT).
The behaviour of iptables is controlled by rules, each of which specifies the action to be taken if a packet meets a particular set of conditions. The rules are organised into chains, and the chains into tables. Chains may be either built-in or user-defined.
For more information about the architecture and configuration of iptables see:
- the netfilter/iptables project homepage,
- the iptables tutorial and
- the Packet Filtering and NAT HOWTOs
Symptoms
The most likely symptoms of an iptables configuration error are:
- traffic being dropped or rejected,
- traffic not being NATted when it should have been,
- traffic being NATted when it shouldn’t have been, or
- traffic being NATted to the wrong address.
A wide variety of other effects are possible, but unlikely unless the configuration is an unusual one.
Scenario
Suppose that a machine has been configured to act as a boundary router between a local area network (connected to interface eth0 with the address 192.168.0.1/24) and the public Internet (connected to interface ppp0 with the address 203.0.113.144/32). The default gateway is 203.0.113.1. Because the local area network uses a private address range, iptables on the boundary router has been configured to SNAT them to its public IP address.
In order to test this configuration you have attempted to ping a machine on the public Internet (198.51.100.1) from a machine on the local area network (192.168.0.2), but this has failed.
Investigation
Strategy
There are two possibilities to consider:
- that
iptableshas not been populated with the rules that you intended, or - that the rules are as intended but do not have the desired effect.
It is usually worth quickly checking the first point before launching into any detailed investigation of the second.
Remember that rule changes made using the iptables command are not persistent unless there is some external mechanism in place to make them persistent. Some distributions provide such a mechanism by default, others do not. Even with a persistence mechanism in place, rules will not necessarily survive a non-graceful shutdown.
If your configuration uses the DROP target (as opposed to REJECT), be aware that this will suppress ICMP packets which are perhaps the most useful source of diagnostic information produced by the network stack. While this may have some small benefit in terms of denying information to an attacker, it will most certainly make troubleshooting more difficult for you.
You may also find troubleshooting more difficult if you are using an automatic configuration tool, because of the extra layer of abstraction it introduces between you and the ruleset.
If you add any rules for diagnostic purposes, remember to remove them out afterwards.
Inspecting the tables
The content of a table can be listed by means of the -L option of the iptables command. For example, to list the POSTROUTING chain of the nat table:
iptables -t nat -L POSTROUTING
Alternatively, all chains in the nat table could be listed using the command:
iptables -t nat -L
If no table is specified using the -t option then this and other iptables commands default to the filter table.
By default IP addresses and port numbers are resolved into names wherever possible, which may or may not be helpful. You can request output in numeric form using the -n option.
Monitoring traffic flow using counters
Each iptables rule has two associated counters: one to record the number of matching packets, and one for the number of matching bytes. By inspecting these counters it is often possible to deduce whether a rule is having the desired effect.
Before performing a test it is usually helpful to zero the counters. This can be done using the -Z option of the iptables command. For example, to zero the counters of the nat table:
iptables -t nat -Z
The counters can then be inspected by adding the -v (verbose) qualifier to the -L option:
iptables -t nat -L -v -n
For the scenario above, if the system had been operating properly and you had performed ten successful pings then the output of this command might look similar to:
Chain PREROUTING (policy ACCEPT 10 packets, 840 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 10 840 SNAT all -- * * 192.168.0.0/24 0.0.0.0/0 to:203.0.113.144 Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination
If instead the postrouting chain had shown:
Chain POSTROUTING (policy ACCEPT 10 packets, 840 bytes)
pkts bytes target prot opt in out source destination
0 0 SNAT all -- * * 192.168.1.0/24 0.0.0.0/0 to:203.0.113.144
the counters would tell you that the SNAT rule is not being invoked, and that the packets are instead reaching the end of the chain (where they are ACCEPTed without modification by the chain policy).
Be aware that the nat table is traversed only by the first packet of each connection. The example above is unusual because every outbound packet is considered to be part of a separate connection. For TCP-based protocols you would typically see only a small fraction of the total traffic in the nat table counters.
It is obviously helpful if you can minimise the amount of extraneous traffic for the duration of the test, but sometimes this will be outside your control. In that case the best way to improve the quality of your data is to be more selective about what you count, perhaps by adding rules that are specifically designed to pick out your test traffic.
To achieve this you may need to insert rules purely for the purpose of counting packets. iptables does not have a ‘no operation’ target as such, but you can achieve the same effect by inserting a rule without a target. For example, the following would count the number of ICMP packets to or from the test machine in the scenario above as they enter the FORWARD chain:
iptables -t filter -I FORWARD 1 -p icmp -s 192.168.0.2 iptables -t filter -I FORWARD 2 -p icmp -d 192.168.0.2
Monitoring traffic flow using the LOG target
If more detail is needed then the LOG target can be used to record packets in the system log. For example, to log packets to or from the test machine in the scenario above:
iptables -t filter -I FORWARD 1 -p icmp -s 192.168.0.2 -j LOG iptables -t filter -I FORWARD 2 -p icmp -d 192.168.0.2 -j LOG
These rules will, of course, have counters associated with them, so you can use them to both count and log at the same time.
Where exactly the log messages are recorded to will depend on the configuration of syslog, but good places to look are /var/log/kern.log and /var/log/messages. The result will typically look similar to:
Dec 9 07:43:03 test-router kernel: [3363279.057635] IN=eth0 OUT=ppp0 SRC=192.168.0.2 DST=203.0.113.144 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=39285 SEQ=1 Dec 9 07:43:04 test-router kernel: [3363279.926618] IN=ppp0 OUT=eth0 SRC=203.0.113.144 DST=192.168.0.2 LEN=84 TOS=0x00 PREC=0x00 TTL=57 ID=47176 PROTO=ICMP TYPE=0 CODE=0 ID=39285 SEQ=1
You may find it helpful to use the --log-prefix option to label the log entry. This is particularly useful if the same packet might be logged several times, because otherwise you may have no way of telling which rule was responsible for the log entry. For example, to obtain full coverage of outbound traffic from the test machine as it passes through the filter and nat tables you could use the following three rules:
iptables -t nat -I PREROUTING 1 -s 192.168.0.2 -j LOG --log-prefix "ICMP (PREROUTING): " iptables -t filter -I FORWARD 1 -s 192.168.0.2 -j LOG --log-prefix "ICMP (FORWARD): " iptables -t nat -I POSTROUTING 1 -s 192.168.0.2 -j LOG --log-prefix "ICMP (POSTROUTING): "
You can log more information about each packet by appending the options --log-tcp-sequence, --log-tcp-options, --log-ip-options and (for locally-generated packets) --log-uid.