You need to plan to test both the implementation of the firewall system and the policy being implemented by the system. To test the implementation of the system:
- Create a list of all field-replaceable system components whose failure would significantly interfere with the ability of the firewall to meet its mission goals.
- For each such component, create a short list of the most likely modes of failure that would affect firewall operations. De-emphasise components whose failure modes are unlikely to occur or those whose failure modes are likely but their impact on firewall operations is unlikely.
- For each relevant failure mode
- Design a test scenario that would either directly cause or simulate the failure.
- Design a mitigation strategy that minimises the impact of the failure.
An example of a test scenario is to assume that the host system on which the firewall software runs has an unrecoverable hardware failure that prevents it from performing any packet forwarding, such as might occur if the network adapter failed. A way to test this failure might be to simply unplug the network from the interface to simulate the failure.
An example of a recovery strategy would be to maintain a totally redundant firewall system, switching to the redundant host when failures occur to minimise the time period during which packet transmittal is affected.
Testing the policy installed in the system is more difficult. It is not feasible to exhaustively test an IP packet filter configuration; there are too many possibilities. We recommend that instead of exhaustive tests, you use boundary tests. In these tests, you identify boundaries in your packet filter rules and then sample test the regions immediately adjacent to each boundary.
To do this for each rule, you identify every boundary in the rule. In general, each constrained parameter in a rule contributes either one or two boundaries. The space being partitioned is a multidimensional packet attribute space.
Common attributes include:
- protocols,
- source addresses,
- destination addresses,
- source ports, and
- destination ports.
Basically, every attribute of a packet that can be independently checked in a packet filter rule defines one dimension of this space.
For example, a rule that permits TCP packets from any host to your Web server host on port 80 has checked three attributes (protocol, destination address, and destination port) which partitions the attribute space into three regions: TCP packets to Web server at ports less than 80, port 80, and ports greater than 80.
For each region, you generate some test traffic that you have engineered to stay within that region. You verify that the firewall either rejects or forwards all traffic for a given region. Within a single region, all traffic should be rejected or forwarded; that is the purpose for partitioning the packet attribute space.
For a complex set of rules, this can be a tedious process and may not be practical. If it is not practical to test the rules, request that several people review them and ask one person to explain to the others what each rule does.
The test plan needs to include
- the test cases,
- configurations, and expected results for testing the routing configuration,
- packet filtering rules (including service-specific testing), and
- logging and alert options
- testing the firewall system as a whole (such as hardware/software failure recovery, sufficient log file sizing, proper archival of logs, performance monitoring)
- exercising both normal conditions and excursion (anomaly) conditions
You also need to describe the tools you intend to use (such as scanners, monitoring tools, and vulnerability detection tools) and the tests you intend to run using those tools.
