Automated pseudo-live testing of firewall configuration enforcement

Network security devices such as firewalls and intrusion detection systems are constantly updated in their implementation to accommodate new features, performance standards and to utilize new hardware optimization. Reliable, yet practical, testing techniques for validating the configuration enforcement after every new software and firmware update become necessary to assure correct configuration realization. Generating random traffic to test the firewall configuration enforcement is not only inaccurate but also impractical as it requires an infeasible number of test cases for a reasonable testing coverage. In addition, in most cases the policies used during testing are manually generated or have limited configuration profiles. We present a framework for automatic testing of the firewall configuration enforcement using efficient and flexible policy and traffic generation. In a typical test session, a large set of different policies are generated based on the access-control list (ACL) grammar and according to custom profiles. Test packets are generated to particularly consider critical segments of the tested policies and to achieve high coverage of the testing space. We also describe our implementation of a fully-automated framework, which includes ACL grammar modeling, the policy generation, test cases generation, capturing and analyzing firewall output, and creating detailed test reports. Our evaluation results show that our security configuration testing is not only achievable but it also offers high coverage with significant degree of confidence.