Proceedings of ISP RAS

Test Readability Optimization in Context of Symbolic Execution

I.A. Yakimov (IKIT SFU, Krasnoyarsk, Russia)
A.S. Kuznetsov (IKIT SFU, Krasnoyarsk, Russia)


Software testing is a time consuming process. In general, software companies spend about 50% of development time on testing. On the other hand, lack of testing implies financial and other risks. In the case of systematic testing a suitable test suite should provide an appropriate code coverage. A lot of code-based test generators have been developed in order to provide systematic code coverage. Such tools tend to produce lots of almost unreadable test suites which hard to verify manually. This problem is formulated as a Human Oracle Cost Problem. This work introduces a method for readability optimization of automatically generated test suites in context of Symbolic Execution. It uses natural language model in order to optimze each test case. This conception has been applied first in the Search Based Testing paradyghm. In contrast of existing search based tool proposed DSE-based tool uses SMT-solver in order to incrementally improve readability of a single test for a single program path. To validate proposed method a tool on top of LLVM and CVC4 frameworks is created. Experimental evaluation on 12 string processing routins from the Linux repository shows that this method can improve test inputs gracefully.


code-based test generation, symbolic execution, human oracle cost, bigram language model


Proceedings of the Institute for System Programming, vol. 28, issue 5, 2016, pp. 227-238.

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2016-28(5)-14

Full text of the paper in pdf (in Russian) Back to the contents of the volume