Coverage Analysis & Profiling with GCC

When developing software, at some point you should perform coverage analysis and profiling to help you work out where you should focus any optimisation efforts and where your test cases are lacking.

This article gives a quick introduction to coverage analysis and profiling using GCC.  These examples are from a current Debian Linux system using GCC 9.2.1 but the principles should work on most Unix platforms and GCC versions.

Coverage analysis

Coverage analysis is a technique that tells you exactly which parts of your program are being executed and how often. This information is valuable for helping you identify potential bottlenecks in your program. That way you can spend effort optimising the right parts of the program rather than optimising something that rarely gets used. In other words, it helps you improve your “bang for the buck” with optimisation. Normally this is done later in the development cycle; first get the program working correctly, and then worry about optimisation.

Another important use of coverage analysis is to help you improve your testing. Ideally your test harnesses (unit tests, integration tests, etc) should cover as much of the code as possible, so if you have meaningful coverage data you can improve your test code so that more of your code is tested.

However, don’t bother trying to cover 100% of the code with your test cases, you’ll find that it gets near impossible to test some situations (e.g. memory starvation, file errors). Use your judgement for what coverage is important to you, but without coverage analysis you’re flying blind.

Profiling

Profiling is a way of measuring exactly how much time is spent in each part of your program.  Whereas coverage analysis tells you how often a piece of code is executed, profiling tells you how much time is spent in each area.  So you typically use coverage analysis to help you improve your testing and profiling to help you improve performance.

Rather than wasting time trying to optimise obscure parts of your program, if you use profile information you can spend your efforts on where they will have the most benefit.

For a real-world example, I’ve been working on a fairly complex C++ application.  Rather than worry about minor performance details as I’ve progressed, I instead ran a repeatable test with profiling enabled once the code was mostly complete.  This showed that my uses of a particular std::set were much more frequent than I expected, and I was able to change this to instead use std::unordered_set which gave me a 20% improvement.

This shows how profiling the actual bottlenecks and fixing those is much better than making guesses and needlessly complicating the wrong code.

Using GCC for coverage analysis

These are the basic steps to perform coverage analysis:

  • For each module in your program, add the -fprofile-arcs and -ftest-coverage compilation switches. Compilation will then produce a .gcno file for each module in addition to the usual output.
  • Link the program against libgcov
  • Run the program. This will produce a .gcda data file for each module when the program exits.

You can now run gcov for each module of interest, e.g.:

gcov *.cpp

This will process all the .gcda files, give some summary information, and generate a .gcov text file for each module.  Read the .gcov file to find the information that we want.  For example, for source module test.cpp you will now have a file test.cpp.gcov which shows, for each line of code in the source file, how many times each line was executed.

For more information, refer to the official gcov documentation.

Using GCC for profiling

To obtain profile information:

  • Add the -pg switch to the compilation of each module.
  • Add the -pg switch to the linker command line.
  • Run the program.

When the program finishes, this will produce a gmon.out data file which can then be analysed by e.g.:

gprof test gmon.out

This will output the analysis to the console, so it’s best if you redirect the output to a file which you can analyse as required.

More information on profiling using GCC can be found in the gprof manual or the gprof man pages on your system.

Hints

  • Make sure you use the same version of gcov/gprof as for gcc. If you mismatch versions you can get strange results.
  • Disable optimisation to make sure you don’t get strange results.
  • The man pages of each tool are quite good, refer to them for more information.
  • If your program crashes, you won’t get any profiling or coverage information, it is generated only if the program exits cleanly.
  • Don’t spend too much time worrying about profiling until your program is largely working.

Sample Makefile

The following Makefile is an example of how you can enable/disable profiling/coverage easily for a given build.  In this example, just comment or uncomment the appropriate fragment.  This particular example is for a simple C++ program with two source modules.

# Comment-out or uncomment these definitions as required
#COVERAGE = 1
#PROFILE = 1

GCCDIR = /usr

CC = ${GCCDIR}/bin/gcc
CPP = ${GCCDIR}/bin/g++

CPPFLAGS += -Wall -pedantic -O0 -g
LINKFLAGS += -g

DETRITUS = *.o *~

ifeq (${COVERAGE},1)
CPPFLAGS += -fprofile-arcs -ftest-coverage
LINKFLAGS +=
LIBS += -lgcov
DETRITUS += *.gcno *.gcda *.gcov
endif

ifeq (${PROFILE},1)
CPPFLAGS += -pg
LINKFLAGS += -pg
DETRITUS += gmon.out
endif

all:	test

test:	test.o foo.o
  ${CPP} ${LINKFLAGS} -o $@ $^ ${LIBS}

test.o:	test.cpp foo.hpp
  ${CPP} ${CPPFLAGS} -o $@ -c
 
lt; foo.o: foo.cpp foo.hpp ${CPP} ${CPPFLAGS} -o $@ -c
 
lt; clean: rm -f test ${DETRITUS}