LuaCov
Coverage analysis for Lua scripts

Overview

LuaCov is a simple coverage analyzer for Lua scripts. When a Lua script is run with the luacov module loaded, it generates a stats file with the number of executions of each line of the script and its loaded modules. The luacov command-line script then processes this file generating a report file which allows one to visualize which code paths were not traversed, which is useful for verifying the effectiveness of a test suite.

LuaCov is free software and uses the same license as Lua (MIT).

Download

LuaCov can be downloaded via LuaRocks: 

luarocks install luacov

There are some C extensions LuaCov can use (if they are available) to improve performance and analysis accuracy. To install LuaCov with these extensions install CLuaCov package instead: 

luarocks install cluacov

LuaCov itself is written in pure Lua and has no external dependencies.

You can also get the code directly from the git repo.

Instructions

Using LuaCov consists of two steps: running your script to collect coverage data, and then running luacov on the collected data to generate a report.

To collect coverage data, your script needs to load the luacov Lua module. This can be done from the command-line, without modifying your script, like this: 

lua -lluacov test.lua

     or 

lua -erequire('luacov.runner')('myconfigfilename') test.lua

(Alternatively, you can add require("luacov") to the first line of your script.)

Once the script is run, a file called luacov.stats.out is generated. If the file already exists, statistics are added to it. This is useful, for example, for making a series of runs with different input parameters in a test suite. To start the accounting from scratch, just delete the stats file.

To generate a report, just run the luacov command-line script. It expects to find a file named luacov.stats.out in the current directory, and outputs a file named luacov.report.out.

This is an example output of the report file:

==============================================================================
test.lua
==============================================================================
 1 if 10 > 100 then
*0    print("I don't think this line will execute.")
   else
 1    print("Hello, LuaCov!")
   end

Note that to generate this report, luacov reads the source files. Therefore, it expects to find them in the same location they were when the luacov module ran (the stats file stores the filenames, but not the sources themselves).

To silence missed line reporting for a group of lines, place inline options luacov: disable and luacov: enable in short comments around them: 

if SOME_DEBUG_CONDITION_THAT_IS_ALWAYS_FALSE_IN_TESTS then
   -- luacov: disable

   -- Lines here are not marked as missed even though they are not covered.

   -- luacov: enable
end

LuaCov saves its stats upon normal program termination. If your program is a daemon -- in other words, if it does not terminate normally -- you can use the luacov.tick module or the tick configuration option, which periodically saves the stats file. For example, to run (on Unix systems) LuaCov on Xavante, just modify the first line of xavante_start.lua so it reads: 

#!/usr/bin/env lua -lluacov.tick
or add this to .luacov config file: 
tick = true
LuaCov includes several configuration options, which have their defaults stored in luacov.defaults module. These are the global defaults. To use project specific configuration, create a Lua script setting options as globals or returning a table with some options and store it as .luacov in the project directory from where luacov is being run. For example, this config informs LuaCov that only foo module and its submodules should be covered and that they are located inside src directory: 
modules = {
   ["foo"] = "src/foo/init.lua",
   ["foo.*"] = "src"
}

History

0.14.0 [Jan 28, 2020]
  • Added option to include untested files in the report
  • Reduce probability of interrupt errors when running LuaCov in a subprocess
0.13.0 [May 5, 2018]
  • Added luacov: disable and luacov: enable inline options that mark source lines between them as impossible to hit.
  • Fixed error when reporing coverage for files with a shebang lines using CLuaCov.
0.12.0 [June 29, 2016]
  • Added support for experimental C extensions (CLuaCov).
  • Changed config format: options are now set by assigning to globals, old format (returning a table) is still supported.
  • Added tickconfig option, equivalent to using luacov.tick module.
  • Fixed coverage data being saved to wrong file when using relative statsfile path and the program running LuaCov changes directories.
  • Improved config loading error handling.
  • Added :on_file_error() stub method to base reporter class, used for reporting problems when analyzing coverage data related to a file.
0.11.0 [April 18, 2016]
  • Fixed several cases of lines falsely reported as missed.
  • Fixed luacov.tick module not working.
  • Improved default reporter output format.
  • Reduced coverage collection overhead.
  • Changed how coverage is saved, it's now possible to start a child Lua process with LuaCov enabled without wrapping the launch in luacov.pause and luacov.resume in the parent.
  • Several minor fixes and improvements.
0.10.0 [February 9, 2016]
  • Added debug_hook() function for use in custom debug hooks.
  • Fixed patterns passed as command-line arguments matching too much.
  • Fixed order in which module name translations are applied
0.9.1 [December 7, 2015]
  • Fixed error when running LuaCov using Lua 5.1.
0.9 [December 6, 2015]
  • with_luacov() function for covering coroutines created via the C API.
  • fix priorities in modules list
  • improve coverage analysis/exclusions list
  • improve handling of multiline function declarations
  • LDoc documentation
0.8 [September 30, 2015]
  • Improved lexer which reduces false positives
  • luacov.pause() and luacov.resume() functions
  • "modules" option for configuration
  • Plus several fixes and code cleanups.
0.7 [January 12, 2015]
  • Improvement in detection of long strings.
  • Added "savestepsize" option.
  • Fix handling "codefromstring" option.
0.6 [September 10, 2014]
  • Support for custom reporter objects
  • Configuration option for processing/skipping strings
  • Several fixes: behavior of on_exit, inclusion/exclusions lists, etc.
0.5 [February 8, 2014]
  • Improved performance in reporter module
  • More improvements in exclusions list
0.4 [December 3, 2013]
  • Lua 5.2 compatibility fixes
  • Several improvements in exclusions list
0.3 [October 10, 2012]
  • Added configuration options and files
  • Summary in report
  • Improved handling of long strings and comments
  • Support for coroutines and os.exit()
0.2 [April 30, 2009]
  • Ignore code loaded from strings.
0.1 [July 16, 2007]
  • Initial release.

Credits

LuaCov was originally designed and implemented by Hisham Muhammad as a tool for testing LuaRocks. A number of people have improved it since: see the Git logs for the full list of contributors!