ReferenceTests.jl is a Julia package that adds a couple of
additional macros to your testing toolbox. In particular, it
focuses on functionality for testing values against reference
files, which in turn the package can help create and update if
need be. ReferenceTests.jl is build on top of
FileIO.jl and designed
to be used alongside
|Package Status||Package Evaluator||Build Status|
It is very common for Julia packages to test the functionality of
their exported functions against known input-to-output
combinations. We will refer to such kind of tests as reference
tests. In most cases these will be quite simple; something along
the line of
@test f(x) == y, where
f is a function of the
user package and
x is some interesting input value for which
the desired output
y is known.
For testing the output of more complex functions, for which the
expected output is more complicated (e.g. anything image
processing related), using
@test can be a little cumbersome to
work with. To that end this package provides the
@test_reference macro, which expects a filename (relative to
the file that invokes the macro) and an expression that evalutes
to the value of interest.
using ReferenceTests @test_reference "stringtest1.txt" string(collect(1:20))
If you put the above code into your
execute the file in an interactive julia session (i.e. with
include), then it will trigger an interactive dialog if the
results don't match or the file does not yet exist. This dialog
allows the user to create and/or update the reference files.
The given file
stringtest1.txt is assumed to be the relative
path to the file that contains the macro invocation. This likely
means that the path is relative to the
test/ folder of your
The file-extension of the filename (here
txt), as well as the
type of the result of evaluating the expression (here
determine how the actual value is compared to the reference
value. The default implementation will do a simple equality check
with the result of
FileIO.load. This means that it is the
user's responsibility to have the required IO package installed.
Colorant arrays (i.e.) receive special treatment. If the
extension of the filename is
txt then the package
will be used to create a string-based crude approximation of the
image. This will have low storage requirements and also allows to
view the reference file in a simple terminal using
using ReferenceTests, TestImages @test_reference "imagetest1.txt" testimage("cameraman")
Note that while a text-based storage of reference images can be
convenient, proper image formats (e.g.
png) are also supported
by the package. Those, however, will require the proper
backends to be installed.
Another special file extension is
sha256 which will cause the
hash of the result of the given expression to be stored and
compared as plain text. This is useful for a convenient
low-storage way of making sure that the return value doesn't
change for selected test cases.
Check out the latest documentation
Additionally, you can make use of Julia's native docsystem.
The following example shows how to get additional information
@test_reference within Julia's REPL:
This code is free to use under the terms of the MIT license.