MLJTestInterface.jl

Package for testing an implementation of the MLJ model interface.

Installation

using Pkg
Pkg.add("MLJTestInterface")

Usage

To test that a collection of model types, models, satisfy the MLJ model interface requirements, use the MLJTestInterface.test function:

MLJTestInterface.test(models, data...; mod=Main, level=2, throw=false, verbosity=1) 
    -> failures, summary

Here data is training data acceptable to all the specified models, as would appear in a call MLJModelInterface.fit(model_instance, verbosity, data...).

For detailed documentation, run using MLJTestInterface; @doc MLJTestInterface.test.

Example

The following tests the model interface implemented by the DecisionTreeClassifier model implemented in the package MLJDecisionTreeInterface.jl.

import MLJDecisionTreeInterface
import MLJTestInterface
using Test
X, y = MLJTestInterface.make_multiclass()
failures, summary = MLJTestInterface.test(
    [MLJDecisionTreeInterface.DecisionTreeClassifier, ],
    X, y,
    verbosity=0, # set to 2 when debugging
    throw=false, # set to `true` when debugging
    mod=@__MODULE__,
)
@test isempty(failures)

Datasets

The following commands generate small datasets of the form (X, y) suitable for interface tests:

• MLJTestInterface.make_binary

• MLJTestInterface.make_multiclass

• MLJTestInterface.make_regression

• MLJTestInterface.make_count