A Julia implementation of the Meta-ICVI method as a separate package.
Author AP6YC
2 Stars
Updated Last
7 Months Ago
Started In
August 2021


A Julia implementation of the Meta-ICVI method as a separate package.

Documentation Build Status Coverage
Stable Dev Build Status Codecov

Table of Contents



You must install PyCallJLD.jl alongside MetaICVI.jl for correct classifier module loading and saving. This is because the ScikitLearn.jl dependency requires saving/loading with the JLD.jl package on PyCall.jl objects, and PyCallJLD correctly loads the serialized object definitions into the current workspace. Otherwise, the classifier is loaded a memory block wrapped in a PyObject type, breaking inference and other operations.

Both PyCallJLD.jl and MetaICVI.jl are distributed as Julia packages, available on JuliaHub. Their installation followa the usual Julia package installation procedure, and they can both be installed simultaneously interactively:

julia> ]
(@v1.8) pkg> add PyCallJLD MetaICVI

or programmatically:

using Pkg

You may also get the most recent changes directly from the GitHub repository with:

julia> ]
(@v1.8) pkg> add

or programmatically, also with the GitHub link:

using Pkg

Basic Usage

First, load both PyCallJLD and MetaICVI with

using PyCallJLD, MetaICVI

Then, create a MetaICVI module with the default constructor

metaicvi = MetaICVIModule()

and retrieve the MetaICVI value iteratively with

get_metaicvi(metaicvi, sample, label)

where sample is a real-valued vector and label is an integer.

Advanced Usage

After loading both PyCallJLD and MetaICVI

using PyCallJLD, MetaICVI

you can specify the MetaICVI options with

opts = MetaICVIOpts(
    classifier_selection = :SGDClassifier,
    classifier_opts = (loss="log", max_iter=30),
    icvi_window = 5,
    correlation_window = 5,
    n_rocket = 5,
    rocket_file = "data/models/rocket.jld2",
    classifier_file = "data/models/classifier.jld",
    display = true,
    fail_on_missing = false
metaicvi = MetaICVIModule(opts)

The options are

  • classifier_selection: a symbol for a linear classifier from ScikitLearn.jl (only used if you are creating and training a new classifier).
  • classifier_opts: the options passed to the classifier during instantiation (also only used if creating and training a new classifier).
  • icvi_window: the number of ICVI criterion values to compute rank correlation across.
  • correlation_window: the number of correlations to compute rocket features across.
  • rocket_file: filename of a saved RocketModule.
  • classifier_file: filename of a saved linear classifier.
  • display: boolean flag for logging info.
  • fail_on_missing: boolean flag for crashing if missing rocket and/or classifier files.


Please raise an issue.




This software is developed by the Applied Computational Intelligence Laboratory (ACIL) of the Missouri University of Science and Technology (S&T) under the supervision of Teledyne Technologies for the DARPA L2M program. Read the License.


This project has a citation file file that generates citation information for the package and corresponding JOSS paper, which can be accessed at the "Cite this repository button" under the "About" section of the GitHub page.

You may also cite this repository with the following BibTeX entry:

  author = "Niklas Melton and Sasha Petrenko and Donald Wunsch",
  title = "{Meta-iCVIs: Ensemble Validity Metrics for Concise Labeling of Correct, Under- or Over-Partitioning in Streaming Clustering}",
  year = "2022",
  month = "12",
  url = "",
  doi = "10.36227/techrxiv.21685214"