ReliabilityDiagrams.jl

Visualization of model calibration
Author devmotion
Popularity
5 Stars
Updated Last
6 Months Ago
Started In
April 2021

ReliabilityDiagrams

Visualization of model calibration

Stable Dev Build Status DOI Coverage Coverage Code Style: Blue Aqua QA

Example of a reliability diagram

Related packages

  • CalibrationErrors.jl: Estimation of calibration errors such as the expected calibration error (ECE), the squared kernel calibration error (SKCE), and the unnormalized calibration mean embedding (UCME).
  • CalibrationTests.jl: Statistical hypothesis tests of calibration.

References

Bröcker, J., & Smith, L. A. (2007). Increasing the reliability of reliability diagrams. Weather and forecasting, 22(3), 651-661.

Murphy, A., & Winkler, R. (1977). Reliability of Subjective Probability Forecasts of Precipitation and Temperature. Journal of the Royal Statistical Society. Series C (Applied Statistics), 26(1), 41-47.

Vaicenavicius, J., Widmann, D., Andersson, C., Lindsten, F., Roll, J. & Schön, T. B. (2019). Evaluating model calibration in classification. Proceedings of Machine Learning Research, in Proceedings of Machine Learning Research 89:3459-3467 (AISTATS 2019).

Used By Packages