This is a Julia implementation of the Non-parametric Hierarchial Performance Testing statistical technique described in Statistical Performance Comparisons of Computers. It is meant to compare the performance of different computers on a common set of benchmarks with high statistical confidence, rather than .
For more information, check the official website maintained by one of the technique's creators.
Each set of benchmark results should be provided as a 2D matrix where each row is comprised of the same benchmark's results.
using HierarchialPerformanceTest
# Computer A's benchmark matrix
a = [86 86 86; 46 46 46; 2.491 2.3 2.314; 5.629 5.31 5.91; 262.69 262.39 262.632; 17.761 16.882 15.541; 3.264 3.205 3.256; 3678 3612 3642; 4251 4220 4170; 58176 56384 56512]
# Computer B's benchmark matrix
b = [83 83 83; 46 46 46; 2.263 2.239 2.228; 5.663 6.488 5.383; 260.977 261.075 260.757; 14.816 11.811 15.633; 2.323 2.153 2.315; 3456 3540 3442; 4009 4118 4090; 57664 58432 54848]
hptSpeedup(a, b) # => 1.013This shows that computer B is 1.013x faster than computer A.