Julia package for benchmarking a system. Contributions very welcome!
Submit your benchmarks to SystemBenchmark.jl/issues/8
GitHub Actions - Linux, MacOS, Windows - x86_64
Run benchmark
julia> using SystemBenchmark
julia> res = runbenchmark();
julia> show(res, allrows=true, allcols=true)
32×4 DataFrame
│ Row │ cat │ testname │ units │ res │
│ │ String │ String │ String │ Any │
├─────┼─────────────┼───────────────────────────┼────────┼──────────────────────────────────────────┤
│ 1 │ info │ SysBenchVer │ │ 0.3.0 │
│ 2 │ info │ JuliaVer │ │ 1.4.1 │
│ 3 │ info │ OS │ │ macOS (x86_64-apple-darwin18.7.0) │
│ 4 │ info │ CPU │ │ Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz │
│ 5 │ info │ WORD_SIZE │ │ 64 │
│ 6 │ info │ LIBM │ │ libopenlibm │
│ 7 │ info │ LLVM │ │ libLLVM-8.0.1 (ORCJIT, skylake) │
│ 8 │ info │ GPU │ │ missing │
│ 9 │ cpu │ FloatMul │ ms │ 1.708e-6 │
│ 10 │ cpu │ FusedMulAdd │ ms │ 1.711e-6 │
│ 11 │ cpu │ FloatSin │ ms │ 4.773e-6 │
│ 12 │ cpu │ VecMulBroad │ ms │ 4.55384e-5 │
│ 13 │ cpu │ CPUMatMul │ ms │ 0.041647 │
│ 14 │ cpu │ MatMulBroad │ ms │ 0.0194466 │
│ 15 │ cpu │ 3DMulBroad │ ms │ 0.0016652 │
│ 16 │ cpu │ peakflops │ flops │ 1.96696e11 │
│ 17 │ cpu │ FFMPEGH264Write │ ms │ 154.421 │
│ 18 │ mem │ DeepCopy │ ms │ 0.000198299 │
│ 19 │ mem │ Bandwidth10kB │ MiB/s │ 96265.8 │
│ 20 │ mem │ Bandwidth100kB │ MiB/s │ 50003.9 │
│ 21 │ mem │ Bandwidth1MB │ MiB/s │ 25855.3 │
│ 22 │ mem │ Bandwidth10MB │ MiB/s │ 11320.0 │
│ 23 │ mem │ Bandwidth100MB │ MiB/s │ 9684.15 │
│ 24 │ diskio │ DiskWrite1KB │ ms │ 0.145131 │
│ 25 │ diskio │ DiskWrite1MB │ ms │ 10.4186 │
│ 26 │ diskio │ DiskRead1KB │ ms │ 0.0969565 │
│ 27 │ diskio │ DiskRead1MB │ ms │ 0.560448 │
│ 28 │ loading │ JuliaLoad │ ms │ 209.872 │
│ 29 │ compilation │ compilecache │ ms │ 338.605 │
│ 30 │ compilation │ success_create_expr_cache │ ms │ 349.521 │
│ 31 │ compilation │ create_expr_cache │ ms │ 13.648 │
│ 32 │ compilation │ output-ji-substart │ ms │ 7.2555 │
Compare benchmark to the default reference (a 2019 MSI Linux i7 Laptop) with:
julia> comp = comparetoref(res)
or to run the benchmark and do the comparison in one move:
julia> comp = comparetoref()
[ Info: CuArrays.functional() == false. No usable GPU detected
Compilation tests100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| Time: 0:01:58
[ Info: Printing of results may be truncated. To view the full results use `show(res, allrows=true)`
32×6 DataFrames.DataFrame
│ Row │ cat │ testname │ units │ ref_res │ test_res │ factor │
│ │ String │ String │ String │ Any │ Any │ Any │
├─────┼─────────────┼───────────────────────────┼────────┼──────────────────────────────────────────┼──────────────────────────────────────────┼───────────┤
│ 1 │ info │ SysBenchVer │ │ 0.3.0 │ 0.3.0 │ Equal │
│ 2 │ info │ JuliaVer │ │ 1.4.1 │ 1.4.1 │ Equal │
│ 3 │ info │ OS │ │ Linux (x86_64-pc-linux-gnu) │ macOS (x86_64-apple-darwin18.7.0) │ Not Equal │
│ 4 │ info │ CPU │ │ Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz │ Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz │ Not Equal │
│ 5 │ info │ WORD_SIZE │ │ 64 │ 64 │ Equal │
│ 6 │ info │ LIBM │ │ libopenlibm │ libopenlibm │ Equal │
│ 7 │ info │ LLVM │ │ libLLVM-8.0.1 (ORCJIT, skylake) │ libLLVM-8.0.1 (ORCJIT, skylake) │ Equal │
│ 8 │ info │ GPU │ │ GeForce GTX 1650 with Max-Q Design │ missing │ Not Equal │
│ 9 │ cpu │ FloatMul │ ms │ 1.1339999999999999e-6 │ 1.715e-6 │ 1.51235 │
│ 10 │ cpu │ FusedMulAdd │ ms │ 1.1339999999999999e-6 │ 1.71e-6 │ 1.50794 │
│ 11 │ cpu │ FloatSin │ ms │ 4.051e-6 │ 5.736e-6 │ 1.41595 │
│ 12 │ cpu │ VecMulBroad │ ms │ 2.9823115577889445e-5 │ 5.12837e-5 │ 1.7196 │
│ 13 │ cpu │ CPUMatMul │ ms │ 0.018683 │ 0.0401375 │ 2.14834 │
│ 14 │ cpu │ MatMulBroad │ ms │ 0.0042223 │ 0.0205616 │ 4.86976 │
│ 15 │ cpu │ 3DMulBroad │ ms │ 0.0010529 │ 0.0017379 │ 1.65058 │
│ 16 │ cpu │ peakflops │ flops │ 1.8061545733047305e11 │ 2.46247e11 │ 1.36338 │
│ 17 │ cpu │ FFMPEGH264Write │ ms │ 107.751337 │ 150.311 │ 1.39498 │
│ 18 │ mem │ DeepCopy │ ms │ 0.00018790257558790594 │ 0.000237082 │ 1.26173 │
│ 19 │ mem │ Bandwidth10kB │ MiB/s │ 104871.77985698299 │ 96519.5 │ 0.920357 │
│ 20 │ mem │ Bandwidth100kB │ MiB/s │ 58579.50346475738 │ 42754.2 │ 0.729848 │
│ 21 │ mem │ Bandwidth1MB │ MiB/s │ 33439.963407070725 │ 31394.6 │ 0.938835 │
│ 22 │ mem │ Bandwidth10MB │ MiB/s │ 5551.476774539766 │ 10848.2 │ 1.95411 │
│ 23 │ mem │ Bandwidth100MB │ MiB/s │ 6910.9558841033295 │ 8858.09 │ 1.28175 │
│ 24 │ diskio │ DiskWrite1KB │ ms │ 0.031895 │ 0.14471 │ 4.53707 │
│ 25 │ diskio │ DiskWrite1MB │ ms │ 0.920052 │ 0.554107 │ 0.602257 │
│ 26 │ diskio │ DiskRead1KB │ ms │ 0.006683666666666667 │ 0.0730455 │ 10.929 │
│ 27 │ diskio │ DiskRead1MB │ ms │ 0.143755 │ 0.592742 │ 4.12328 │
│ 28 │ loading │ JuliaLoad │ ms │ 91.08045 │ 213.571 │ 2.34486 │
│ 29 │ compilation │ compilecache │ ms │ 208.532099 │ 336.518 │ 1.61375 │
│ 30 │ compilation │ success_create_expr_cache │ ms │ 235.4855485 │ 334.724 │ 1.42142 │
│ 31 │ compilation │ create_expr_cache │ ms │ 0.894126 │ 12.2583 │ 13.7098 │
│ 32 │ compilation │ output-ji-substart │ ms │ 32.639706000000004 │ 42.0636 │ 1.28873 │
Save to disk (includes a system report)
savebenchmark(path::String, res::DataFrame)
Compare two benchmarks
compare(ref::DataFrame, res::DataFrame)
Please consider submitting results to SystemBenchmark.jl/issues/8.
julia> using SystemBenchmark
julia> res = runbenchmark();
julia> savebenchmark("results.txt", res)
julia> show(res, allrows=true, allcols=true)
Then append the txt file and copy the printed table into a new comment on SystemBenchmark.jl/issues/8.
Any .txt results files posted to that issue can easily be collated into a dataframe with:
julia> comparison = getsubmittedbenchmarks()
or to specify another repo/issue:
getsubmittedbenchmarks(;repo::String="IanButterworth/SystemBenchmark.jl", issue::Int=8, refname::String="ref.txt", transpose::Bool=true)
Some basic plotting exists at reporting/plotting.jl, which hasn't been included in the main package due to limitations in platform compatability of Plots.jl.
