# LocalScopeBenchmarks.jl

**tl;dr:** Tired of adding `$`

everywhere when you `@benchmark`

or `@btime`

? Try `@localbtime f(x)`

instead of `@btime f($x)`

.

## Introduction

BenchmarkTools.jl is an amazingly useful package for benchmarking Julia code. But it does have one notable gotcha: the expression being benchmarked is run at *global* scope, so any variables it references are by necessity global variables. Julia enthusiasts may recall that the very first Julia performance tip is to avoid non-`const`

global variables because their type can change at any time and therefore can't be relied upon by the compiler. The result is that naively using `BenchmarkTools.jl`

often results in benchmarks that appear to run much slower than they should due to the performance cost of accessing globals.

For example, a common benchmarking mistake looks like:

```
julia> using BenchmarkTools
julia> x = 1.0
1.0
julia> @btime sin(x)
18.011 ns (1 allocation: 16 bytes)
```

18 nanoseconds is a bit slow, and we have an unexpected 16 bytes of heap memory allocation.

The solution is easy, and is explained in the BenchmarkTools.jl manual: you just have to interpolate the *value* of `x`

into the benchmark expression rather than force it to be looked up as a global variable:

```
julia> @btime sin($x)
7.059 ns (0 allocations: 0 bytes)
0.8414709848078965
```

much faster, and no unexpected memory allocation. By interpolating `$x`

, we get a result which is representative of how fast `sin`

would behave for an input of this type inside a function, which is where it's almost always going to be found.

This interpolation trick is easy to do, but I've found it consistently annoying, and it's caused an impressive amount of confusion (kudos to `@NiclasMattsson`

on Discourse for finding most of these):

- https://discourse.julialang.org/t/trig-functions-very-slow/15335/54
- https://discourse.julialang.org/t/another-blas-and-julia-comparison/15411/2
- https://discourse.julialang.org/t/allocation-by-staticarrays-in-anonymous-function-macro/14774/2
- https://discourse.julialang.org/t/how-to-optimise-and-be-faster-than-java/14457/22
- https://discourse.julialang.org/t/interval-arithmetic-computation-time/14633/14
- https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/9
- https://discourse.julialang.org/t/vector-of-matrices-vs-multidimensional-arrays/9602/3
- https://discourse.julialang.org/t/spurious-allocation/3751/7
- https://discourse.julialang.org/t/improve-the-performance-of-multiplication-of-an-arbitrary-number-of-matrices/10835/19
- https://discourse.julialang.org/t/a-generator-with-two-for-keywords-is-slow/7407/5

## There Has To Be a Better Way!

Enter `LocalScopeBenchmarks.jl`

. This package tries to do exactly one thing: save you from having to remember to add `$`

all over the place when benchmarking. Observe:

```
julia> using LocalScopeBenchmarks
julia> x = 1.0
1.0
julia> @localbtime sin(x)
6.556 ns (0 allocations: 0 bytes)
0.8414709848078965
```

We got the same measurement as `@btime sin($x)`

without having to add a `$`

. Hooray!

### Installation

```
using Pkg
Pkg.add("https://github.com/rdeits/LocalScopeBenchmarks.jl")
```

or just press `]`

at the Julia REPL and then enter `add https://github.com/rdeits/LocalScopeBenchmarks.jl`

### Usage

`julia> using LocalScopeBenchmarks`

This package provides `@localbtime`

, `@localbenchmark`

, and `@localbelapsed`

, analogous to `@btime`

, `@benchmark`

, and `@belapsed`

from BenchmarkTools.jl. Each should support the same inputs and return the same types as their BenchmarkTools.jl equivalents (in fact, each of them is just a thin wrapper around the existing BenchmarkTools macros). The only change is that the `@local*`

versions try to interpolate local variables into the benchmarked expression rather than treating those variables as global.

Since we're just using BenchmarkTools under the hood, the `setup`

and `evals`

keyword arguments work as normal:

```
julia> x = 1.0
1.0
julia> @localbtime f(x) setup=(f = sin)
6.791 ns (0 allocations: 0 bytes)
```

```
julia> @localbtime x^2 evals=100
0.290 ns (0 allocations: 0 bytes)
```

You can also still interpolate values into the expression with `$`

if you really want:

```
# Includes the time spent calling `rand(1000)`
julia> @localbtime sum(rand(1000))
1.084 μs (1 allocation: 7.94 KiB)
```

```
# Interpolates the *value* of `rand(1000)` so that it's not
# computed inside the benchmark:
julia> @localbtime sum($(rand(1000)))
69.160 ns (0 allocations: 0 bytes)
```