ArrayAllez.jl
] add ArrayAllez
⊙ = \odot
Matrix multiplication, on the last index of one tensor & the first index of the next:
three = rand(2,2,5);
mat = rand(5,2);
p1 = three ⊙ mat
p2 = reshape(reshape(three,:,5) * mat ,2,2,2) # same
using Einsum
@einsum p3[i,j,k] := three[i,j,s] * mat[s,k] # same
There are also variants ⊙ˡ, ⊙ʳ
with different gradient definitions,
specifying that only what's on the left (or right) needs to be tracked.
(Likewise *ˡ, *ʳ
for ordinary *
.)
bmm == ⨱ (\timesbar)
Batched matrix multiplication, which understands all trailing dimensions:
four = rand(2,3,8,9);
three = rand(3,8,9);
size(four ⨱ three) == (2, 8, 9)
(four ⨱ three)[:,1,1] ≈ four[:,:,1,1] * three[:,1,1]
using Einsum
@einsum out[i,x,y] := four[i,j,x,y] * three[j,x,y];
out ≈ four ⨱ three
Corresponding ⨱ˡ, ⨱ʳ
are not yet defined.
dimnames
Both ⊙
and ⨱
will propagate names from NamedDims.jl.
log! ∘ exp!
This began as a way to more conveniently choose between Yeppp!
and AppleAccelerate
and IntelVectorMath,
without requiring that any by installed.
The fallback version is just a loop, with @threads
for large enough arrays.
x = rand(1,100);
y = exp0(x) # precisely = exp.(x)
x ≈ log!(y) # inplace, just a loop
using AppleAccelerate # or using IntelVectorMath, or using Yeppp
y = exp!(x) # with ! mutates
x = log_(y) # with _ copies
Besides log!
and exp!
, there is also scale!
which understands rows/columns.
And iscale!
which divides, and inv!
which is an elementwise inverse.
All have nonmutating versions ending _
instead of !
, and simple broadcasted versions with 0
.
m = ones(3,7)
v = rand(3)
r = rand(7)'
scale0(m, 99) # simply m .* 99
scale_(m, v) # like m .* v but using rmul!
iscale!(m, r) # like m ./ r but mutating.
m
∇
These commands all make some attempt to define gradients for use with
Tracker ans
Zygote, but caveat emptor.
There is also an exp!!
which mutates both its forward input and its backward gradient,
which may be a terrible idea.
using Tracker
x = param(randn(5));
y = exp_(x)
Tracker.back!(sum_(exp!(x)))
x.data == y # true
x.grad
This package also defines gradients for prod
(overwriting an incorrect one) and cumprod
,
as in this PR.
Array_
An experiment with LRUCache for working space:
x = rand(2000)' # turns off below this size
copy_(:copy, x)
similar_(:sim, x)
Array_{Float64}(:new, 5,1000) # @btime 200 ns, 32 bytes
inv_(:inv, x) # most of the _ functions can optin
@dropdims
This macro wraps reductions like sum(A; dims=...)
in dropdims()
.
It understands things like this:
@dropdims sum(10 .* randn(2,10); dims=2) do x
trunc(Int, x)
end
See Also

Vectorize.jl is a more comprehensive wrapper.

Strided.jl adds
@threads
to broadcasting. 
LoopVectorization.jl adds AVX black magic.