Complex number support

[MikeInnes]

This gets the absolute basics going, and hopefully can serve as a platform for anyone who wants to add more complex derivatives to our library. (You can make a PR against this one if it's not merged in yet.)

julia> Tracker.gradient(x -> abs2(log(x)), 1+2im)
(1.2076065567220924 - 0.20091567785600395im (tracked),)

Adding this was a bit less simple than it should have been due to the lack of an AbstractComplex equivalent, which we have to hack around (and that's quite tied to the AD internals).

I'd probably like to get complex broadcast working before merging this, but that's a bit fiddly, so it'll depend on whether I get round to it.

[simonbyrne]

Adding this was a bit less simple than it should have been due to the lack of an AbstractComplex equivalent, which we have to hack around (and that's quite tied to the AD internals).

JuliaLang/julia#26666

[ssfrr]

I think another necessary feature would be the ability to do arithmetic on the tracked complex numbers. With this PR I tried a scalar loss function:

target = rand() + im*rand()
loss(x) = abs2(target-x)
dloss(x) = Tracker.gradient(loss, x)[1]

# initial value
x = rand() + im*rand()
@show loss(x)

for _ in 1:10
  global x -= 0.1*dloss(x)
  @show loss(x)
end

This gives an MethodError trying to apply loss. For a more basic example you get errors with these:

x = 1.0+1.0im

param(x) + x
x + param(x)
param(x)+param(x)

[chriscoey]

any updates on this? I have the need for taking gradients and Hessians of logdet of a Hermitian PSD matrix

[MikeInnes]

Closing in favour of FluxML/Tracker.jl#16 since Tracker has moved out of Flux now.

It's not that likely that this PR will develop any further, though; if you want complex support you should check out Zygote.