Debugging, profiling and accelerating Julia code

1.1 Stacktraces

🏗

1.2 Don’t be uninterruptible!

If you try to make a script robust against miscellaneous errors by catching and ignoring generic exceptions (e.g. because it has to work with IO or some badly-behaved algorithm that can fail in various ways), it is easy to cause it to be uninterruptible because interrupting is also an exception. This seems to be the idiom to cause it to try to grind on with boring errors but exit on user request:

try
    dosomethingtricky()
catch e
    if !isa(e, InterruptException)
        @warn e
    else
        throw(e)
    end
end

However, this will also continue through other errors that I would probably like to stop on (e.g. TypeError, MethodError, ArgumentError) because they imply I did something foolish. Hmm. Possibly there is no one-size-fits-all solution here.

3.1 Analysing

If you want to know how you made some weird mistake in type design, try:

• Traceur.jl - runs your code and tells you about any obvious performance traps
• Cthulhu.jl - descend into Julia type dispatching.

3.2 Profiling

Here is a convenient overview of timer functions that help work out what to optimise. The built-in timing tool is called time. It can even time arbitrary begin/end blocks.

Going deeper, try Profile the statistical profiler:

julia> using Profile
julia> @profile myfunc()

ProfileView is advisable for making sense of the output. It doesn’t work in jupyterlab, but from jupyterlab (or anywhere) you can invoke the Gtk interface by setting PROFILEVIEW_USEGTK = true in the Main module before using ProfileView.

It’s kind of the opposite of profiling, but you can give use feedback about what’s happening using a ProgressMeter.

TimerOutputs is a macro system that can accumulate various bits of timing information under tags filed by category.

3.3 Near-magical performance annotations

@simd

equivocally endorsed in the manual, as it may or may not be needed to squeeze best possible performance out of the inner loop. Caution says try it out and see.

@inbounds

disables bounds checking, but be careful with your indexing. Bounds checking is expensive.

@fastmath

violates strict IEEE math setup for performance, in a variety of ways some of which sound benign and some not so much.

3.4 Caching

Doing heavy calculations? Want to save time? Caching notionally helps, but of course is a quagmire of bikeshedding.

The Anamemsis creator(s) have opinion about this:

Julia is a scientific programming language which was designed from the ground up with computational efficiency as one of its goals. As such, even ad hoc and “naive” functions written in Julia tend to be extremely efficient. Furthermore, any memoization implementation is liable to have some performance pain points: in general they contain type unstable code (even if they have type stable output) and they must include a value cache which is accessible from the same scope as the function itself so that global objects are potentially involved. Because of this, memoization comes with a significant overhead, even with Julia’s highly efficient AbstractDict implementations.

My own opinion of caching is that I’m for it. Despite the popularity of the automatic memoization pattern I’m against that as a rule. I usually end up having to think carefully about caching, and magically making functions cache never seems to help with the hard part of that. Caching logic should, for projects I’ve worked on at least, be explicit, opt-in, and that should be an acceptable design overhead because I should only be doing it for a few expensive functions, rather than creating magic cache functions everywhere.

LRUCache is the most commonly recommended cache. It provides a simple, explicit, opt-in, in-memory, typed cache structure, so it should be performant. It promises sensible, if not granular, locking for multi-threaded access: one thread at a time can access any key of the cache. AFAICT this meets 100% of my needs.

Invocations look like

function foo(a::Float64, b::Float64)
    sleep(100)
    result = a * b
end

const lru = LRU{Tuple{Float64, Float64}, Float64}()
function cached_foo(a::Float64, b::Float64)
    get!(lru, (a, b)) do
      foo(a,b)
    end
end

Memoize provides an automatic caching macro but primitive in-memory caching using plain old dictionaries. AFAICT it is possibly compatible with LRUCache, but weirdly I could not find discussion of that. However, I don’t like memoisation, so that’s the end of that story.

Anamemsis has advanced cache decorators and lots of levers to pull to alter caching behaviour.

Caching is especially full-featured with macros, disk persistence and configurable evictions etc. It wants type annotations to be used in the cached function definitions.

Cleverman Simon Byrne notes that you can roll your own fancy caching using wild Julia introspection stunts via Cassette.jl, but I don’t know how battle-tested this is. I am not smart enough for that much magic.