Julia In SWAN

1

Dear Julia enthusiasts,

With the introduction of the LCG_100 stack, Julia version 1.6.0 is now available in SWAN. However, the Jupyter kernel that allows you to create Julia notebooks is not directly available. The user needs to install the Julia kernel and required packages on his/her EOS home directory. This is done by executing the following commands in a SWAN terminal:

julia -e 'import Pkg; Pkg.add("IJulia")'
julia -e 'using IJulia; installkernel("julia", env=merge(Dict("JULIA_NUM_THREADS"=>"4"), Dict(ENV)))'

Files will be placed in $SWAN_HOME/.local/share/jupyter and $SWAN_HOME/.julia. The execution of this commands is needed to be done only once (as long as the version of Julia is non changing). Additional ‘imported’ packages by the user will also be placed in $SWAN_HOME/.julia and survive across sessions.

2 Likes
2

need

export JUPYTER_DATA_DIR=$HOME/.local/share/jupyter

btw, or IJulia won’t pass Pkg.build() step

3

Thanks! The correct full instructions are:

export JUPYTER_DATA_DIR=$HOME/.local/share/jupyter
julia -e 'import Pkg; Pkg.add("IJulia")'
julia -e 'using IJulia; installkernel("julia", env=merge(Dict("JULIA_NUM_THREADS"=>"4"), Dict(ENV)))'
4

the most relevant part is:

Improved performance for handling registries on Windows and distributed file systems

Instead of using a thread pool to speed up closing files, we decided to take a different route. Julia comes bundled with p7zip and together with the standard library Tar.jl it is possible to directly read the compressed tarball into memory without materializing any files at all. By doing so the problem of materializing files is obliviated which significantly improves the performance of the registry on Windows, NFS and other distributed file systems like those typically used in HPC systems.

this should significantly improve performance on SWAN, which previously suffers from dealing with many-large-small files for the Julia registry.

1 Like