Import software from CVMFS

Dear SWAN team,

I am probably missing something, but I didn’t find a way in order to use some software located under CVMFS. On lxplus I used to set un an environment, and work with that, but I am not able to do that on SWAN.
I also try to use the startup script to set up the environment but it seems to fails.

Thank you for your patience

Dear Piero,

Where is that software located on CVMFS and how do you usually set your environment on lxplus to use it?

Dear Enric,

Thank you for the response. I would like to use some packages located at:

/cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v32r2/Gaudi/python/Gaudi

Usually, after logging on lxplus, after the automatic group login, I set up the environment with

lb-run -c best LHCb/v50r6p1 bash --norc

Now I am able to perform the group login in the swan startup script, but not to set the environment.
However those command run without errors in the swan terminal.
Is there any way to use that modules in the notebooks?

Thank you again

Dear Piero,

The way the SWAN startup script works is by exporting new values for the variables you would like to set, for example:

export PYTHONPATH=/cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v32r2/Gaudi/python:$PYTHONPATH

Do the commands you run inside your SWAN startup script do such kind of export?

Dear Enric,

Yes I tried to export manually the variables in my script, but it seems that I am not able to achieve the same result as

lb-run -c best LHCb/v50r6p1 bash --norc

Actually running it in the SWAN terminal, will correctly load the environment. I would like to have the same environment in the notebooks. Also in a notebook:

%%bash
lb-run -c best LHCb/v50r6p1 bash --norc

works, but the environment is only available in the current cell.

Thank you again

Dear Piero,

My guess is that

lb-run -c best LHCb/v50r6p1 bash --norc

creates a new bash process that has the desired environment, so it works from the SWAN terminal. Running it from a bash cell of a notebook only affects that cell, because a new process is spawned to run it and dies right after.

In the SWAN env script things work differently, since what we do is spawn a new process that sources your script, gets all the environment variables that have been set by it and then records them to configure the environment of the notebook.

Can you share what lb-run actually does?

Dear Enric,
lb-run sets the environment (PATH, PYTHONPATH, LD_LIBRARYPATH etc) for a specific part of the LHCb software and then executes a program. That’s why it usually comes with many arguments. It is put in the path by the aforementioned login script.
source /cvmfs/lhcb.cern.ch/group_login.sh
which lb-run
/cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v9r2p6/InstallArea/scripts/lb-run
Hope this helps,
Albert

Thank you Albert

Anyway I managed to set up the environment by manually exporting the variables needed in the startup script, and for now it seems to work. I am just worried about the fact the I am probably missing some variable and it will not work anymore for other module related to LHCb software, but for now it is ok.

By the way thank you all for your support :slight_smile:
Piero

From what I see in:

/cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/LBSCRIPTS_v9r2p6/InstallArea/scripts/lb-run

lb-run will just spawn a new python process, set the environment there and execute. That explains why it has no effect when running it inside a SWAN environment script (the python script does not leave any trace in its parent process).

Unfortunately, I don’t see any other solution right now that manually exporting the variables as you do, so that the notebook python process is configured with them by SWAN.

1 Like

Ok I will do it manually then.
Thank you again :slight_smile:

Hi all,
there is an option in lb-run to set the environment variables in the current shell without spawning a new one:

eval $(lb-run --sh -A LHCb/v50r6p1)

There is still a pending issue which is that for some (yet unknown) reason this command unset an environment variable that’s used by SWAN to configure some jupyter extensions. The name of the variable is KERNEL_PROFILEPATH. I sent an email asking the LHCb computing experts whether they know why this happens. For now, the workaround is to reset the original value of the variable after running the lb-run command. I will report back if any news.

Hope this helps!