Local modules

Jupyter Notebooks
1

Hello there,

I have recently moved some work from my local machine to SWAN.
I’m having some issues with importing some local modules containing functions I have wrote in the years.

Importing modules from a folder seems to work only if the modules are in a subfolder in the same directory as the notebook I’m using.
see here:

Unfortunately my task are very layered, i.e. many folder/subfolder doing similar analysis on different data.
Making different copies in each subfolder seems a bit of waste.
Is there an easy solution I haven’t thought about?

thanks a lot in advance
R

2

Dear Roberto,

You can install those modules on your CERNBox user site packages folder so that Python finds them, for example if they have a setup.py and you can pip install them, as described here:

Note you also need to tick the corresponding box in the form when starting your session, as also detailed in the link above.

If your modules do not have any mechanism of installation, you can also copy them manually to the user site packages folder (you can create it if it does not exist). To find such folder, please run from the SWAN terminal:

python -m site --user-site

Best,
Enric

3

Hello Eric,

thanks for your reply.
went with the second option, these are just *.py files with custom functions, nothing installable.

I’ve tried to find the folder but here’s what happen on the terminal

[rrossi@jupyter-rrossi ~]$ python -m site --user-site
/eos/user/r/rrossi/.local/lib/python3.11/site-packages
[rrossi@jupyter-rrossi ~]$
[rrossi@jupyter-rrossi ~]$ cd /eos/user/r/rrossi/.local/lib/python3.11/site-packages
bash: cd: /eos/user/r/rrossi/.local/lib/python3.11/site-packages: No such file or directory

in the folder /eos/user/r/rrossi/.local/lib/ there’s no python3.11 anyway…

The modules file I need are located in a separate SWAN project.
Would it be possible to point to that project?

r

5

Hi Roberto, I’m not an expert but have you tried adding the path of your python files? That worked for me in a similar situation.
For example, in your notebook cell:

import sys
sys.path.append("/path/to/your/files")

you shold be able to import whatever is in /path/to/your/files after that (being it another SWAN project in your /eos/user/r/… or any other folder under /eos/experiment/…).

6

Hi Valerio,

yes this should work, thanks
as you point out, it needs to be loaded in each notebook or at least once every time one open a new SWAN session

I wonder if there is a way to write a configuration file like jupyter_notebook_config.py
so that the sys.path can be loaded automatically.

r

7

You can also create an “enviroment script”, e.g. a file named startup_script.sh file in your Swan project folder with a line like this to add the path each time you spawn a SWAN session:
export PYTHONPATH=/eos/user/r/.../here_are_your_files/:$PYTHONPATH

When starting a SWAN session you have to write the script name in the “environment script” box (see also here Select a configuration - Help).

But yet you have to do it each time.

1 Like
8

Hi Roberto,

You can create the user site packages directory if it does not exist:

mkdir -p /eos/user/r/rrossi/.local/lib/python3.11/site-packages

and move there all your .py files. Then if you restart your session and tick the box I mentioned in my first reply, your modules should be visible.

An alternative is what @vmascagn proposed (thanks Valerio!), you can create a user environment script as explained here:

That user environment script would just have a redefinition of PYTHONPATH as Valerio explained. This would allow you to place the modules anywhere in your CERNBox (not just the user site packages directory), but you will need to specify the script path every time you start your session (and not just ticking a box).

Hope this helps!

Enric