CERN Accelerating science

Update pandas version in swan notebook

#1

I have done the update by using ‘pip install --user --upgrade pandas’.
Collecting pandas
Using cached https://files.pythonhosted.org/packages/f9/b9/9ad570258ce4fe504bd23002154f9e6f09bf7110359d271e4ba1664f7281/pandas-1.0.2-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: python-dateutil>=2.6.1 in /usr/local/lib/swan (from pandas) (2.8.1)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /cvmfs/sft.cern.ch/lcg/views/LCG_96python3/x86_64-centos7-gcc8-opt/lib/python3.6/site-packages (from pandas) (2019.1)
Requirement already satisfied, skipping upgrade: numpy>=1.13.3 in /cvmfs/sft.cern.ch/lcg/views/LCG_96python3/x86_64-centos7-gcc8-opt/lib/python3.6/site-packages (from pandas) (1.16.4)
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/local/lib/swan (from python-dateutil>=2.6.1->pandas) (1.13.0)
Installing collected packages: pandas
Successfully installed pandas-1.0.2
Note: you may need to restart the kernel to use updated packages.

However, my notebook script still picks up the old version 0.24.2. Just wondering where should I run the pip command? Run it at my cernbox home directory?
I did ‘pip3 install --upgrade pandas’ and indeed I can import pandas locally with the latest version. In the jupter notebook, pandas still picks up the old version.

#2

Dear Zhen,

You will need to add your CERNBox directory as PYTHONPATH:
export PYTHONPATH=$CERNBOX_HOME/.local/lib/python3.5/site-packages:$PYTHONPATH

Please, follow the instructions provided at https://github.com/swan-cern/help/blob/master/advanced/install_packages.md

Regards,
Enrico

#3

Hi Enrico,

I start my jupyter notebook from swan.web.cern.ch , and my lxplus home directory is not my CERNBOX home directory at the moment. Therefore, I don’t know which pythonpath I should add to which configuration file. For instance, I have add /eos/user/z/zyan/.local/lib/python3.6/site-packages into PYTHONPATH in my .zshrc login file, however the jupyter still doesn’t know it at all.
How can I define my CERNBOX_HOME as my lxplus home directory?

#4

Dear Zhen,

It might be that your home directory on lxplus is still on AFS.

SWAN, however, uses EOS/CERNBox only, hence the path you should add to the PYTHONPATH is /eos/user/z/zyan/.local/lib/python3.6/site-packages.

You should set this path in a bash script that is loaded when starting your sessions.
For instance,

  • create the file /eos/user/z/zyan/swan_config.sh with content export PYTHONPATH=/eos/user/z/zyan/.local/lib/python3.6/site-packages:$PYTHONPATH
  • use this file (/eos/user/z/zyan/swan_config.sh) in the field “Environment script” when you are asked to configure your environment at swan.cern.ch

Regards,
Enrico

#6

Just let you know I got the configuration work now.

#7

Excellent! Happy to know it works!