TensorFlow 2 support?

mpalteng · April 10, 2020, 9:06am

Hi to everyone,

SHORT-TERM PROBLEM:
I was looking for TensorFlow 2, but I was only able to import TensorFlow 1, I downloaded it as local user but It still import the older version.
In case this is already possible, I kindly ask for your help to make it work.

LONG-TERM PROPOSAL:
In case this is not supported, I suggest the introduction of something similar to this approach of Colab in which they have a magic command to tell the notebook to use TensorFlow 2.
https://colab.research.google.com/notebooks/tensorflow_version.ipynb

Thanks in advance and have a nice day

mpalteng · April 10, 2020, 10:11am

I managed to install tensorFlow 2 just for me as user, this might be helpful to someone else.
I used the following commands:

!python3 -m pip install --user tensorflow==2.1
import os
os.chdir(“/eos/user/m/mpalteng/.local/lib/python3.7/site-packages”) # change mpalteng with your username
import tensorflow as tf
print(tf._version_)

If there is a better solution please share it. Thanks

etejedor · April 14, 2020, 7:32am

Hi Matteo,

Thank you for sharing your solution. The solution we recommend is explained in the help of SWAN here:

github.com

swan-cern/help/blob/master/advanced/install_packages.md

# Install packages in CERNBox

LCG releases on CVMFS incorporate new packages quite frequently, so if you think there is a missing package that can be 
potentially useful for a significant number of users, please [let the SWAN team know](mailto:swan-talk@cern.ch) and 
[contact the librarians directly](https://sft.its.cern.ch/jira/projects/SPI).

On the other hand, you can install packages on your CERNBox and, if necessary, configure your environment to pick them 
up in SWAN. 

## Python

A typical case is the installation of Python packages, which requires to run pip from a SWAN terminal:

   `pip install --user package_name`

If this fails because you are trying to install an updated version of a package that already exists in CVMFS, you will need to add the `--upgrade` flag.

Then, it would be necessary to add the local installation path to `PYTHONPATH`, by creating a bash startup script that configures that variable (don't forget to call this startup script in the session configuration menu):

    export PYTHONPATH=$CERNBOX_HOME/.local/lib/python3.5/site-packages:$PYTHONPATH

This file has been truncated. show original

which relies on a user environment script to set the PYTHONPATH (so that you don’t have to change the working directory to your site-packages in every notebook).

As for your long-term proposal, thank you for pointing us to that feature. However, given how SWAN sessions work (you pick an LCG software stack with a given version of a set of packages), it is more likely that, in the future, you will choose the version of tensorflow you want to work with by using an older stack or a newer stack. For the moment, I am not aware of any plans to update tensorflow to 2.0 in the LCG releases, but as soon as that happens we will have it available in SWAN right away (first in the bleeding edge software stack).

mato · April 14, 2020, 9:00am

@etejedor, we are preparing a new stack with tensorflow 2.1 that could appear in bleeding edge in the next few days if the experiments give the green light to upgrade the version of tensorflow, which is not API compatible. To be discussed at the LIM meeting today.

etejedor · April 14, 2020, 9:53am

Ok thanks for the update @mato !