CERN Accelerating science

Eos access on spark clusters

Dear SWAN team,

I discovered that EOS is not mounted on spark clusters, thus, using eos files with spark one must have to add (for the COMPASS experiment) " root://eoscompass.cern.ch/" as prefix for all file paths inside the eos/experiment/compass folder.

This works like a charm, however I had recently to move from my experiment folder to my cernbox. I tried to add the prefix “root://eosuser.cern.ch/” this time, but this does not work. I also changed my $EOS_MGM_URL variable (if that has to do something with it) but same issue.

[vbrian@lxplus779 vbrian]$ root -l root://eosuser.cern.ch//eos/home-v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root
root [0] 
Attaching file root://eosuser.cern.ch//eos/home-v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root as _file0...
Error in <TNetXNGFile::Open>: [ERROR] Server responded with an error: [3011] Unable to open file /eos/home-v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root; No such file or directory

(TFile *) nullptr

Then my question is the following, what prefix should I use then, or what variable should I set up in order to access files on my cernbox, using spark clusters ?

NB. I use the PyRDF module, and removing the prefix and using the local backend (not spark) it obviously works. When using the spark backend, nothing works anymore besides working on my experiment folder.

Bests,
Brian

1 Like

Dear Brain

EOS is not mounted on the spark clusters and recommended way is to access using xrootd. I believe you should access eosuser instance as root://eosuser.cern.ch/eos/user/v/vbrian

I tried below to access a root file from my cernbox and it is fine

-bash-4.2$ root -l root://eosuser.cern.ch//eos/user/p/pkothuri/0.root
root [0]
Attaching file root://eosuser.cern.ch//eos/user/p/pkothuri/0.root as _file0...
(TFile *) 0x210a1f0

Please try and let us know, thank you

Best Regards,
Prasanth

Dear Brian,

your cernbox home directory is the home directory on SWAN, so simply use “~” to refer to your cernbox.

HTH,
cheers, andreas

Dear Prasanth, Dear Andreas,

Here is what I got from SWAN console, am I doing something wrong ?

bash-4.2$ root -l /eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root
root [0]
Attaching file /eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root as _file0...
(TFile *) 0x29632d0
root [1] .q
bash-4.2$ root -l root://eosuser.cern.ch//eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root
root [0]
Attaching file root://eosuser.cern.ch//eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root as _file0...
Error in <TNetXNGFile::Open>: [ERROR] Server responded with an error: [3011] Unable to open file /eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root; No such file or directory

(TFile *) nullptr

Bests,
Brian

Hi Brian,
instead of:

can you simply try:

root -l ~/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root

?
HTH,
cheers, andreas

Dear Andreas,
I have exactly the very same output as above, except that xrootd does not like relative paths…so this does not solve my issue…

bash-4.2$ root -l ~/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root
root [0]
Attaching file /eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root as _file0...
(TFile *) 0x18dba40
root [1] .q
bash-4.2$ root -l root://eosuser.cern.ch/~/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root
root [0]
Attaching file root://eosuser.cern.ch/~/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root as _file0...
Error in <TNetXNGFile::Open>: [ERROR] Server responded with an error: [3010] Opening relative path '~/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root' is disallowed.

(TFile *) nullptr

Bests,
Brian

Hi @vbrian,

Can you try:

bash-4.2$ root -l
root [0] auto f = TFile::Open("root://eosuser.cern.ch//eos/user/v/vbrian/outputs/dvcs2016-P09slot5.1-08October/SWAN_WS/dvcs2016-P09slot5.1-08October.root")

The URL seems well formed to me, @dalvesde is eosuser.cern.ch something that can still be used in the URL?

Hi Enric,

I tried and I got the same error. Also it seems to work for Prasanth as a previous example. The fact is that it may have never worked for me ; I contacted the service desk and an eos-guy to help me with that, since I had previous problems months ago with my cernbox due to a migration of accounts. So this is a very ponctual issue in terms of affected people.
Anyhow, will keep this topic updated with news when solved for anyone in the same case.

Bests,
Brian

Dear everyone,

So that was quick.

Hi Brian,     

Could you try again now? We realized there was a link missing for your user to the new EOS backend (that would explain why it was not redirecting through eosuser.ch).

Apparently this could have been due to a faulty migration process' step last year.

Please check it, and let me know if that worked.

Cheers,
Aritz.

And my swan is happy now ! (it works). Sorry for having bothered you with an issue which was not SWAN’s fault.

Bests,
Brian

Dear Brain

Good to know it works ! The post at the least provides the correct way to reference eosuser with xrootd , root://eosuser.cern.ch//eos/user/v/vbrian instead of root://eosuser.cern.ch//eos/home-v/vbrian

Best Regards,
Prasanth