Dear all,
I am trying to open a file stored on eos like this:
spark.read.load(“root://eosuser.cern.ch//eos/user/n/nraab/SWAN_projects/MC_efficiency/mytupleMC16d.root”)
However I get the error that the Path does not exist. Do I need to mount eos in any way before bein able to use it? When not specifying “root://eosuser.cern.ch”, the default directory seems to be “hdfs://analytix/”
Many thanks in advance,
Naomi
@pkothuri can you give a hand to Naomi with this issue? Does she need to create a kerberos ticket and send it to the workers?
@nraab can you try using the cloud containers clusters, we do send the krb to the spark executors for the ‘cloud containers’ cluster
Hi @pkothuri , many thanks, by doing that I can now access eos from SWAN, however, simple tasks like
sql = spark.sql
sql(“select ‘Hello World!’”).show()
don’t seem to work with that configuration, it get’s stuck in ‘running’ with 0 cores and 0 executors.