No spark monitor display

Spark
1

Hello,
I don’t know if I messed up with my notebook configuration but I noticed that when I access the NXCALS cluster and query for some data I don’t get the spark notebook monitor anymore. Do you know how I can debug or what I can check to address this issue?

An example of cell that I was running and that had the monitor is this one (after connecting to the NXCALS cluster):

from cern.nxcals.api.extraction.data.builders import * 
df = DataQuery.builder(spark).byEntities().system('WINCCOA') \
    .startTime('2020-01-01 00:00:00.000').endTime('2020-02-01 00:00:00.000') \
    .entity().keyValue('variable_name', 'ATLMDT_XH_FIT5010.POSST') \
    .buildDataset() 
df.show()

Gianluca

2

Hello @grigolet,

also for me the monitor does not show up lately.

Cheers, Michał.

3

Hi

Thank you for reporting this. We debugged this and were able to reproduce the issue in the NXCALS cluster.

This is happening due to a python version incompatibility in this environment.

The sparkmonitor code is using a python API supported only in Python 3.7+, and this cluster is currently using python 3.6.5 from cvmfs.

We will fix the monitor to be compatible in the coming days.

Regards
Krishnan

1 Like