Read large files using swan

Jupyter Notebooks
1

Hello people,

I have one question: How can I read a .root file with 2 GB using swan?

I’m using the following library to read the root file:

from ROOT import TFile, gROOT
from pandas import DataFrame

And the code to open the file:

gROOT.ProcessLine(dataframe_h)
file_new_e = TFile(f'{filesElectron[ii][0]}/{filesElectron[ii][1]}')
tree_e     = file_new_e.Get('physics')
single_e = readDirectCells(tree_e, eventMax=None, particle='e')

set(single_e['layer'])

The problem is that when I ran the code, the memory for the swan section (16 GB) was fully used, and the kernel died!

2

Hello,

Could you give more details about your code? If you read the file just with ROOT (TFile and TTree interfaces) the amount of memory that is used should be small. I see that you also import pandas. Do you use pandas to load data into memory (e.g. an in memory dataframe)?

In SWAN you have an indicator of the memory consumed by your session (in the notebook interface, above the first cell), so if you run your program in multiple cells you can check which one is responsible for the memory consumption. That will give you a better idea of where the issue is.