# Problem using PyTimber.pagestore on SWAN

**URL:** <https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304>\
**Category:** Uncategorized\
**Created:** [October 23, 2020, 1:41pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304 "2020-10-23T13:41:59Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [October 23, 2020, 1:42pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/1 "2020-10-23T13:42:00Z")

</div>

Hello,

I want to use the PyTimber pagestore function to save data files through SWAN on to CERNBox. When I try to run the pagestore function, the files however do not seem to be saved correctly. It does work OK when I run it locally on my PC. Similarly, if I try to read these pagestore files, the data does not seem to be read correctly, even if I try to read data that I have already confirmed should work. Any ideas…?

Some code that uses said function and saves the data (this exact code runs without problem on my local machine but not on SWAN!):  
[https://gitlab.cern.ch/abt-projects/pulsed-power/agnieszka-vacuum-scripts/-/blob/master/import\_data.py](https://gitlab.cern.ch/abt-projects/pulsed-power/agnieszka-vacuum-scripts/-/blob/master/import_data.py)

Thank you,  
Oskar

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [October 26, 2020, 10:09am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/2 "2020-10-26T10:09:44Z")

</div>

Hello Oskar,

I can not access the link to take a look at your code,  
can you may post some minimal example and some instructions to reproduce the problem?

Best  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [October 28, 2020, 9:20am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/3 "2020-10-28T09:20:43Z")

</div>

Hi Omar,

Sorry, I didn’t realize you couldn’t access it.

The following code seems to reproduce the problem as well - let me know if you need anything else! Thank you.

Oskar

```
import pytimber
from pytimber import pagestore
import time, calendar
#from numba import jit
%matplotlib notebook
ldb = pytimber.LoggingDB()

mydbPressureMDB = pagestore.PageStore('lhcPressure.db', './')

t1 = "2012-07-01 00:00:00"
t2 = "2012-07-01 23:59:59"
ts1 = calendar.timegm(time.strptime(t1,"%Y-%m-%d %H:%M:%S")) - 0 * 3600
ts2 = calendar.timegm(time.strptime(t2,"%Y-%m-%d %H:%M:%S")) - 0 * 3600

keys_beam1 = ['MKI.D5L2.B1:PRESSURE', 'VGPB.118.5L2.C.PR', 'MKI.B5L2.B1:PRESSURE', 'MKI.A5L2.B1:PRESSURE', 'VGPB.137.5L2.B.PR', 'VGPB.14.5L2.B.PR', 'VGPB.176.5L2.B.PR', 'VGPB.192.5L2.B.PR', 'VGPB.59.5L2.B.PR', 'VGPB.98.5L2.B.PR', 'LHC.BCTFR.A6R4.B1:BEAM_INTENSITY', 'MKI.A5L2.B1:PRESSURE_INT', 'MKI.B5L2.B1:PRESSURE_INT', 'MKI.C5L2.B1:PRESSURE_INT', 'MKI.D5L2.B1:PRESSURE_INT']
keys_beam2 = ['MKI.D5R8.B2:PRESSURE', 'MKI.C5R8.B2:PRESSURE', 'MKI.B5R8.B2:PRESSURE', 'MKI.A5R8.B2:PRESSURE', 'VGPB.138.5R8.R.PR', 'VGPB.14.5R8.R.PR', 'VGPB.176.5R8.R.PR', 'VGPB.192.5R8.R.PR', 'VGPB.59.5R8.R.PR', 'VGPB.98.5R8.R.PR', 'LHC.BCTFR.A6R4.B2:BEAM_INTENSITY', 'MKI.A5R8.B2:PRESSURE_INT', 'MKI.B5R8.B2:PRESSURE_INT', 'MKI.C5R8.B2:PRESSURE_INT', 'MKI.D5R8.B2:PRESSURE_INT']

def Store_data(keys):
    start_time = time.time()
    timber_data = ldb.getAligned(keys, ts1, ts2)
    end_time = time.time()
    for key in keys:
        mydbPressureMDB.store_variable(key, timber_data['timestamps'], timber_data[key]);

print('Aligning and storing data from ',t1,' to ',t2)
Store_data(keys_beam1+keys_beam2)
```

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [October 30, 2020, 10:28am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/4 "2020-10-30T10:28:45Z")

</div>

Hello Oskar,

I did it, I ran it in SWAN and in my computer using LCG\_97a.  
But I can not see the difference in ‘lhcPressure.db’

I did a small code to see the values in the table pages in sqlite in SWAN swan001.cern.ch

 ![image](https://swan-community.web.cern.ch/uploads/default/original/1X/0ecebea7b3f7a8f62eb65f8c0500f284352c30e3.png)

and I ran it in my local host

 ![image](https://swan-community.web.cern.ch/uploads/default/original/1X/45d7a5d3ca559ff19140b8db4ffcf348e3f1ba24.png)

which is your error?

Best,  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [October 30, 2020, 2:37pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/5 "2020-10-30T14:37:29Z")

</div>

Hi Omar,

How do you mean “can’t see the difference in lhcPressure.db’?”

The error that I am getting is

FileNotFoundError: [Errno 2] No such file or directory: ‘./1.idx’

I find this strange because this file 1.idx is one of the files that the program will output if I run it on my local machine.

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [October 30, 2020, 2:49pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/6 "2020-10-30T14:49:29Z")

</div>

Hi Oskar,

I can see that files too, can you tell me which stack are you using and which SWAN node?

 ![image](https://swan-community.web.cern.ch/uploads/default/original/1X/3ade017a8db34aa413630fc6b8debb3d3562be1c.png)

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [November 5, 2020, 9:37am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/7 "2020-11-05T09:37:55Z")

</div>

Hi Omar,

I am just running it on the default stack, 97a. How do I see which SWAN Node I am using?

Thank you!

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [November 5, 2020, 10:06am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/8 "2020-11-05T10:06:44Z")

</div>

Hi Oskar,

in the url in the browser, in the previous post, you can see the image where the url  
is swan001.cern.ch.

Cheers  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [November 9, 2020, 7:57am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/9 "2020-11-09T07:57:28Z")

</div>

Hi Omar,

I tried just now on node 001 and the problem remains the same. I’m pretty sure that I’ve used other nodes too.

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [November 9, 2020, 1:43pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/10 "2020-11-09T13:43:08Z")

</div>

Hello Oskar,

I was able to reproduce the problem,  
The notebook on EOS is not producing the files, I created a notebook in /tmp using a symlink and it works.

I am going to ask our experts on EOS.  
Best  
Omar.

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [November 11, 2020, 3:49pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/11 "2020-11-11T15:49:56Z")

</div>

Hi @objorkqv ,

can you please try to run your code again?  
we think that the problem was solved.

Cheers  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [November 24, 2020, 1:22pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/12 "2020-11-24T13:22:25Z")

</div>

Hello Omar,

I’m very sorry that I am responding so slowly - the SWAN community page is not sending me any notifications for some reason.

I tried running it again but the error message was the same:

[Errno 2] No such file or directory: ‘/eos/…/1.idx’

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [December 9, 2020, 1:30pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/13 "2020-12-09T13:30:43Z")

</div>

Hi Omar,

Do you have any news on this topic? Many thanks.

Oskar

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [December 9, 2020, 1:54pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/14 "2020-12-09T13:54:49Z")

</div>

Hi Oskar,

Yes, I talked today with a colleage that is a expert on EOS,  
we think may some cache is causing the problem, our plan is to restart the EOS services,  
to see if this solves the problem, otherwise I think we need to open a ticket to the EOS experts.

As soon as we do the restart of the services I will let you know to do the test.

Apologize for the late reply, this is wierd issue and we are trying to figure out what is happening.

Best Regards  
Omar.

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [December 10, 2020, 11:54am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/15 "2020-12-10T11:54:24Z")

</div>

Hi @objorkqv  
can you please try your code again in the machine swan005.cern.ch, it was already rebooted  
and the code works for me

 ![image](https://swan-community.web.cern.ch/uploads/default/original/1X/f8278a359fb710798c5385f0975377d7848f09d0.png)

Cheers  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [December 11, 2020, 8:09am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/16 "2020-12-11T08:09:30Z")

</div>

Hi @ozapatam

I tried going directly to swan005.cern.ch but it says ‘This machine is scheduled for maintenance.  
Please, always use [swan.cern.ch](https://swan.cern.ch/) instead of going directly to a specific machine.’

Cheers,  
Oskar

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [December 14, 2020, 10:27am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/17 "2020-12-14T10:27:19Z")

</div>

Hello Oskar,

Can you please try again?

It should be fine now 😃  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [December 14, 2020, 11:31am UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/18 "2020-12-14T11:31:13Z")

</div>

Hi Omar,

I could access swan005.cern.ch now but the scripts still does not run. Sorry!

Cheers,  
Oskar

---

<div class="post-metadata">

**Author:** ![ozapatam](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/a183cd/32.png) [@ozapatam](https://swan-community.web.cern.ch/u/ozapatam)\
**Post date:** [December 14, 2020, 12:31pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/19 "2020-12-14T12:31:27Z")

</div>

Hi @objorkqv

In that case please open a ticket for the EOS project experts

> **[Service Element - CERN Service Portal: easy access to services at CERN](https://cern.service-now.com/service-portal?id=service_element&name=eos-service)**

please add me in the watch list to see if they need something from SWAN side.

Best  
Omar.

---

<div class="post-metadata">

**Author:** ![objorkqv](https://swan-community.web.cern.ch/letter_avatar_proxy/v4/letter/o/8baadc/32.png) [@objorkqv](https://swan-community.web.cern.ch/u/objorkqv)\
**Post date:** [December 14, 2020, 1:15pm UTC](https://swan-community.web.cern.ch/t/problem-using-pytimber-pagestore-on-swan/304/20 "2020-12-14T13:15:56Z")

</div>

Hi Omar,

Will do. Thanks for your efforts to help us out anyway.

Kind regards,  
Oskar