Hello all!
I am running into the following error, while extracting data from PyTimber.
It looks to me that the Apache Hadoop token is expired. Is there a way to set a longer expiration date for the token or to automatically (or manually) renew it?
org.apache.spark.SparkException: org.apache.spark.SparkException: Job aborted due to stage failure: Task 15798 in stage 10.0 failed 1 times, most recent failure: Lost task 15798.0 in stage 10.0 (TID 71685) (ee37c7f18b81 executor driver): org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (token for edfarina: HDFS_DELEGATION_TOKEN owner=edfarina, renewer=yarn, realUser=hswan/swan-proto4.cern.ch@CERN.CH, issueDate=1644563832326, maxDate=1645168632326, sequenceNumber=1396218, masterKeyId=1947) is expired, current time: 2022-02-12 08:32:17,871+0100 expected renewal time: 2022-02-12 08:17:12,326+0100
Many thanks for any help you may provide me!
Best,
Edoardo Farina