Configure spark.driver.maxResultSize

[topereir (Tomas Silva)]

Hello,
I am trying to increase the spark.driver.maxResultSize to 2g instead of the 1g by default, but I get the following error message:

Are we not allowed to change this parameter or is there another way to increase the maxResultSize?

Thanks in advance,
Tomas

[rcastell (Riccardo Castellotti)]

Hi,

yes you can change the parameter on your side.
You will need to do it from the ‘SparkConnector’ interface (which sets this option before the Spark Application is started). e.g.

Cheers,
Riccardo

[topereir (Tomas Silva)]

Thank you!