Java – spark accumulator reset

There is no way to reset the spark accumulator through the driver so that the actuator accumulator will also be cleaned

I have an accumulator, actually a list > all executors add tuples to the accumulator and the driver reads them The problem is that the execution heap is full

When the driver reads all the data, the executor does not need local data

thank you

Solution

I wonder if you have ever tried to reset the driver using the accumulator's public method setvalue()

from https://issues.apache.org/jira/browse/SPARK-3885