Java – spark job server: “the server cannot respond to your request in time”

I'm using spark job server to run spark jobs. It works normally But when I try to do a big job (it takes more than 40 seconds), I receive this error:

Do you need some configuration to wait for the server to answer? What should I do?

thank you

Solution

Several different timeouts can be achieved using synchronous job submission Your is from the spray can server You can configure it through the conf file:

You can set both values to infinite to completely disable these timeouts

In general, I observe at least three common different timeouts and can return your request before the job is completed:

>Akka based request timed out If the synchronization job is not completed before x seconds, a JSON response with "error" status will be returned The default value in SJS is 10 seconds, but you can override it by passing the timeout = Y parameter to the post / job request. > Spray can server timeout: idle timeout, request timeout The default spray response will be returned, making it more difficult to capture They default to 60 seconds and 40 seconds respectively. > Another timeout may come from your client rest Library... Some of them configure their default values to disable any timeout on this layer, and some of them may consider the timeout of the server and configure their own behavior according to this... For example, once you disable the timeout #2, the C rest SDK library will start within 30 seconds of winhttp timeout, #2 will wait for 40 seconds