Pyspark: exception: the Java gateway process exited before sending the driver’s port number

I tried to run pyspark. Com on my MacBook When I try to start it, I get an error:

Exception: Java gateway process exited before sending the driver its port number

When SC = sparkcontext() is called at startup I tried to run the following command:

./bin/pyspark
./bin/spark-shell
export PYSPARK_SUBMIT_ARGS="--master local[2] pyspark-shell"

of no avail. I've also seen it here:

Spark + Python – Java gateway process exited before sending the driver its port number?

But the problem has never been solved Please help! thank you.

Solution

This should help you

One solution is to add the pyspark shell to the shell environment variable pyspark_ SUBMIT_ In args:

export PYSPARK_SUBMIT_ARGS="--master local[2] pyspark-shell"

python / pyspark / java_ gateway. Py has a change if the user sets pyspark_ SUBMIT_ Args variable, pyspark is required_ SUBMIT_ Args contains pyspark shell