在jupyter notebook 中运行
import findspark
findspark.init()
出现报错:Couldn't find Spark, make sure SPARK_HOME env is set or Spark is in an expected location (e.g. from homebrew installation)
import findspark
findspark.init()
Couldn't find Spark, make sure SPARK_HOME env is set or Spark is in an expected location (e.g. from homebrew installation)
正确运行,不出现 SPARK_HOME env设置和路径位置的报错
一般我会在一开始设置好SPARK_HOME
环境变量:
import os
os.environ['SPARK_HOME'] = /path/to/your/spark
os.environ['PYSPARK_PYTHON'] = /path/to/pyspark虚拟环境的python解释器
os.environ['PYSPARK_DRIVER_PYTHON'] = /path/to/pyspark虚拟环境的python解释器