File "C:\Users\tongmig\PycharmProjects\PySpark\00_example\HelloWorld.py", line 10, in <module>

运行python spark报错Traceback(most recent call last)


C:\Users\tongmig\PycharmProjects\pythonProject\venv\Scripts\python.exe C:/Users/tongmig/PycharmProjects/PySpark/00_example/HelloWorld.py 
Traceback (most recent call last):
  File "C:\Users\tongmig\PycharmProjects\PySpark\00_example\HelloWorld.py", line 10, in 
    sc = SparkContext(conf=conf)
  File "C:\Users\tongmig\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\context.py", line 144, in __init__
    SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
  File "C:\Users\tongmig\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\context.py", line 339, in _ensure_initialized
    SparkContext._gateway = gateway or launch_gateway(conf)
  File "C:\Users\tongmig\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\java_gateway.py", line 101, in launch_gateway
    proc = Popen(command, **popen_kwargs)
  File "C:\Users\tongmig\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Users\tongmig\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 1420, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] 系统找不到指定的文件。

代码如下


```python


# coding:utf8
from pyspark import SparkConf, SparkContext
import os

if __name__ == '__main__':
    os.environ['SPARK_HOME'] = "C:\\Users\\tongmig\\AppData\\Local\\Programs\\Python\\Python39\\Lib\\site-packages" \
                               "\\pyspark "
    conf = SparkConf().setAppName("WordCountHelloWorld")
    # 通过SparkConf对象构建SparkContext对象
    sc = SparkContext(conf=conf)

    # 需求 : wordcount单词计数, 读取HDFS上的words.txt文件, 对其内部的单词统计出现 的数量
    # 读取文件
    file_rdd = sc.textFile("C:\\Users\\tongmig\\Desktop\\word_count.txt")

    # 将单词进行切割, 得到一个存储全部单词的集合对象
    words_rdd = file_rdd.flatMap(lambda line: line.split(" "))

    # 将单词转换为元组对象, key是单词, value是数字1
    words_with_one_rdd = words_rdd.map(lambda x: (x, 1))

    # 将元组的value 按照key来分组, 对所有的value执行聚合操作(相加)
    result_rdd = words_with_one_rdd.reduceByKey(lambda a, b: a + b)

    # 通过collect方法收集RDD的数据打印输出结果
    print(result_rdd.collect())

检查一下这个路径是否正确: file_rdd = sc.textFile("C:\Users\tongmig\Desktop\word_count.txt")