hive on spark的inser报错，尝试了很多方法
如果是单独的hive，不基于spark的话，insert是正常mapreduce的，但是on spark就有问题了
```bash
2023-03-03T10:33:09,958 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] reexec.ReOptimizePlugin: ReOptimization: retryPossible: false
2023-03-03T10:33:09,958 ERROR [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Spark job failed during runtime. Please check stacktrace for the root cause.
2023-03-03T10:33:09,959 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Completed executing command(queryId=atguigu_20230303103233_f2944011-8f5f-43fc-a1a5-34028b44216d); Time taken: 32.994 seconds
2023-03-03T10:33:09,959 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-03T10:33:09,980 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] conf.HiveConf: Using the default value passed in for log id: d03963e7-550c-4536-ab5e-706ceff71f0a
2023-03-03T10:33:09,980 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] session.SessionState: Resetting thread name to main
2023-03-03T10:35:11,954 INFO [main] conf.HiveConf: Using the default value passed in for log id: d03963e7-550c-4536-ab5e-706ceff71f0a
2023-03-03T10:35:11,954 INFO [main] session.SessionState: Updating thread name to d03963e7-550c-4536-ab5e-706ceff71f0a main
2023-03-03T10:35:11,956 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Compiling command(queryId=atguigu_20230303103511_29a38ccd-7535-4d35-a89d-8c50da46748e): insert into table ido values(1,'a')
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore...
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=Cleaning up thread local RawStore...
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-03T10:35:11,970 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Starting Semantic Analysis
2023-03-03T10:35:11,973 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore
2023-03-03T10:35:11,974 WARN [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored
2023-03-03T10:35:11,974 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.ObjectStore: ObjectStore, initialize called
2023-03-03T10:35:12,015 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL
2023-03-03T10:35:12,016 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.ObjectStore: Initialized ObjectStore
2023-03-03T10:35:12,016 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=atguigu (auth:SIMPLE) retries=1 delay=1 lifetime=0
2023-03-03T10:35:12,016 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,016 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,049 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis
2023-03-03T10:35:12,049 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for source tables
2023-03-03T10:35:12,049 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for subqueries
2023-03-03T10:35:12,049 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for destination tables
2023-03-03T10:35:12,049 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,049 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,075 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis
2023-03-03T10:35:12,089 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Context: New scratch dir is hdfs://hadoop102:8020/tmp/hive/atguigu/d03963e7-550c-4536-ab5e-706ceff71f0a/hive_2023-03-03_10-35-11_968_509342825293119927-1
2023-03-03T10:35:12,102 INFO [Thread-19] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-03-03T10:35:12,131 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_not_null_constraints : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,131 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_not_null_constraints : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,135 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_primary_keys : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,135 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_primary_keys : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,138 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_primary_keys : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,138 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_primary_keys : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,140 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_unique_constraints : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,140 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_unique_constraints : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,143 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=_dummy_database foreigntbl=_dummy_table
2023-03-03T10:35:12,144 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=_dummy_database foreigntbl=_dummy_table
2023-03-03T10:35:12,172 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_databases: @hive#
2023-03-03T10:35:12,172 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_databases: @hive#
2023-03-03T10:35:12,176 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_materialized_views_for_rewriting: db=@hive#default
2023-03-03T10:35:12,176 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_materialized_views_for_rewriting: db=@hive#default
2023-03-03T10:35:12,188 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for source tables
2023-03-03T10:35:12,188 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for subqueries
2023-03-03T10:35:12,189 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for source tables
2023-03-03T10:35:12,189 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,189 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive._dummy_database._dummy_table
2023-03-03T10:35:12,193 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for subqueries
2023-03-03T10:35:12,193 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for destination tables
2023-03-03T10:35:12,193 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for destination tables
2023-03-03T10:35:12,193 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,193 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,221 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Context: New scratch dir is hdfs://hadoop102:8020/tmp/hive/atguigu/d03963e7-550c-4536-ab5e-706ceff71f0a/hive_2023-03-03_10-35-11_968_509342825293119927-1
2023-03-03T10:35:12,227 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] common.FileUtils: Creating directory if it doesn't exist: hdfs://hadoop102:8020/user/hive/warehouse/ido/.hive-staging_hive_2023-03-03_10-35-11_968_509342825293119927-1
2023-03-03T10:35:12,233 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_not_null_constraints : tbl=hive.default.ido
2023-03-03T10:35:12,233 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_not_null_constraints : tbl=hive.default.ido
2023-03-03T10:35:12,238 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_check_constraints : tbl=hive.default.ido
2023-03-03T10:35:12,239 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_check_constraints : tbl=hive.default.ido
2023-03-03T10:35:12,243 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,243 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,270 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Generate an operator pipeline to autogather column stats for table default.ido in query insert into table ido values(1,'a')
2023-03-03T10:35:12,272 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,272 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,294 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for source tables
2023-03-03T10:35:12,294 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,294 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,320 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for subqueries
2023-03-03T10:35:12,320 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Get metadata for destination tables
2023-03-03T10:35:12,327 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Context: New scratch dir is hdfs://hadoop102:8020/tmp/hive/atguigu/d03963e7-550c-4536-ab5e-706ceff71f0a/hive_2023-03-03_10-35-12_270_5676270708991864014-1
2023-03-03T10:35:12,329 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] common.FileUtils: Creating directory if it doesn't exist: hdfs://hadoop102:8020/tmp/hive/atguigu/d03963e7-550c-4536-ab5e-706ceff71f0a/hive_2023-03-03_10-35-12_270_5676270708991864014-1/-mr-10000/.hive-staging_hive_2023-03-03_10-35-12_270_5676270708991864014-1
2023-03-03T10:35:12,332 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan.
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for FS(4)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for FS(11)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for SEL(10)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for GBY(9)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for RS(8)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for GBY(7)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for SEL(6)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for SEL(3)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for UDTF(2)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for SEL(1)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ppd.OpProcFactory: Processing for TS(0)
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] optimizer.ColumnPrunerProcFactory: RS 8 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]}
2023-03-03T10:35:12,333 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] optimizer.ColumnPrunerProcFactory: RS 8 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]}
2023-03-03T10:35:12,339 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SetSparkReducerParallelism: Number of reducers for sink RS[8] was already determined to be: 1
2023-03-03T10:35:12,342 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] metastore.HiveMetaStore: 0: get_table : tbl=hive.default.ido
2023-03-03T10:35:12,342 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] HiveMetaStore.audit: ugi=atguigu ip=unknown-ip-addr cmd=get_table : tbl=hive.default.ido
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Examining input format to see if vectorization is enabled.
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Map vectorization enabled: false
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Map vectorized: false
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Map vectorizedVertexNum: 0
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Map enabledConditionsMet: [hive.vectorized.use.vectorized.input.format IS true]
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Map enabledConditionsNotMet: [Could not enable vectorization due to partition column names size 1 is greater than the number of table column names size 0 IS false]
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Map inputFileFormatClassNameSet: [org.apache.hadoop.hive.ql.io.NullRowsInputFormat]
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Validating and vectorizing ReduceWork... (vectorizedVertexNum 1)
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Reduce vectorization enabled: true
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Reduce vectorized: false
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Reduce notVectorizedReason: Aggregation Function expression for GROUPBY operator: UDF compute_stats not supported
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Reduce vectorizedVertexNum: 1
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Reducer hive.vectorized.execution.reduce.enabled: true
2023-03-03T10:35:12,367 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] physical.Vectorizer: Reducer engine: spark
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] common.HiveStatsUtils: Error requested is 20.0%
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] common.HiveStatsUtils: Choosing 16 bit vectors..
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] parse.CalcitePlanner: Completed plan generation
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Semantic Analysis Completed (retrial = false)
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col1, type:int, comment:null), FieldSchema(name:col2, type:string, comment:null)], properties:null)
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Completed compiling command(queryId=atguigu_20230303103511_29a38ccd-7535-4d35-a89d-8c50da46748e); Time taken: 0.412 seconds
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] reexec.ReExecDriver: Execution #1 of query
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-03T10:35:12,368 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Executing command(queryId=atguigu_20230303103511_29a38ccd-7535-4d35-a89d-8c50da46748e): insert into table ido values(1,'a')
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Query ID = atguigu_20230303103511_29a38ccd-7535-4d35-a89d-8c50da46748e
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Total jobs = 1
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Launching Job 1 out of 1
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SparkTask: In order to change the average load for a reducer (in bytes):
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SparkTask: set hive.exec.reducers.bytes.per.reducer=
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SparkTask: In order to limit the maximum number of reducers:
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SparkTask: set hive.exec.reducers.max=
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SparkTask: In order to set a constant number of reducers:
2023-03-03T10:35:12,369 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] spark.SparkTask: set mapreduce.job.reduces=
2023-03-03T10:35:12,383 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Context: New scratch dir is hdfs://hadoop102:8020/tmp/hive/atguigu/d03963e7-550c-4536-ab5e-706ceff71f0a/hive_2023-03-03_10-35-11_968_509342825293119927-1
2023-03-03T10:35:12,468 INFO [RPC-Handler-2] client.SparkClientImpl: Received result for 767a7788-8c54-4c6b-a81a-96b5e0a6c063
2023-03-03T10:35:13,437 ERROR [d03963e7-550c-4536-ab5e-706ceff71f0a main] status.SparkJobMonitor: Job failed with java.lang.ClassNotFoundException: org.apache.spark.AccumulatorParam
java.lang.NoClassDefFoundError: org/apache/spark/AccumulatorParam
at org.apache.hive.spark.counter.SparkCounterGroup.createCounter(SparkCounterGroup.java:52)
at org.apache.hive.spark.counter.SparkCounters.createCounter(SparkCounters.java:71)
at org.apache.hive.spark.counter.SparkCounters.createCounter(SparkCounters.java:67)
at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient$JobStatusJob.call(RemoteHiveSparkClient.java:350)
at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:378)
at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:343)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.AccumulatorParam
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more
2023-03-03T10:35:13,446 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] reexec.ReOptimizePlugin: ReOptimization: retryPossible: false
2023-03-03T10:35:13,447 ERROR [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Spark job failed during runtime. Please check stacktrace for the root cause.
2023-03-03T10:35:13,447 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Completed executing command(queryId=atguigu_20230303103511_29a38ccd-7535-4d35-a89d-8c50da46748e); Time taken: 1.079 seconds
2023-03-03T10:35:13,447 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-03T10:35:13,454 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] conf.HiveConf: Using the default value passed in for log id: d03963e7-550c-4536-ab5e-706ceff71f0a
2023-03-03T10:35:13,454 INFO [d03963e7-550c-4536-ab5e-706ceff71f0a main] session.SessionState: Resetting thread name to main
```
应该是spark没装好吧 你在spark上执行别的sql能正常运行吗
有没有解决了，兄弟，我也遇见了这个问题