hive 使用count()函数计算出错

2022-06-06T00:39:50,064 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] impl.YarnClientImpl: Killed application application_1654437990295_0002
2022-06-06T00:39:50,070 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] reexec.ReOptimizePlugin: ReOptimization: retryPossible: false
2022-06-06T00:39:50,070 ERROR [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] ql.Driver: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
2022-06-06T00:39:50,070 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] ql.Driver: MapReduce Jobs Launched:
2022-06-06T00:39:50,106 WARN [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2022-06-06T00:39:50,106 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] ql.Driver: Stage-Stage-1: Map: 5 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
2022-06-06T00:39:50,106 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] ql.Driver: Total MapReduce CPU Time Spent: 0 msec
2022-06-06T00:39:50,106 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] ql.Driver: Completed executing command(queryId=root_20220606003856_619b3f7f-b6dd-4d20-a1ff-1a66d9aaea18); Time taken: 52.687 seconds
2022-06-06T00:39:50,107 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2022-06-06T00:39:50,107 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] exec.ListSinkOperator: RECORDS_OUT_OPERATOR_LIST_SINK_10:0, RECORDS_OUT_INTERMEDIATE:0,
2022-06-06T00:39:50,279 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] conf.HiveConf: Using the default value passed in for log id: e03cadd3-2ce9-410f-916a-0ede19bd45ce
2022-06-06T00:39:50,279 INFO [e03cadd3-2ce9-410f-916a-0ede19bd45ce main] session.SessionState: Resetting thread name to main

img

[2022-06-06 00:31:52.475]Container [pid=4958,containerID=container_1654437990295_0001_02_000018] is running 267115008B beyond the 'VIRTUAL' memory limit. Current usage: 73.1 MB of 1 GB physical memory used; 2.3 GB of 2.1 GB virtual memory used. Killing container. Dump of the process-tree for container_1654437990295_0001_02_000018 : |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE |- 4958 4957 4958 4958 (bash) 0 1 9789440 290 /bin/bash -c /opt/module/jdk1.8.0_212/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/root/appcache/application_1654437990295_0001/container_1654437990295_0001_02_000018/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1654437990295_0001/container_1654437990295_0001_02_000018 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 192.168.137.142 42274 attempt_1654437990295_0001_m_000002_1003 18 1>/opt/module/hadoop-3.1.3/logs/userlogs/application_1654437990295_0001/container_1654437990295_0001_02_000018/stdout 2>/opt/module/hadoop-3.1.3/logs/userlogs/application_1654437990295_0001/container_1654437990295_0001_02_000018/stderr |- 4970 4958 4958 4958 (java) 215 104 2512183296 18422 /opt/module/jdk1.8.0_212/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/root/appcache/application_1654437990295_0001/container_1654437990295_0001_02_000018/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1654437990295_0001/container_1654437990295_0001_02_000018 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 192.

你好,请问你解决了?

你这个报错是内存超标了
Container [pid=4958,containerID=container_1654437990295_0001_02_000018] is running 267115008B beyond the 'VIRTUAL' memory limit. Current usage: 73.1 MB of 1 GB physical memory used; 2.3 GB of 2.1 GB virtual memory used. Killing container.
设置hive的两个参数即可
set mapreduce.map.memory.mb
set mapreduce.map.java.opts
set mapreduce.reduce.memory.mb
set mapreduce.reduce.java.opts