FLINK提交任务失败

问题遇到的现象和发生背景

flink提交cdc任务失败，本地idea环境可以正常运行

运行结果及报错内容

2022-08-17 17:08:36,915 WARN org.apache.flink.runtime.webmonitor.handlers.JarRunHandler [] - Configuring the job submission via query parameters is deprecated. Please migrate to submitting a JSON request instead.
2022-08-17 17:08:37,112 INFO org.apache.flink.client.ClientUtils [] - Starting program (detached: true)
2022-08-17 17:08:37,123 DEBUG org.apache.hadoop.security.UserGroupInformation [] - Hadoop login
2022-08-17 17:08:37,124 INFO cloud.tiemao.bigdata.utils.PropertyUtil [] - 开始加载application.properties文件内容.......
2022-08-17 17:08:37,124 INFO cloud.tiemao.bigdata.utils.PropertyUtil [] - 加载application.properties文件内容完成...........
2022-08-17 17:08:37,144 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.rpc.address, h02.edp.x.com
2022-08-17 17:08:37,144 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.rpc.port, 6123
2022-08-17 17:08:37,144 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.process.size, 1024m
2022-08-17 17:08:37,144 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.process.size, 1024m
2022-08-17 17:08:37,144 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2022-08-17 17:08:37,144 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: parallelism.default, 1
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.execution.failover-strategy, region
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: rest.port, 10081
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: security.kerberos.login.use-ticket-cache, true
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: security.kerberos.login.keytab, /opt/flink-1.12.1/conf/hive.edp.keytab
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: security.kerberos.login.principal, hive/edp@EDP.X.COM
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: env.log.dir, /opt/flink-1.12.1/log
2022-08-17 17:08:37,145 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: web.log.path, /opt/flink-1.12.1/log
2022-08-17 17:08:37,145 DEBUG org.apache.flink.runtime.util.HadoopUtils [] - Searching Hadoop configuration files in HADOOP_HOME: /opt/hadoop-3.3.3
2022-08-17 17:08:37,145 DEBUG org.apache.flink.runtime.util.HadoopUtils [] - Adding /opt/hadoop-3.3.3/etc/hadoop/core-site.xml to hadoop configuration
2022-08-17 17:08:37,145 DEBUG org.apache.flink.runtime.util.HadoopUtils [] - Adding /opt/hadoop-3.3.3/etc/hadoop/hdfs-site.xml to hadoop configuration
2022-08-17 17:08:37,145 DEBUG org.apache.flink.runtime.util.HadoopUtils [] - Searching Hadoop configuration files in HADOOP_CONF_DIR: /opt/hadoop-3.3.3/etc/hadoop
2022-08-17 17:08:37,145 DEBUG org.apache.flink.runtime.util.HadoopUtils [] - Adding /opt/hadoop-3.3.3/etc/hadoop/core-site.xml to hadoop configuration
2022-08-17 17:08:37,146 DEBUG org.apache.flink.runtime.util.HadoopUtils [] - Adding /opt/hadoop-3.3.3/etc/hadoop/hdfs-site.xml to hadoop configuration
2022-08-17 17:08:37,178 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler [] - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not execute application.
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(JarRunHandler.java:107) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836) [?:1.8.0_322]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811) [?:1.8.0_322]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) [?:1.8.0_322]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609) [?:1.8.0_322]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_322]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_322]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_322]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_322]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_322]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_322]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_322]
Caused by: java.util.concurrent.CompletionException: java.lang.VerifyError: Stack map does not match the one at exception handler 70
Exception Details:
Location:
org/apache/iceberg/hive/HiveCatalog.loadNamespaceMetadata(Lorg/apache/iceberg/catalog/Namespace;)Ljava/util/Map; @70: astore_2
Reason:
Type 'org/apache/hadoop/hive/metastore/api/NoSuchObjectException' (current frame, stack[0]) is not assignable to 'org/apache/thrift/TException' (stack map, stack[0])
Current Frame:
bci: @26
flags: { }
locals: { 'org/apache/iceberg/hive/HiveCatalog', 'org/apache/iceberg/catalog/Namespace' }
stack: { 'org/apache/hadoop/hive/metastore/api/NoSuchObjectException' }
Stackmap Frame:
bci: @70
flags: { }
locals: { 'org/apache/iceberg/hive/HiveCatalog', 'org/apache/iceberg/catalog/Namespace' }
stack: { 'org/apache/thrift/TException' }
Bytecode:
0x0000000: 2a2b b700 1e9a 0015 bb00 3159 1232 04bd
0x0000010: 0033 5903 2b53 b700 34bf 2ab4 0012 2bba
0x0000020: 008b 0000 b900 2302 00c0 008c 4d2a 2cb7
0x0000030: 008d 4eb2 002d 128e 2b2d b900 8301 00b9
0x0000040: 002f 0400 2db0 4dbb 0031 592c 1232 04bd
0x0000050: 0033 5903 2b53 b700 88bf 4dbb 0036 59bb
0x0000060: 0037 59b7 0038 1289 b600 3a2b b600 3b12
0x0000070: 6db6 003a b600 3c2c b700 3dbf 4db8 003f
0x0000080: b600 40bb 0036 59bb 0037 59b7 0038 128a
0x0000090: b600 3a2b b600 3b12 6db6 003a b600 3c2c
0x00000a0: b700 3dbf
Exception Handler Table:
bci [26, 69] => handler: 70
bci [26, 69] => handler: 70
bci [26, 69] => handler: 90
bci [26, 69] => handler: 124
Stackmap Table:
same_frame(@26)
same_locals_1_stack_item_frame(@70,Object[#53])
same_locals_1_stack_item_frame(@90,Object[#53])
same_locals_1_stack_item_frame(@124,Object[#62])

at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) ~[?:1.8.0_322]
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) ~[?:1.8.0_322]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606) ~[?:1.8.0_322]
... 7 more

Caused by: java.lang.VerifyError: Stack map does not match the one at exception handler 70
Exception Details:
Location:
org/apache/iceberg/hive/HiveCatalog.loadNamespaceMetadata(Lorg/apache/iceberg/catalog/Namespace;)Ljava/util/Map; @70: astore_2
Reason:
Type 'org/apache/hadoop/hive/metastore/api/NoSuchObjectException' (current frame, stack[0]) is not assignable to 'org/apache/thrift/TException' (stack map, stack[0])
Current Frame:
bci: @26
flags: { }
locals: { 'org/apache/iceberg/hive/HiveCatalog', 'org/apache/iceberg/catalog/Namespace' }
stack: { 'org/apache/hadoop/hive/metastore/api/NoSuchObjectException' }
Stackmap Frame:
bci: @70
flags: { }
locals: { 'org/apache/iceberg/hive/HiveCatalog', 'org/apache/iceberg/catalog/Namespace' }
stack: { 'org/apache/thrift/TException' }
Bytecode:
0x0000000: 2a2b b700 1e9a 0015 bb00 3159 1232 04bd
0x0000010: 0033 5903 2b53 b700 34bf 2ab4 0012 2bba
0x0000020: 008b 0000 b900 2302 00c0 008c 4d2a 2cb7
0x0000030: 008d 4eb2 002d 128e 2b2d b900 8301 00b9
0x0000040: 002f 0400 2db0 4dbb 0031 592c 1232 04bd
0x0000050: 0033 5903 2b53 b700 88bf 4dbb 0036 59bb
0x0000060: 0037 59b7 0038 1289 b600 3a2b b600 3b12
0x0000070: 6db6 003a b600 3c2c b700 3dbf 4db8 003f
0x0000080: b600 40bb 0036 59bb 0037 59b7 0038 128a
0x0000090: b600 3a2b b600 3b12 6db6 003a b600 3c2c
0x00000a0: b700 3dbf
Exception Handler Table:
bci [26, 69] => handler: 70
bci [26, 69] => handler: 70
bci [26, 69] => handler: 90
bci [26, 69] => handler: 124
Stackmap Table:
same_frame(@26)
same_locals_1_stack_item_frame(@70,Object[#53])
same_locals_1_stack_item_frame(@90,Object[#53])
same_locals_1_stack_item_frame(@124,Object[#62])

at java.lang.Class.forName0(Native Method) ~[?:1.8.0_322]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_322]
at org.apache.iceberg.common.DynConstructors$Builder.impl(DynConstructors.java:155) ~[iceberg-flink-runtime-1.12-0.13.2.jar:?]
at org.apache.iceberg.CatalogUtil.loadCatalog(CatalogUtil.java:174) ~[iceberg-flink-runtime-1.12-0.13.2.jar:?]
at org.apache.iceberg.flink.CatalogLoader$HiveCatalogLoader.loadCatalog(CatalogLoader.java:112) ~[iceberg-flink-runtime-1.12-0.13.2.jar:?]
at org.apache.iceberg.flink.FlinkCatalog.(FlinkCatalog.java:110) ~[iceberg-flink-runtime-1.12-0.13.2.jar:?]
at org.apache.iceberg.flink.FlinkCatalogFactory.createCatalog(FlinkCatalogFactory.java:131) ~[iceberg-flink-runtime-1.12-0.13.2.jar:?]
at org.apache.iceberg.flink.FlinkCatalogFactory.createCatalog(FlinkCatalogFactory.java:118) ~[iceberg-flink-runtime-1.12-0.13.2.jar:?]
at org.apache.flink.table.api.internal.TableEnvironmentImpl.createCatalog(TableEnvironmentImpl.java:1121) ~[flink-table_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeOperation(TableEnvironmentImpl.java:1019) ~[flink-table_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:666) ~[flink-table_2.12-1.12.1.jar:1.12.1]
at cloud.tiemao.bigdata.pg2iceberg.PGCDC2IcebergCategoryTree.main(PGCDC2IcebergCategoryTree.java:81) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_322]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_322]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_322]
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:343) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:213) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:102) ~[flink-dist_2.12-1.12.1.jar:1.12.1]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) ~[?:1.8.0_322]
... 7 more

2022-08-17 17:08:40,469 DEBUG org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Release TaskExecutor c07b9242537c775f2d62f4f7bd0cff6d because it exceeded the idle timeout.
2022-08-17 17:08:40,469 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Worker 10.10.10.12:36759-e7b24c could not be stopped.
2022-08-17 17:08:41,066 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Trigger heartbeat request.
2022-08-17 17:08:41,066 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Trigger heartbeat request.
2022-08-17 17:08:41,070 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Received heartbeat from 10.10.10.12:36759-e7b24c.
2022-08-17 17:08:41,070 DEBUG org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Received slot report from instance c07b9242537c775f2d62f4f7bd0cff6d: SlotReport{SlotStatus{slotID=10.10.10.12:36759-e7b24c_0, allocationID=null, jobID=null, resourceProfile=ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=25.600mb (26843542 bytes), taskOffHeapMemory=0 bytes, managedMemory=230.400mb (241591914 bytes), networkMemory=64.000mb (67108864 bytes)}}}.
2022-08-17 17:08:41,071 DEBUG org.apache.flink.runtime.io.network.partition.ResourceManagerPartitionTrackerImpl [] - Processing cluster partition report from task executor 10.10.10.12:36759-e7b24c: PartitionReport{entries=[]}.
2022-08-17 17:08:51,087 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Trigger heartbeat request.
2022-08-17 17:08:51,087 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Trigger heartbeat request.
2022-08-17 17:08:51,091 DEBUG org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Received heartbeat from 10.10.10.12:36759-e7b24c.
2022-08-17 17:08:51,091 DEBUG org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Received slot report from instance c07b9242537c775f2d62f4f7bd0cff6d: SlotReport{SlotStatus{slotID=10.10.10.12:36759-e7b24c_0, allocationID=null, jobID=null, resourceProfile=ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=25.600mb (26843542 bytes), taskOffHeapMemory=0 bytes, managedMemory=230.400mb (241591914 bytes), networkMemory=64.000mb (67108864 bytes)}}}.
2022-08-17 17:08:51,091 DEBUG org.apache.flink.runtime.io.network.partition.ResourceManagerPartitionTrackerImpl [] - Processing cluster partition report from task executor 10.10.10.12:36759-e7b24c: PartitionReport{entries=[]}.
2022-08-17 17:09:00,176 DEBUG org.apache.flink.runtime.rest.handler.legacy.files.StaticFileServerHandler [] - Responding 'NOT MODIFIED' for file '/tmp/flink-web-8d9c25af-f10e-48d9-a642-97d52e59f1b2/flink-web-ui/assets/images/process_mem_model.svg'
2022-08-17 17:09:00,449 DEBUG org.apache.flink.runtime.rest.handler.legacy.metrics.MetricFetcherImpl [] - Start fetching metrics.
2022-08-17 17:09:00,453 DEBUG org.apache.flink.runtime.rest.handler.legacy.metrics.MetricFetcherImpl [] - Retrieve metric query service gateway for akka.tcp://flink-metrics@h02.edp.x.com:33753/user/rpc/MetricQueryService
2022-08-17 17:09:00,453 DEBUG org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Try to connect to remote RPC endpoint with address akka.tcp://flink-metrics@h02.edp.x.com:33753/user/rpc/MetricQueryService. Returning a org.apache.flink.runtime.webmonitor.retriever.MetricQueryServiceGateway gateway.
2022-08-17 17:09:00,455 DEBUG org.apache.flink.runtime.rest.handler.legacy.metrics.MetricFetcherImpl [] - Query metrics for akka.tcp://flink-metrics@h02.edp.x.com:33753/user/rpc/MetricQueryService.
2022-08-17 17:09:00,460 DEBUG org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Try to connect to remote RPC endpoint with address akka.tcp://flink-metrics@10.10.10.12:36579/user/rpc/MetricQueryService_10.10.10.12:36759-e7b24c. Returning a org.apache.flink.runtime.webmonitor.retriever.MetricQueryServiceGateway gateway.
2022-08-17 17:09:00,471 DEBUG org.apache.flink.runtime.rest.handler.legacy.metrics.MetricFetcherImpl [] - Query metrics for akka.tcp://flink-metrics@10.10.10.12:36579/user/rpc/MetricQueryService_10.10.10.12:36759-e7b24c.

看看flink控制台和您引入的flink版本是否匹配，有些版本不兼容的