目标使用 sqoop 导入mysql 表到 hive,当前这个脚本有些表是没问题的,这个表就有问题,分隔符失效了
mysql 表结构如下:
添加 --hive-drop-import-delims 后:
导出文件没有任何变化:
恳请大家指点一下
我试了一下,没有问题,导出有分隔符。
生成数据
DROP TABLE IF EXISTS ct_industry;
CREATE TABLE ct_industry(
ID bigint(20) NOT NULL COMMENT 'ID',
IndustryNum int(11) NOT NULL COMMENT '行业编码',
IndustryName varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT '行业名称',
IndustryCode varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT '所属行业代码',
IndustryPlate varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '行业板块',
XGRQ datetime(0) NOT NULL COMMENT '修改日期',
JSID bigint(20) NOT NULL COMMENT 'JSID',
UNIQUE INDEXIX_CT_Industry(IndustryNum) USING BTREE,
UNIQUE INDEXIX_CT_Industry_JSID(JSID) USING BTREE,
UNIQUE INDEXPK_CT_Industry(ID) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
INSERT INTO ct_industry VALUES (4747621348152, 1, '农、林、牧、渔业', 'A', 'A', '2006-06-14 11:43:44', 203600622544);
INSERT INTO ct_industry VALUES (4747623710988, 2, '农业', 'A01', 'A01', '2006-06-14 11:43:44', 203600622545);
INSERT INTO ct_industry VALUES (4747625764139, 3, '种植业', 'A0101', 'A01', '2006-06-14 11:43:44', 203600622546);
INSERT INTO ct_industry VALUES (4747631517547, 4, '其他农业', 'A0199', 'A01', '2006-06-14 11:43:44', 203600622547);
INSERT INTO ct_industry VALUES (4747636155935, 5, '林业', 'A03', 'A03', '2006-06-14 11:43:44', 203600622548);
INSERT INTO ct_industry VALUES (4747638884790, 6, '畜牧业', 'A05', 'A05', '2006-06-14 11:43:44', 203600622549);
INSERT INTO ct_industry VALUES (4747641912105, 7, '牲畜饲养放牧业', 'A0501', 'A05', '2006-06-14 11:43:44', 203600622550);
INSERT INTO ct_industry VALUES (4747644131080, 8, '家禽饲养业', 'A0505', 'A05', '2006-06-14 11:43:44', 203600622551);
INSERT INTO ct_industry VALUES (4747646482277, 9, '其他畜牧业', 'A0599', 'A05', '2006-06-14 11:43:44', 203600622552);
INSERT INTO ct_industry VALUES (4747653097931, 10, '渔业', 'A07', 'A07', '2006-06-14 11:43:44', 203600622553);
INSERT INTO ct_industry VALUES (4747654632813, 11, '海洋渔业', 'A0701', 'A07', '2006-06-14 11:43:44', 203600622554);
INSERT INTO ct_industry VALUES (4747656185728, 12, '淡水渔业', 'A0705', 'A07', '2006-06-14 11:43:44', 203600622555);
导出命令
sqoop import \
--connect jdbc:mysql://{mysql}:3306/test \
--username root \
--password root \
--table ct_industry \
-m 1 \
--hive-import \
--create-hive-table \
--fields-terminated-by ',' \
--hive-table test.ct_industry
导出结果
4747621348152,1,农、林、牧、渔业,A,A,2006-06-14 11:43:44.0,203600622544
4747623710988,2,农业,A01,A01,2006-06-14 11:43:44.0,203600622545
4747625764139,3,种植业,A0101,A01,2006-06-14 11:43:44.0,203600622546
4747631517547,4,其他农业,A0199,A01,2006-06-14 11:43:44.0,203600622547
4747636155935,5,林业,A03,A03,2006-06-14 11:43:44.0,203600622548
4747638884790,6,畜牧业,A05,A05,2006-06-14 11:43:44.0,203600622549
4747641912105,7,牲畜饲养放牧业,A0501,A05,2006-06-14 11:43:44.0,203600622550
4747644131080,8,家禽饲养业,A0505,A05,2006-06-14 11:43:44.0,203600622551
4747646482277,9,其他畜牧业,A0599,A05,2006-06-14 11:43:44.0,203600622552
4747653097931,10,渔业,A07,A07,2006-06-14 11:43:44.0,203600622553
4747654632813,11,海洋渔业,A0701,A07,2006-06-14 11:43:44.0,203600622554
4747656185728,12,淡水渔业,A0705,A07,2006-06-14 11:43:44.0,203600622555
试一下改成
\,
应该没什么问题啊,你是不是看错文件啥的了,我试了一下可以的啊
应该是分隔符和数据冲突了,所以sqoop直接把所有的分隔符都去掉了
有可能是内容中存在分隔符
看了 问题 看了 回答 感觉 我也解决不了 哈哈哈哈
有些软件会自动删去这些分隔符的,mysql有专门的复制操作
等着大佬解决,我上次看了,想了好久没想明白,强迫症
我试了没问题啊
试了,可以呀,这问题问错了吧
那是不是,分隔符有特殊格式要求呀