HSQL更新 尽量详细点多谢啦

先全量更新历史数据 在增量更新每日数据 ?
更新以下这个需求{最近30.7天昨天每天异常交易的占比} 数据量200w左右 。
可以用伪表伪字段代替,麻烦把逻辑说下 ...


全量导出
HQL示例:insert overwrite  directory ‘/user/root/export/test’ row format delimited fields terminated by ‘,’ STORED AS textfile select F1,F2,F3 from <sourceHiveTable>;
 
SQOOP脚本:sqoop export --connect jdbc:mysql://localhost:3306/wht --username root --password cloudera --table <targetTable> --fields-terminated-by ','  --columns F1,F2,F3 --export-dir /user/root/export/test
 
增量导出(insert模式)
HQL示例:insert overwrite  directory ‘/user/root/export/test’ row format delimited fields terminated by ‘,’ STORED AS textfile select F1,F2,F3 from <sourceHiveTable> where <condition>;
 
SQOOP脚本:sqoop export --connect jdbc:mysql://localhost:3306/wht --username root --password cloudera --table <targetTable> --fields-terminated-by ‘,’  --columns F1,F2,F3 --update-key F4 --update-mode  allowinsert --export-dir /user/root/export/test
 
 
更新导出(update模式)
HQL示例:insert overwrite  directory ‘/user/root/export/test’ row format delimited fields terminated by ‘,’ STORED AS textfile select F1,F2,F3 from <sourceHiveTable> where <condition>;
 
SQOOP脚本:sqoop export --connect jdbc:mysql://localhost:3306/wht --username root --password cloudera --table <targetTable> --fields-terminated-by ‘,’  --columns F1,F2,F3 --update-key F4 --update-mode  updateonly --export-dir /user/root/export/test