flink-cdc 同步多张mysql分表到StarRocks 集群的一张表里,最终同步成功,中途遇到几次报错,在这里记录一下,以供参考:
1、 报错:
[ERROR] Could not execute SQL statement. Reason:
java.lang.IllegalArgumentException: Couldn’t get the sink table’s column info.
【解决办法】:smt 生成的建表语句建完数据库之后要授权给正在使用的非root 的mysql 账号
2、 执行flink-sql 报错
Unable to deploy mysql debezium connector on GKE - User does not have the ‘LOCK TABLES’ privileges
【解决办法】:需要在建表语句添加:
‘debezium.snapshot.locking.mode’ = ‘none’
3、 执行flink-sql 报错
还是报错:Access denied; you need (at least one of) the SUPER, REPLICATION CLIENT privilege(s) for this operation
【解决办法】:需要找运维修改账号权限:
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON . TO ‘user’ IDENTIFIED BY ‘password’;
FLUSH PRIVILEGES;
4、 刚开始报错找不到be节点
1064 - fail to create tablet: timed out. unfinished replicas(1/1): 49073(X.X.X.X) timeout=10s
时间: 10.027s
【解决办法】:重启be解决
最后终于启动起来,实现实时同步!!!
【版本信息】:
flink-1.13.3
flink-sql-connector-mysql-cdc-1.4.0.jar
flink-connector-starrocks-1.1.13_flink-1.13.jar