[kettle] StarRocks kettle connector 测试问题

为了更快的定位您的问题,请提供以下信息,谢谢
【详述】 查看官方文档,目前有StarRocks kettle connector,咨询下目前这个是否稳定,是否生产可用?目前 JDBC作为源,starRocks作为目标库,效率和性能如何?
starRocks 作为目标库 ,支持哪些starRocks的版本?
kettle 支持哪些版本的kettle?
image


【背景】RDBMS 数据库的数据抽入到starRocks 库表
【业务影响】
【StarRocks版本】 3.0.9
【集群规模】 3fe(1 follower+2observer)+5be(fe与be混部)
【机器信息】 48C/64G/万兆
【联系方式】为了在解决问题过程中能及时联系到您获取一些日志信息,请补充下您的联系方式,例如:社区群4-小李或者邮箱,谢谢
【附件】
  • fe.log/beINFO/相应截图
  • 慢查询:
    • Profile信息
    • 并行度: show variables like ‘%parallel_fragment_exec_instance_num%’;
      Variable_name |Value|
      -----------------------------------±----+
      parallel_fragment_exec_instance_num|1 |
    • pipeline是否开启:show variables like ‘%pipeline%’;
      Variable_name |Value|
      -------------------------------±----+
      enable_pipeline_engine |true |
      enable_pipeline_query_statistic|true |
      max_pipeline_dop |64 |
      pipeline_dop |4 |
      pipeline_profile_level |1 |

      – 建议:
      starRocks connector 这里需要加上一个连接测试的按钮,填写一堆东西没法确认是否能连接到starRocks 数据库服务器。
      – 报错日志:
      2024/03/27 13:59:08 - ods_dc_sys__sstk_unset_store_rt_dc_f - Dispatching started for transformation [ods_dc_sys__sstk_unset_store_rt_dc_f]
      2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Failed to connect to Starrocks Manager.
      2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : java.lang.IllegalArgumentException: Failed to get StarRocks version. Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.steps.starrockskettleconnector.starrocks.StarRocksQueryVisitor.getStarRocksVersion(StarRocksQueryVisitor.java:83)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.steps.starrockskettleconnector.StarRocksKettleConnector.getProperties(StarRocksKettleConnector.java:391)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.steps.starrockskettleconnector.StarRocksKettleConnector.init(StarRocksKettleConnector.java:264)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at java.lang.Thread.run(Thread.java:748)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:828)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at java.sql.DriverManager.getConnection(DriverManager.java:664)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at java.sql.DriverManager.getConnection(DriverManager.java:247)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.steps.starrockskettleconnector.starrocks.StarRocksJdbcConnectionProvider.getConnection(StarRocksJdbcConnectionProvider.java:36)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.steps.starrockskettleconnector.starrocks.StarRocksQueryVisitor.executeQuery(StarRocksQueryVisitor.java:88)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at org.pentaho.di.trans.steps.starrockskettleconnector.starrocks.StarRocksQueryVisitor.getStarRocksVersion(StarRocksQueryVisitor.java:73)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - … 4 more
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:317)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:203)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1369)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.NativeSession.connect(NativeSession.java:133)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:948)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - … 12 more
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - Caused by: javax.net.ssl.SSLException: Received fatal alert: protocol_version
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2033)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1135)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:335)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:194)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:101)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:308)
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - … 17 more
2024/03/27 13:59:09 - StarRocks Kettle Connector.0 - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Error initializing step [StarRocks Kettle Connector]
2024/03/27 13:59:09 - ods_dc_sys__sstk_unset_store_rt_dc_f - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Step [StarRocks Kettle Connector.0] failed to initialize!
2024/03/27 13:59:09 - 7484.0 - Finished reading query, closing connection
2024/03/27 13:59:09 - Spoon - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : ods_dc_sys__sstk_unset_store_rt_dc_f: preparing transformation execution failed
2024/03/27 13:59:09 - Spoon - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : org.pentaho.di.core.exception.KettleException:
2024/03/27 13:59:09 - Spoon - We failed to initialize at least one step. Execution can not begin!
2024/03/27 13:59:09 - Spoon -
2024/03/27 13:59:09 - Spoon -
2024/03/27 13:59:09 - Spoon - at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1301)
2024/03/27 13:59:09 - Spoon - at org.pentaho.di.ui.spoon.trans.TransGraph$30.run(TransGraph.java:4282)
2024/03/27 13:59:09 - Spoon - at java.lang.Thread.run(Thread.java:748)
2024/03/27 13:59:09 - ods_dc_sys__sstk_unset_store_rt_dc_f - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Errors detected!
2024/03/27 13:59:09 - ods_dc_sys__sstk_unset_store_rt_dc_f - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Errors detected!