外部表查询速度

[U_1653643648463_0745]

【详述】千万级别外部表查询速度 200S
【背景】外部单表查询1400W 查询时间超200S
【业务影响】
【StarRocks版本】社区2.1
【集群规模】例如：node1：fe be broker node2：fe be broker node3：be broker
【机器信息】8C16G
【附件】

• 慢查询：
  • Profile信息
    Query:
    Summary:
    • Query ID: 1432c177-dd9c-11ec-b868-52540081cbca
    • Start Time: 2022-05-27 17:04:56
    • End Time: 2022-05-27 17:05:20
    • Total: 24s97ms
    • Query Type: Query
    • Query State: EOF
    • StarRocks Version: 2.1.6
    • User: root
    • Default Db: default_cluster:network_monitoring
    • Sql Statement: select * from mysql_bas_net_kd_d
      Execution Profile 1432c177-dd9c-11ec-b868-52540081cbca:(Active: 24s96ms[24096783607ns], % non-child: 100.00%)
      Fragment 0:
      Instance 1432c177-dd9c-11ec-b868-52540081cbcb (host=TNetworkAddress(hostname:192.169.128.151, port:9060)):(Active: 24s89ms[24089463248ns], % non-child: 38.91%)
      • AverageThreadTokens: 4607182418800017400.00
      • MemoryLimit: 2.00 GB
      • PeakMemoryUsage: 332.88 KB
      • RowsProduced: 14.590948M (14590948)
        DataBufferSender (dst_fragment_instance_id=1432c177-dd9c-11ec-b868-52540081cbcb):
        • AppendBatchTime: 7s960ms
          • ResultRendTime: 1s398ms
          • TupleConvertTime: 7s926ms
        • NumSentRows: 14.590948M (14590948)
          MYSQL_SCAN_NODE (id=0):(Active: 14s713ms[14713971283ns], % non-child: 61.06%)
        • BytesRead: 0.00
        • NumDiskAccess: 0
        • PeakMemoryUsage: 0.00
        • RowsRead: 0
        • RowsReturned: 14.590948M (14590948)
        • RowsReturnedRate: 991.639K /sec
        • ScannerThreadsInvoluntaryContextSwitches: 0
        • ScannerThreadsTotalWallClockTime: 0ns
          • MaterializeTupleTime(*): 14s702ms
          • ScannerThreadsSysTime: 0ns
          • ScannerThreadsUserTime: 0ns
        • ScannerThreadsVoluntaryContextSwitches: 0
        • TotalRawReadTime(*): 0ns
        • TotalReadThroughput: 0.0 /sec

[shemplle]

可以辛苦把profile以文件形式发送下么？

[shemplle]

新建 文本文档.txt (1.8 KB)
您好，请问下您的mysql跟sr是跨网络的么？
咱们的sr fe的配置如何呢？
建议先做下部分列的查询看下情况如何。

[U_1653643648463_0745]

FE使用官方文档默认配置 ```

元数据目录

meta_dir = ${STARROCKS_HOME}/meta

JVM配置

JAVA_OPTS = “-Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$STARROCKS_HOME/log/fe.gc.log”

MySQL 和SR是在同一个局域网里面。

[U_1653898868352_8163]

@U_1653643648463_0745 不是只用了24s吗？

[U_1653898868352_8163]

@U_1653643648463_0745你们是千兆网卡吗