StarRocks分级存储(SSD + SATA)功能测试失败

💬 StarRocks 用户问答
#1

【详述】使用sr的分级存储功能测试
【业务影响】
【StarRocks版本】例如：2.1.4
【集群规模】3fe（1 follower+2observer）+3be（fe与be混部）
【机器信息】16C/32G/万兆，每个be上挂两块盘，一块ssd挂在/mnt目录下，一块hdd挂在/mnt1目录下
操作：
新部署sr，be配置中增加分级存储配置如下：
storage_root_path = /mnt1/starrocks/be/storage,medium:HDD,capacity:90;/mnt/starrocks/be/storage,medium:SSD,capacity:890

通过客户端连上starrocks，在库中创建两张表，一张表创建时，指定存储在ssd盘中

CREATE TABLE aaa_ssd (
dt_time datetime NOT NULL COMMENT “分区时间”,
id bigint(20) NOT NULL COMMENT “主键id”,
…
dt int(11) NOT NULL COMMENT “分区年”,
extract_time datetime NOT NULL COMMENT “aaaaaaa”
) ENGINE=OLAP
PRIMARY KEY(dt_time, id)
COMMENT “aaa”
PARTITION BY RANGE(dt_time)
(PARTITION p2016 VALUES [(‘2016-01-01 00:00:00’), (‘2017-01-01 00:00:00’)),
PARTITION p2017 VALUES [(‘2017-01-01 00:00:00’), (‘2018-01-01 00:00:00’)),
PARTITION p2018 VALUES [(‘2018-01-01 00:00:00’), (‘2019-01-01 00:00:00’)),
PARTITION p2019 VALUES [(‘2019-01-01 00:00:00’), (‘2020-01-01 00:00:00’)),
PARTITION p2020 VALUES [(‘2020-01-01 00:00:00’), (‘2021-01-01 00:00:00’)),
PARTITION p2021 VALUES [(‘2021-01-01 00:00:00’), (‘2022-01-01 00:00:00’)),
PARTITION p202201 VALUES [(‘2022-01-01 00:00:00’), (‘2022-02-01 00:00:00’)),
PARTITION p202202 VALUES [(‘2022-02-01 00:00:00’), (‘2022-03-01 00:00:00’)),
PARTITION p202203 VALUES [(‘2022-03-01 00:00:00’), (‘2022-04-01 00:00:00’)),
PARTITION p202204 VALUES [(‘2022-04-01 00:00:00’), (‘2022-05-01 00:00:00’)),
PARTITION p20220501 VALUES [(‘2022-05-01 00:00:00’), (‘2022-05-02 00:00:00’)),
PARTITION p20220502 VALUES [(‘2022-05-02 00:00:00’), (‘2022-05-03 00:00:00’)),
PARTITION p20220503 VALUES [(‘2022-05-03 00:00:00’), (‘2022-05-04 00:00:00’)),
PARTITION p20220504 VALUES [(‘2022-05-04 00:00:00’), (‘2022-05-05 00:00:00’)),
PARTITION p20220505 VALUES [(‘2022-05-05 00:00:00’), (‘2022-05-06 00:00:00’)),
PARTITION p20220506 VALUES [(‘2022-05-06 00:00:00’), (‘2022-05-07 00:00:00’)),
PARTITION p20220507 VALUES [(‘2022-05-07 00:00:00’), (‘2022-05-08 00:00:00’)),
PARTITION p20220508 VALUES [(‘2022-05-08 00:00:00’), (‘2022-05-09 00:00:00’)),
PARTITION p20220509 VALUES [(‘2022-05-09 00:00:00’), (‘2022-05-10 00:00:00’)),
PARTITION p20220510 VALUES [(‘2022-05-10 00:00:00’), (‘2022-05-11 00:00:00’)),
PARTITION p20220511 VALUES [(‘2022-05-11 00:00:00’), (‘2022-05-12 00:00:00’)),
PARTITION p20220512 VALUES [(‘2022-05-12 00:00:00’), (‘2022-05-13 00:00:00’)),
PARTITION p20220513 VALUES [(‘2022-05-13 00:00:00’), (‘2022-05-14 00:00:00’)),
PARTITION p20220514 VALUES [(‘2022-05-14 00:00:00’), (‘2022-05-15 00:00:00’)),
PARTITION p20220515 VALUES [(‘2022-05-15 00:00:00’), (‘2022-05-16 00:00:00’)),
PARTITION p20220516 VALUES [(‘2022-05-16 00:00:00’), (‘2022-05-17 00:00:00’)),
PARTITION p20220517 VALUES [(‘2022-05-17 00:00:00’), (‘2022-05-18 00:00:00’)),
PARTITION p20220518 VALUES [(‘2022-05-18 00:00:00’), (‘2022-05-19 00:00:00’)),
PARTITION p20220519 VALUES [(‘2022-05-19 00:00:00’), (‘2022-05-20 00:00:00’)),
PARTITION p20220520 VALUES [(‘2022-05-20 00:00:00’), (‘2022-05-21 00:00:00’)),
PARTITION p20220521 VALUES [(‘2022-05-21 00:00:00’), (‘2022-05-22 00:00:00’)),
PARTITION p20220522 VALUES [(‘2022-05-22 00:00:00’), (‘2022-05-23 00:00:00’)),
PARTITION p20220523 VALUES [(‘2022-05-23 00:00:00’), (‘2022-05-24 00:00:00’)),
PARTITION p20220524 VALUES [(‘2022-05-24 00:00:00’), (‘2022-05-25 00:00:00’)),
PARTITION p20220525 VALUES [(‘2022-05-25 00:00:00’), (‘2022-05-26 00:00:00’)),
PARTITION p20220526 VALUES [(‘2022-05-26 00:00:00’), (‘2022-05-27 00:00:00’)),
PARTITION p20220527 VALUES [(‘2022-05-27 00:00:00’), (‘2022-05-28 00:00:00’)),
PARTITION p20220528 VALUES [(‘2022-05-28 00:00:00’), (‘2022-05-29 00:00:00’)),
PARTITION p20220529 VALUES [(‘2022-05-29 00:00:00’), (‘2022-05-30 00:00:00’)),
PARTITION p20220530 VALUES [(‘2022-05-30 00:00:00’), (‘2022-05-31 00:00:00’)),
PARTITION p20220531 VALUES [(‘2022-05-31 00:00:00’), (‘2022-06-01 00:00:00’)),
PARTITION p202206 VALUES [(‘2022-06-01 00:00:00’), (‘2022-07-01 00:00:00’)))
DISTRIBUTED BY HASH(id) BUCKETS 16
PROPERTIES (
“replication_num” = “3”,
“dynamic_partition.enable” = “true”,
“dynamic_partition.time_unit” = “MONTH”,
“dynamic_partition.time_zone” = “Asia/Shanghai”,
“dynamic_partition.start” = “-2147483648”,
“dynamic_partition.end” = “1”,
“dynamic_partition.prefix” = “p”,
“dynamic_partition.buckets” = “16”,
“dynamic_partition.replication_num” = “3”,
“dynamic_partition.start_day_of_month” = “1”,
“in_memory” = “false”,
“storage_format” = “DEFAULT”,
“storage_medium” = “SSD”
);

第二张表，指定存储在hdd中

CREATE TABLE aaa_hdd (
dt_time datetime NOT NULL COMMENT “分区时间”,
id bigint(20) NOT NULL COMMENT “主键id”,
…
dt int(11) NOT NULL COMMENT “分区年”,
extract_time datetime NOT NULL COMMENT “aaa”
) ENGINE=OLAP
PRIMARY KEY(dt_time, id)
COMMENT “aaa”
PARTITION BY RANGE(dt_time)
(PARTITION p2016 VALUES [(‘2016-01-01 00:00:00’), (‘2017-01-01 00:00:00’)),
PARTITION p2017 VALUES [(‘2017-01-01 00:00:00’), (‘2018-01-01 00:00:00’)),
PARTITION p2018 VALUES [(‘2018-01-01 00:00:00’), (‘2019-01-01 00:00:00’)),
PARTITION p2019 VALUES [(‘2019-01-01 00:00:00’), (‘2020-01-01 00:00:00’)),
PARTITION p2020 VALUES [(‘2020-01-01 00:00:00’), (‘2021-01-01 00:00:00’)),
PARTITION p2021 VALUES [(‘2021-01-01 00:00:00’), (‘2022-01-01 00:00:00’)),
PARTITION p202201 VALUES [(‘2022-01-01 00:00:00’), (‘2022-02-01 00:00:00’)),
PARTITION p202202 VALUES [(‘2022-02-01 00:00:00’), (‘2022-03-01 00:00:00’)),
PARTITION p202203 VALUES [(‘2022-03-01 00:00:00’), (‘2022-04-01 00:00:00’)),
PARTITION p202204 VALUES [(‘2022-04-01 00:00:00’), (‘2022-05-01 00:00:00’)),
PARTITION p20220501 VALUES [(‘2022-05-01 00:00:00’), (‘2022-05-02 00:00:00’)),
PARTITION p20220502 VALUES [(‘2022-05-02 00:00:00’), (‘2022-05-03 00:00:00’)),
PARTITION p20220503 VALUES [(‘2022-05-03 00:00:00’), (‘2022-05-04 00:00:00’)),
PARTITION p20220504 VALUES [(‘2022-05-04 00:00:00’), (‘2022-05-05 00:00:00’)),
PARTITION p20220505 VALUES [(‘2022-05-05 00:00:00’), (‘2022-05-06 00:00:00’)),
PARTITION p20220506 VALUES [(‘2022-05-06 00:00:00’), (‘2022-05-07 00:00:00’)),
PARTITION p20220507 VALUES [(‘2022-05-07 00:00:00’), (‘2022-05-08 00:00:00’)),
PARTITION p20220508 VALUES [(‘2022-05-08 00:00:00’), (‘2022-05-09 00:00:00’)),
PARTITION p20220509 VALUES [(‘2022-05-09 00:00:00’), (‘2022-05-10 00:00:00’)),
PARTITION p20220510 VALUES [(‘2022-05-10 00:00:00’), (‘2022-05-11 00:00:00’)),
PARTITION p20220511 VALUES [(‘2022-05-11 00:00:00’), (‘2022-05-12 00:00:00’)),
PARTITION p20220512 VALUES [(‘2022-05-12 00:00:00’), (‘2022-05-13 00:00:00’)),
PARTITION p20220513 VALUES [(‘2022-05-13 00:00:00’), (‘2022-05-14 00:00:00’)),
PARTITION p20220514 VALUES [(‘2022-05-14 00:00:00’), (‘2022-05-15 00:00:00’)),
PARTITION p20220515 VALUES [(‘2022-05-15 00:00:00’), (‘2022-05-16 00:00:00’)),
PARTITION p20220516 VALUES [(‘2022-05-16 00:00:00’), (‘2022-05-17 00:00:00’)),
PARTITION p20220517 VALUES [(‘2022-05-17 00:00:00’), (‘2022-05-18 00:00:00’)),
PARTITION p20220518 VALUES [(‘2022-05-18 00:00:00’), (‘2022-05-19 00:00:00’)),
PARTITION p20220519 VALUES [(‘2022-05-19 00:00:00’), (‘2022-05-20 00:00:00’)),
PARTITION p20220520 VALUES [(‘2022-05-20 00:00:00’), (‘2022-05-21 00:00:00’)),
PARTITION p20220521 VALUES [(‘2022-05-21 00:00:00’), (‘2022-05-22 00:00:00’)),
PARTITION p20220522 VALUES [(‘2022-05-22 00:00:00’), (‘2022-05-23 00:00:00’)),
PARTITION p20220523 VALUES [(‘2022-05-23 00:00:00’), (‘2022-05-24 00:00:00’)),
PARTITION p20220524 VALUES [(‘2022-05-24 00:00:00’), (‘2022-05-25 00:00:00’)),
PARTITION p20220525 VALUES [(‘2022-05-25 00:00:00’), (‘2022-05-26 00:00:00’)),
PARTITION p20220526 VALUES [(‘2022-05-26 00:00:00’), (‘2022-05-27 00:00:00’)),
PARTITION p20220527 VALUES [(‘2022-05-27 00:00:00’), (‘2022-05-28 00:00:00’)),
PARTITION p20220528 VALUES [(‘2022-05-28 00:00:00’), (‘2022-05-29 00:00:00’)),
PARTITION p20220529 VALUES [(‘2022-05-29 00:00:00’), (‘2022-05-30 00:00:00’)),
PARTITION p20220530 VALUES [(‘2022-05-30 00:00:00’), (‘2022-05-31 00:00:00’)),
PARTITION p20220531 VALUES [(‘2022-05-31 00:00:00’), (‘2022-06-01 00:00:00’)),
PARTITION p202206 VALUES [(‘2022-06-01 00:00:00’), (‘2022-07-01 00:00:00’)))
DISTRIBUTED BY HASH(id) BUCKETS 16
PROPERTIES (
“replication_num” = “3”,
“dynamic_partition.enable” = “true”,
“dynamic_partition.time_unit” = “MONTH”,
“dynamic_partition.time_zone” = “Asia/Shanghai”,
“dynamic_partition.start” = “-2147483648”,
“dynamic_partition.end” = “1”,
“dynamic_partition.prefix” = “p”,
“dynamic_partition.buckets” = “16”,
“dynamic_partition.replication_num” = “3”,
“dynamic_partition.start_day_of_month” = “1”,
“in_memory” = “false”,
“storage_format” = “DEFAULT”,
“storage_medium” = “HDD”
);

然后每张表灌入1G的数据。然后查看磁盘确实存储了对应的数据。

561M /mnt/starrocks/be/storage/
572M /mnt1/starrocks/be/storage/

当我使用alter table命令，将ods_ssd的部分分区改为hdd存储，通过show partitions 命令可以看到这些分区的StorageMedium已经变为HDD，而且CooldownTime也变成了99999999-12-31 23:59:59。
但是磁盘存储大小却没有变化，这些分区没有迁移到HDD磁盘中

如果我想实现将存储在SSD中的表的指定分区，迁移到HDD中，应该如何操作。

#2

您好，辛苦往表内部在插入数据看下。

#3

你好，迁移应该是storage_root_path配置ssd和hdd存储目录，表的分区设置ssd属性，并写配置coodown_time为某个时间点，过了这个时间点这个分区的数据会移动到hdd盘。不是直接把分区属性改为hdd

#4

只能通过设置cooldown_time这种方式来实现ssd到hdd的迁移，不能通过人为的修改分区属性来实现吗？
cooldown_time默认是一个月，如果我想让分区快速迁移到hdd，就直接设置为最近时间就行了吧是吧。

#5

是的，不能人为修改分区属性来实现。要想快速迁移，cooldown_time调为最近的某个时间点。

京ICP备2022026421号-1