Apache IoTDB 2.0.6 三节点单副本集群部署配置
Apache IoTDB 2.0.6 三节点单副本集群部署配置
集群名称
my-iotdb
集群节点
iotdb-1: 同步 IP 172.16.20.32, 业务 IP 172.16.30.32
iotdb-2: 同步 IP 172.16.20.33, 业务 IP 172.16.30.33
iotdb-3: 同步 IP 172.16.20.34, 业务 IP 172.16.30.34
存储策略
副本数为 1(schema_replication_factor=1, data_replication_factor=1)。数据通过分片机制分布在各节点,灾备由底层基础架构(PVE虚拟机快照/备份)兜底。
目录规划
统一挂载并存储于 /data 目录下。
环境准备
在启动前,必须确保目标目录已存在并具备读写权限(三个服务器都要有这些目录)。
# 创建数据与日志目录
mkdir -p /data/iotdb/confignode/system
mkdir -p /data/iotdb/confignode/consensus
mkdir -p /data/iotdb/datanode/system
mkdir -p /data/iotdb/datanode/data
mkdir -p /data/iotdb/datanode/consensus
mkdir -p /data/iotdb/datanode/wal
核心配置
- iotdb-1 节点配置
####################
### Cluster Configuration
####################
# 集群名称
cluster_name=my-iotdb
####################
### Seed ConfigNode
####################
# ⚠️ 种子节点统一指向 S2 的内部高速网 IP (VLAN 20)
cn_seed_config_node=172.16.20.32:10710
dn_seed_config_node=172.16.20.32:10710
####################
### Replication configuration
####################
# ⚠️ 落实存储策略:数据仅存 1 份,交由集群分片,PVE 负责灾备
schema_replication_factor=1
data_replication_factor=1
####################
### Directory Configuration
####################
cn_system_dir=/data/iotdb/confignode/system
cn_consensus_dir=/data/iotdb/confignode/consensus
dn_system_dir=/data/iotdb/datanode/system
dn_data_dirs=/data/iotdb/datanode/data
dn_consensus_dir=/data/iotdb/datanode/consensus
dn_wal_dirs=/data/iotdb/datanode/wal
####################
### Node RPC Configuration (S2)
####################
# 绑在 VLAN 20 (同步网)
cn_internal_address=172.16.20.32
dn_internal_address=172.16.20.32
# 绑在 VLAN 30 (业务网,供外部直连)
dn_rpc_address=172.16.30.32
# 端口保持默认即可
cn_internal_port=10710
cn_consensus_port=10720
dn_rpc_port=6667
dn_internal_port=10730
dn_mpp_data_exchange_port=10740
dn_schema_region_consensus_port=10750
dn_data_region_consensus_port=10760
read_thread_pool_size=24
sub_raw_query_thread_pool_size=24
compaction_thread_count=16
storage_query_schema_free_memory_proportion=3:3:2:2
enable_parameter_adapter=true
wal_buffer_size_in_byte=16777216
dn_max_connection_num=2000
schema_region_group_extension_policy=AUTO
schema_region_series_slot_num=30000
series_slot_threshold=0.7
schema_region_per_data_node=500
- iotdb-2 节点配置
####################
### Cluster Configuration
####################
# 集群名称
cluster_name=my-iotdb
####################
### Seed ConfigNode
####################
# ⚠️ 种子节点统一指向 S2 的内部高速网 IP (VLAN 20)
cn_seed_config_node=172.16.20.32:10710
dn_seed_config_node=172.16.20.32:10710
####################
### Replication configuration
####################
# ⚠️ 落实存储策略:数据仅存 1 份,交由集群分片,PVE 负责灾备
schema_replication_factor=1
data_replication_factor=1
####################
### Directory Configuration (咱们之前建的 /data 目录)
####################
cn_system_dir=/data/iotdb/confignode/system
cn_consensus_dir=/data/iotdb/confignode/consensus
dn_system_dir=/data/iotdb/datanode/system
dn_data_dirs=/data/iotdb/datanode/data
dn_consensus_dir=/data/iotdb/datanode/consensus
dn_wal_dirs=/data/iotdb/datanode/wal
####################
### Node RPC Configuration (S3)
####################
cn_internal_address=172.16.20.33
dn_internal_address=172.16.20.33
dn_rpc_address=172.16.30.33
cn_internal_port=10710
cn_consensus_port=10720
dn_rpc_port=6667
dn_internal_port=10730
dn_mpp_data_exchange_port=10740
dn_schema_region_consensus_port=10750
dn_data_region_consensus_port=10760
read_thread_pool_size=24
sub_raw_query_thread_pool_size=24
compaction_thread_count=16
storage_query_schema_free_memory_proportion=3:3:2:2
enable_parameter_adapter=true
wal_buffer_size_in_byte=16777216
dn_max_connection_num=2000
schema_region_group_extension_policy=AUTO
schema_region_series_slot_num=30000
series_slot_threshold=0.7
schema_region_per_data_node=500
- iotdb-3 节点配置
####################
### Cluster Configuration
####################
# 集群名称
cluster_name=my-iotdb
####################
### Seed ConfigNode
####################
# ⚠️ 种子节点统一指向 S2 的内部高速网 IP (VLAN 20)
cn_seed_config_node=172.16.20.32:10710
dn_seed_config_node=172.16.20.32:10710
####################
### Replication configuration
####################
# ⚠️ 落实存储策略:数据仅存 1 份,交由集群分片,PVE 负责灾备
schema_replication_factor=1
data_replication_factor=1
####################
### Directory Configuration
####################
cn_system_dir=/data/iotdb/confignode/system
cn_consensus_dir=/data/iotdb/confignode/consensus
dn_system_dir=/data/iotdb/datanode/system
dn_data_dirs=/data/iotdb/datanode/data
dn_consensus_dir=/data/iotdb/datanode/consensus
dn_wal_dirs=/data/iotdb/datanode/wal
####################
### Node RPC Configuration (S4)
####################
cn_internal_address=172.16.20.34
dn_internal_address=172.16.20.34
dn_rpc_address=172.16.30.34
cn_internal_port=10710
cn_consensus_port=10720
dn_rpc_port=6667
dn_internal_port=10730
dn_mpp_data_exchange_port=10740
dn_schema_region_consensus_port=10750
dn_data_region_consensus_port=10760
read_thread_pool_size=24
sub_raw_query_thread_pool_size=24
compaction_thread_count=16
storage_query_schema_free_memory_proportion=3:3:2:2
enable_parameter_adapter=true
wal_buffer_size_in_byte=16777216
dn_max_connection_num=2000
schema_region_group_extension_policy=AUTO
schema_region_series_slot_num=30000
series_slot_threshold=0.7
schema_region_per_data_node=500
环境配置
在/etc/profile中添加JAVA_HOME环境变量
nano /etc/profile
# 在文件最后一行下加入
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
export PATH=$PATH:$JAVA_HOME/bin
刷新环境变量
source /etc/profile
服务配置
iotdb安装至/opt/iotdb/中,先创建confignode服务文件。
nano /etc/systemd/system/iotdb-confignode.service
输入内容
[Unit]
Description=Apache IoTDB ConfigNode
After=network.target
[Service]
Type=simple
LimitNOFILE=655350
LimitNPROC=65535
Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64"
ExecStart=/opt/iotdb/sbin/start-confignode.sh
ExecStop=/opt/iotdb/sbin/stop-confignode.sh
Restart=on-failure
RestartSec=10
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target
再创建datanode服务文件。
nano /etc/systemd/system/iotdb-datanode.service
输入内容
[Unit]
Description=Apache IoTDB DataNode
After=network.target iotdb-confignode.service
[Service]
Type=simple
LimitNOFILE=655350
LimitNPROC=65535
Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64"
ExecStart=/opt/iotdb/sbin/start-datanode.sh
ExecStop=/opt/iotdb/sbin/stop-datanode.sh
Restart=on-failure
RestartSec=10
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target
启动服务
启动时记得先启动confignode服务,再启动datanode服务。在三个节点上都要执行。
systemctl daemon-reload
systemctl enable iotdb-confignode.service
systemctl enable iotdb-datanode.service
systemctl start iotdb-confignode.service
systemctl start iotdb-datanode.service
systemctl status iotdb-confignode.service
systemctl status iotdb-datanode.service
显示如下时才算启动成功。
---------------------
Starting IoTDB Cli
---------------------
_____ _________ ______ ______
|_ _| | _ _ ||_ _ `.|_ _ \
| | .--.|_/ | | \_| | | `. \ | |_) |
| | / .'`\ \ | | | | | | | __'.
_| |_| \__. | _| |_ _| |_.' /_| |__) |
|_____|'.__.' |_____| |______.'|_______/ version 2.0.6 (Build: c6ecbde)
Successfully login at 172.16.30.32:6667
IoTDB> show CLUSTER
+------+----------+-------+---------------+------------+-------+---------+
|NodeID| NodeType| Status|InternalAddress|InternalPort|Version|BuildInfo|
+------+----------+-------+---------------+------------+-------+---------+
| 0|ConfigNode|Running| 172.16.20.32| 10710| 2.0.6| c6ecbde|
| 4|ConfigNode|Running| 172.16.20.33| 10710| 2.0.6| c6ecbde|
| 5|ConfigNode|Running| 172.16.20.34| 10710| 2.0.6| c6ecbde|
| 1| DataNode|Running| 172.16.20.32| 10730| 2.0.6| c6ecbde|
| 2| DataNode|Running| 172.16.20.33| 10730| 2.0.6| c6ecbde|
| 3| DataNode|Running| 172.16.20.34| 10730| 2.0.6| c6ecbde|
+------+----------+-------+---------------+------------+-------+---------+
Total line number = 6
It costs 0.023s
