doris的单节点安装部署(详细)

节点安装部署(详细)"/>

doris的单节点安装部署(详细)

doris的单节点安装部署
环境准备：
1.jdk8
进入官网:/
在linux服务上创建jdk安装的目录
mkdir /itsoftware/jdk #创建jdk安装路径
cd /itsoftware/jdk #进入jdk安装目录
解压缩包

cd /itsoftware/jdk  #进入jdk安装目录
tar  -zxvf  jdk-8u212-linux-x64.tar.gz #对压缩包解压

配置环境变量

vim /etc/profile   #编辑配置文件

在文件末尾添加

export JAVA_HOME=/itsoftware/jdk/jdk1.8.0_212
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}

重新加载配置文件

source /etc/profile

验证jdk是否安装成功

javac -version

修改limits文件

vim /etc/security/limits.conf

在文件的末尾处添加或者修改

* soft nofile 204800
* hard nofile 204800
* soft nproc 204800
* hard nproc 204800

vim /etc/sysctl.conf

#添加

fs.file-max = 6553560
vm.max_map_count = 2000000

#关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

执行生效

sysctl -p

安装doris
官网网址：
解压 tar.gz 文件

 tar -zxvf apache-doris-2.0.2-bin-x64.tar.gzmv apache-doris-2.0.2-bin-x64 /opt/doris

配置环境：

[root@10-63-0-181 conf]# vim /etc/profile.d/doris.sh
export DORIS_HOME=/opt/doris
export PATH=$PATH:$DORIS_HOME/fe/bin:$DORIS_HOME/be/bin

配置生效：

source /etc/profile.d/doris.sh

FE配置
进入doris下的fe目录，修改其中conf目录下的fe.conf文件

注意：
这个参数我们在安装的时候是必须要配置的，特别是当一台机器拥有多个IP地址的时候，我们要为 FE 指定唯一的IP地址。
这里假设你的节点 IP 是 172.23.16.32，那么我们可以通过掩码的方式配置为 172.23.16.0/24。

[root@10-63-0-181 conf]# pwd
/opt/doris/fe/conf
[root@10-63-0-181 conf]# vim fe.conf

更改配置：

meta_dir = /data/doris-meta
priority_networks = 10.63.0.0/24

启动 FE
在 FE 安装目录下执行下面的命令，来完成 FE 的启动。

./bin/start_fe.sh --daemon

查看 FE 运行状态
你可以通过下面的命令来检查 Doris 是否启动成功

[root@10-63-0-181 fe]# curl :8030/api/bootstrap
{"msg":"success","code":0,"data":{"replayedJournalId":0,"queryPort":0,"rpcPort":0,"version":""},"count":0}[root@10-63-0-181 fe]#
[root@10-63-0-181 fe]#

注意：

1. 这里我们使用 Doris 内置的默认用户 root 进行登录，密码是空
2. 这是一个 Doris 的管理界面，只能拥有管理权限的用户才能登录，普通用户不能登录。

连接 FE
我们下面通过 MySQL 客户端来连接 Doris FE，下载免安装的 MySQL 客户端
解压刚才下载的 MySQL 客户端，在 bin/ 目录下可以找到 mysql 命令行工具。然后执行下面的命令连接 Doris。

[root@10-63-0-181 bin]# ./mysql -uroot -P9030 -h10.63.0.181
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 0
Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show frontends\G;
*************************** 1. row ***************************Name: fe_2b76e132_4efe_4425_8919_f95876206edaHost: 10.63.0.181EditLogPort: 9010HttpPort: 8030QueryPort: 9030RpcPort: 9020Role: FOLLOWERIsMaster: trueClusterId: 1092995366Join: trueAlive: true
ReplayedJournalId: 319LastHeartbeat: 2023-10-23 16:01:08IsHelper: trueErrMsg:Version: doris-2.0.2-rc05-ae923f7CurrentConnected: Yes
1 row in set (0.06 sec)

停止 FE 节点
Doris FE 的停止可以通过下面的命令完成

./bin/stop_fe.sh

配置 BE
进入doris下的be目录，修改其中conf目录下的be.conf文件
[root@10-63-0-181 conf]# pwd

/opt/doris/be/conf

[root@10-63-0-181 conf]# vim be.conf

更改配置：

priority_networks = 10.63.0.0/24
storage_root_path = /data/doris-data

启动 BE
在 BE 安装目录下执行下面的命令，来完成 BE 的启动。
./bin/start_be.sh --daemon
添加 BE 节点到集群
通过MySQL 客户端连接到 FE 之后执行下面的 SQL，将 BE 添加到集群中

ALTER SYSTEM ADD BACKEND "10.60.0.181:9050";
[root@10-63-0-181 bin]# ./mysql -uroot -P 9030 -h 10.63.0.181
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> SHOW BACKENDS\G;
*************************** 1. row ***************************BackendId: 10065Host: 10.63.0.181HeartbeatPort: 9050BePort: 9060HttpPort: 8040BrpcPort: 8060LastStartTime: 2023-10-24 10:40:27LastHeartbeat: 2023-10-25 15:44:35Alive: trueSystemDecommissioned: falseTabletNum: 22DataUsedCapacity: 1.417 KBTrashUsedCapcacity: 0.000AvailCapacity: 199.782 GBTotalCapacity: 199.902 GBUsedPct: 0.06 %MaxDiskUsedPct: 0.06 %RemoteUsedCapacity: 0.000Tag: {"location" : "default"}ErrMsg:Version: doris-2.0.2-rc05-ae923f7Status: {"lastSuccessReportTabletsTime":"2023-10-25 15:44:34","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false}

1. Alive : true表示节点运行正常
   

验证：
创建数据表
创建一个数据库

create database demo;

创建数据表

use demo;CREATE TABLE IF NOT EXISTS demo.example_tbl
(`user_id` LARGEINT NOT NULL COMMENT "用户id",`date` DATE NOT NULL COMMENT "数据灌入日期时间",`city` VARCHAR(20) COMMENT "用户所在城市",`age` SMALLINT COMMENT "用户年龄",`sex` TINYINT COMMENT "用户性别",`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 1
PROPERTIES ("replication_allocation" = "tag.location.default: 1"
);
示例数据
10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
将上面的数据保存在test.csv文件中。1. 导入数据
这里我们通过Stream load 方式将上面保存到文件中的数据导入到我们刚才创建的表里。
[root@10-63-0-181 ~]# vi test.csv
[root@10-63-0-181 ~]# cat test.csv
10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
[root@10-63-0-181 ~]# curl  --location-trusted -u root: -T test.csv -H "column_separator:," :8030/api/demo/example_tbl/_stream_load
{"TxnId": 2,"Label": "3d320737-a9db-4546-a145-5543e56d44a5","Comment": "","TwoPhaseCommit": "false","Status": "Success","Message": "OK","NumberTotalRows": 7,"NumberLoadedRows": 7,"NumberFilteredRows": 0,"NumberUnselectedRows": 0,"LoadBytes": 399,"LoadTimeMs": 284,"BeginTxnTimeMs": 20,"StreamLoadPutTimeMs": 199,"ReadDataTimeMs": 0,"WriteDataTimeMs": 11,"CommitAndPublishTimeMs": 51
}
[root@10-63-0-181 bin]# ./mysql -uroot -P 9030 -h 10.63.0.181
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> use demo;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
mysql> select * from example_tbl;
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date       | city   | age  | sex  | last_visit_date     | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10000   | 2017-10-01 | 北京   |   20 |    0 | 2017-10-01 07:00:00 |   35 |             10 |              2 |
| 10001   | 2017-10-01 | 北京   |   30 |    1 | 2017-10-01 17:05:45 |    2 |             22 |             22 |
| 10002   | 2017-10-02 | 上海   |   20 |    1 | 2017-10-02 12:59:12 |  200 |              5 |              5 |
| 10003   | 2017-10-02 | 广州   |   32 |    0 | 2017-10-02 11:20:00 |   30 |             11 |             11 |
| 10004   | 2017-10-01 | 深圳   |   35 |    0 | 2017-10-01 10:00:15 |  100 |              3 |              3 |
| 10004   | 2017-10-03 | 深圳   |   35 |    0 | 2017-10-03 10:20:22 |   11 |              6 |              6 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
6 rows in set (0.07 sec)mysql> select * from example_tbl where city='上海';
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date       | city   | age  | sex  | last_visit_date     | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10002   | 2017-10-02 | 上海   |   20 |    1 | 2017-10-02 12:59:12 |  200 |              5 |              5 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
1 row in set (0.09 sec)mysql> select city, sum(cost) as total_cost from example_tbl group by city;
+--------+------------+
| city   | total_cost |
+--------+------------+
| 上海   |        200 |
| 深圳   |        111 |
| 北京   |         37 |
| 广州   |         30 |
+--------+------------+
4 rows in set (0.13 sec)mysql>