### 介绍 `购买的是拨号江苏服务器，但是仅提供一个远程端口，不提供其他端口映射，但是我们的爬虫是跑本机，所以必须要通过外网去连代理服务器，所以就考虑使用端口复用技术解决。` ### 准备工作 `端口复用软件`: `sslh` `代理软件`: `v2ray` &emsp; ### 拨号服务器,获取外网IP ```bash adsl-start #拨号，不同提供商的命令不一样,有些提供商对命令进行了封装 ``` &emsp; ### 服务器初始化 ```bash yum install epel-release -y #安装epel-release ``` &emsp; ### 用ssh去连拨号获取的IP `执行这步是测试拨号IP是否有端口限制以及防止后面sslh端口复用失败而不能远程连接的问题。` `如果成功执行下一步,失败排查下原因。` &emsp; ### sslh #### 安装 ```bash yum install sslh -y ``` &emsp; #### 配置 ```bash vim /etc/sslh.cfg ``` ```yaml # This is a basic configuration file that should provide # sensible values for "standard" setup. verbose: false; foreground: true; inetd: false; numeric: false; transparent: false; timeout: 2; user: "sslh"; # Change hostname with your external address name. listen: ( { host: "0.0.0.0"; port: "33890"; } #这里为拨号供应商映射的ssh端口(非22)，所以端口复用需要使用和原理ssh端口号保持一致 ); protocols: ( { name: "ssh"; service: "ssh"; host: "localhost"; port: "22"; fork: true; }, #ssh协议包转发给22端口 { name: "anyprot"; host: "localhost"; port: "27073"; } #其他协议包转发给27073(v2ray端口) ); ``` &emsp; #### 修改ssh监听端口 ```bash vim /etc/ssh/sshd_config ``` ```bash Port 22 #修改原来的33890为22端口 ... ``` &emsp; #### 重启ssh和启动sslh ```bash systemctl restart sshd&&systemctl start sslh #先重启sshd让其监听22，然后再重启sslh监听33890 systemctl enable sslh #配置开机启动 ``` &emsp; #### ssh测试重接供应商提供的远程主机和端口 `如果重连成功，说明sslh端口转发到ssh成功` &emsp; ### V2ray #### 安装 ```bash bash <(curl -L -s https://install.direct/go.sh) ``` #### 配置 ```bash vim /etc/v2ray/config.json ``` ```yaml { "inbounds": [{ "port": 27073, //修改为上面sslh转发到的端口号 "listen": "127.0.0.1", //监听回环地址即可 "protocol": "vmess", "settings": { "clients": [ { "id": "62f8c0f5-69fa-41f8-a7b0-97d43014d478", "level": 1, "alterId": 64 } ] } }], "outbounds": [{ "protocol": "freedom", "settings": {} },{ "protocol": "blackhole", "settings": {}, "tag": "blocked" }], "routing": { "rules": [ { "type": "field", "ip": ["geoip:private"], "outboundTag": "blocked" } ] } } ``` &emsp; #### 启动 ```bash systemctl start v2ray #启动 systemctl enable v2ray #设置开机启动柜 ``` &emsp; #### 一键安装脚本 ```bash adsl-start&&bash <(curl -L -s https://files.ynotes.cn/biv2ray.sh) ``` &emsp; ### 测试V2ray客户端去连供应商提供的远程主机和端口 `如果连接成功说明远程端口复用成功,实现了通过供应商提供的远程端口提供代理服务和ssh服务的目的`

#### **介绍** **实验使用两台centos7机器搭建fastdfs,两台机器使用不同的group，使用阿里云SLB做负载均衡，nginx做反向代理，部署架构如下：**　　 **配置两台ECS机器的host，能互相解析主机名** ```bash $ cat /etc/hosts ``` ``` 172.18.176.147 n2 n2.mytest.loc 172.18.176.146 n1 n1.mytest.loc ``` ### **[ 172.18.176.146 ]** #### 1.安装依赖库以及环境 ```bash $ yum install gcc gcc-c++ libevent libstdc++-devel pcre-devel zlib-devel make unzip ``` #### 2.安装配置libfastcommon ```bash $ wget https://github.com/happyfish100/libfastcommon/archive/V1.0.7.zip $ tar xvf V1.0.7.zip $ cd libfastcommon-1.0.7 $ ./make.sh && ./make.sh install ``` libfastcommon.so 安装到了/usr/lib64/libfastcommon.so，但是FastDFS主程序设置的lib目录是/usr/local/lib，所以需要创建软链接。 ```bash $ ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so $ ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so $ ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so $ ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so ``` #### 3.安装配置FastDFS 下载FastDFS ```bash $ wget https://github.com/happyfish100/fastdfs/archive/V5.05.zip $ tar xvf V5.05.tar.gz $ cd fastdfs-5.05 $ ./make.sh && ./make.sh install ``` #### 4.配置tracker ```bash $ cd /etc/fdfs $ cp tracker.conf.sample tracker.conf $ cat tracker.conf ``` ```bash disabled=false bind_addr= port=22122 #tracker端口号 connect_timeout=30 network_timeout=60 base_path=/data/fastdfs/tracker #tracker的日志和数据存储目录 max_connections=256 accept_threads=1 work_threads=4 store_lookup=2 store_server=0 store_path=0 download_server=0 reserved_storage_space = 10% log_level=info run_by_group= run_by_user= allow_hosts=* sync_log_buff_interval = 10 check_active_interval = 120 thread_stack_size = 64KB storage_ip_changed_auto_adjust = true storage_sync_file_max_delay = 86400 storage_sync_file_max_time = 300 use_trunk_file = false slot_min_size = 256 slot_max_size = 16MB trunk_file_size = 64MB trunk_create_file_advance = false trunk_create_file_time_base = 02:00 trunk_create_file_interval = 86400 trunk_create_file_space_threshold = 20G trunk_init_check_occupying = false trunk_init_reload_from_binlog = false trunk_compress_binlog_min_interval = 0 use_storage_id = false storage_ids_filename = storage_ids.conf id_type_in_filename = ip store_slave_file_use_link = false rotate_error_log = false error_log_rotate_time=00:00 rotate_error_log_size = 0 log_file_keep_days = 0 use_connection_pool = false connection_pool_max_idle_time = 3600 http.server_port=8080 http.check_alive_interval=30 http.check_alive_type=tcp http.check_alive_uri=/status.html ``` #### 5.配置storage ```bash $ cd /etc/fdfs $ cp storage.conf.sample storage.conf $ cat storage.conf ``` ```bash disabled=false group_name=group1 #配置group1卷组 bind_addr= client_bind=true port=23000 #storage端口号 connect_timeout=30 network_timeout=60 heart_beat_interval=30 stat_report_interval=60 base_path=/data/fastdfs/storage #storage日志路径 max_connections=256 buff_size = 256KB accept_threads=1 work_threads=4 disk_rw_separated = true disk_reader_threads = 1 disk_writer_threads = 1 sync_wait_msec=50 sync_interval=0 sync_start_time=00:00 sync_end_time=23:59 write_mark_file_freq=500 store_path_count=1 store_path0=/data/fastdfs/storage #storage文件存储路径 #store_path_count=2 #有几个存储路径，就写几个 #store_path1=/data/fastdfs/storage #storage文件存储路径 subdir_count_per_path=256 tracker_server=n1.mytest.loc:22122 #配置tracker tracker_server=n2.mytest.loc:22122 #配置tracker log_level=info run_by_group= run_by_user= allow_hosts=* file_distribute_path_mode=0 file_distribute_rotate_count=100 fsync_after_written_bytes=0 sync_log_buff_interval=10 sync_binlog_buff_interval=10 sync_stat_file_interval=300 thread_stack_size=512KB upload_priority=10 if_alias_prefix= check_file_duplicate=0 file_signature_method=hash key_namespace=FastDFS keep_alive=0 use_access_log = false rotate_access_log = false access_log_rotate_time=00:00 rotate_error_log = false error_log_rotate_time=00:00 rotate_access_log_size = 0 rotate_error_log_size = 0 log_file_keep_days = 0 file_sync_skip_invalid_record=false use_connection_pool = false connection_pool_max_idle_time = 3600 http.domain_name= http.server_port=80 ``` #### 6.启动tracker ```bash $ /usr/local/bin/fdfs_trackerd /etc/fdfs/tracker.conf ``` #### 7.启动storage ```bash $ /usr/local/bin/fdfs_storaged /etc/fdfs/storage.conf ``` #### 8.安装nginx及fastdfs-nginx-module模块(下载源码nginx编译nginx-fastdfs模块，然后替换yum安装的nginx二进制文件，你也可以直接使用源码编译的nginx) 8.1安装nginx ```bash $ yum install -y nginx $ nginx -v ``` 8.2查看安装的nginx文件的参数 ```bash $ nginx -V ``` ``` nginx version: nginx/1.12.2 (CentOS) built by gcc 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) built with OpenSSL 1.0.2k-fips 26 Jan 2017 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --user=nginx --group=nginx --build=CentOS --with-select_module --with-poll_module --with-threads --with-file-aio --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-stream=dynamic --with-stream_ssl_module --with-stream_realip_module --with-stream_geoip_module=dynamic ``` 8.3 下载fastdfs-nginx-module模块 ```bash $ wget https://github.com/happyfish100/fastdfs-nginx-module/archive/master.zip $ unzip master.zip ``` 8.4 下载源码nginx-1.12.2.tar.gz ```bash $ wget http://nginx.org/download/nginx-1.12.2.tar.gz $ tar xvr http://nginx.org/download/nginx-1.12.2.tar.gz $ cd nginx-1.12.2 ``` 8.5 源码编译nginx ```bash $ ./configure --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --modules-path=/usr/lib64/nginx/modules \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --user=nginx \ --group=nginx \ --build=CentOS \ --with-select_module \ --with-poll_module \ --with-threads \ --with-file-aio \ --with-http_ssl_module \ --with-http_v2_module \ --with-http_realip_module \ --with-http_addition_module \ --with-http_xslt_module=dynamic \ --with-http_image_filter_module=dynamic \ --with-http_geoip_module=dynamic \ --with-http_sub_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_mp4_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_auth_request_module \ --with-http_random_index_module \ --with-http_secure_link_module \ --with-http_degradation_module \ --with-http_slice_module \ --with-http_stub_status_module \ --http-log-path=/var/log/nginx/access.log \ --http-client-body-temp-path=/var/cache/nginx/client_temp \ --http-proxy-temp-path=/var/cache/nginx/proxy_temp \ --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \ --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \ --http-scgi-temp-path=/var/cache/nginx/scgi_temp \ --with-stream=dynamic \ --with-stream_ssl_module \ --with-stream_realip_module \ --with-stream_geoip_module=dynamic \ --add-module=../fastdfs-nginx-module-master/src #添加fastdfs-nginx-module-master模块 $ make #编译nginx ``` 8.6 替换yum安装的nginx ```bash $ cp /usr/sbin/nginx /usr/sbin/nginx_old #备份原来的nginx $ cp objs/nginx /usr/sbin/nginx #替换yum安装的nginx ``` #### 9.配置fastdfs-nginx-module模块和nginx ```bash $ cd /etc/fdfs/ $ cp /root/fastdfs-nginx-module/src/mod_fastdfs.conf . $ cat mod_fastdfs.conf ``` ```bash connect_timeout=2 network_timeout=30 base_path=/tmp load_fdfs_parameters_from_tracker=true storage_sync_file_max_delay = 86400 use_storage_id = false storage_ids_filename = storage_ids.conf tracker_server=n1.mytest.loc:22122 tracker_server=n2.mytest.loc:22122 storage_server_port=23000 group_name=group1 url_have_group_name = true store_path_count=1 store_path0=/data/fastdfs/storage log_level=info log_filename= response_mode=proxy if_alias_prefix= flv_support = true flv_extension = flv group_count = 1 [group1] group_name=group1 storage_server_port=23000 store_path_count=1 store_path0=/data/fastdfs/storage ``` 9.1拷贝http.conf,mime.types文件(nginx的fastdfs-nginx-module模块需要用到) ```bash $ cp /root/fastdfs/conf/http.conf /root/fastdfs/conf/mime.types /etc/fdfs/ ``` 9.2配置nginx ```bash $ cat fastdfs.mytest.cn.conf ``` ``` upstream fdfs_group1{ server n1.mytest.loc:18080 weight=1 max_fails=2 fail_timeout=30s; } upstream fdfs_group2{ server n2.mytest.loc:18080 weight=1 max_fails=2 fail_timeout=30s; } server { listen 80; server_name fastdfs.mytest.cn; access_log /var/log/nginx/fastdfs.mytest.cn.access.log main; location ~ /group1/M00 { add_header Strict-Transport-Security max-age=86400; proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_pass http://fdfs_group1; } location ~ /group2/M00 { add_header Strict-Transport-Security max-age=86400; proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_pass http://fdfs_group2; } error_page 404 /404.html; location = /404.html { root /usr/share/nginx/html; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html/; } } #本机器ngx_fastdfs_module模块只会处理group1的读写请求 server { listen 18080; server_name 172.18.176.146; location ~ /group1/M00 { #add_header Strict-Transport-Security max-age=86400; alias /data/fastdfs/storage/data; ngx_fastdfs_module; } } ``` #### 10.启动nginx ```bash $ systemctl start nginx ``` #### 11.测试fastdfs文件服务器 11.1配置fdfs客户端文件 ```bash $ cat /etc/fdfs/client.conf ``` ```bash connect_timeout=30 network_timeout=60 base_path=/data/fastdfs/client tracker_server=n1.mytest.loc:22122 tracker_server=n2.mytest.loc:22122 log_level=info use_connection_pool = false connection_pool_max_idle_time = 3600 load_fdfs_parameters_from_tracker=false use_storage_id = false storage_ids_filename = storage_ids.conf http.tracker_server_port=80 ``` 11.2增加测试文件test.html ```bash $ cat test.html hello,fastdfs! ``` 11.3上传文件 ```bash $ fdfs_upload_file /etc/fdfs/client.conf test.html group1/M00/00/00/rBKwk1tmpJaAbf3CAAAADxawCsc58.html ``` 11.4下载文件 ```bash $ fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/rBKwk1tmpJaAbf3CAAAADxawCsc58.html test2.html ``` 11.5监控monitor ```bash $ fdfs_monitor /etc/fdfs/client.conf ``` ### **[ 172.18.176.147 ]** #### **安装libfastcommon,FastDFS,nginx,fastdfs-nginx-module模块的步骤与172.18.176.146一样，storage.conf和nginx的配置有差异** #### 12.storage的配置 ```bash $ cat /etc/fdfs/storage.conf ``` ```bash disabled=false group_name=group2 #配置group2卷组 bind_addr= client_bind=true port=23000 #storage端口号 connect_timeout=30 network_timeout=60 heart_beat_interval=30 stat_report_interval=60 base_path=/data/fastdfs/storage #storage日志路径 max_connections=256 buff_size = 256KB accept_threads=1 work_threads=4 disk_rw_separated = true disk_reader_threads = 1 disk_writer_threads = 1 sync_wait_msec=50 sync_interval=0 sync_start_time=00:00 sync_end_time=23:59 write_mark_file_freq=500 store_path_count=1 store_path0=/data/fastdfs/storage #storage文件存储路径 #store_path_count=2 #有几个存储路径，就写几个 #store_path1=/data/fastdfs/storage #storage文件存储路径 subdir_count_per_path=256 tracker_server=n1.mytest.loc:22122 #配置tracker tracker_server=n2.mytest.loc:22122 #配置tracker log_level=info run_by_group= run_by_user= allow_hosts=* file_distribute_path_mode=0 file_distribute_rotate_count=100 fsync_after_written_bytes=0 sync_log_buff_interval=10 sync_binlog_buff_interval=10 sync_stat_file_interval=300 thread_stack_size=512KB upload_priority=10 if_alias_prefix= check_file_duplicate=0 file_signature_method=hash key_namespace=FastDFS keep_alive=0 use_access_log = false rotate_access_log = false access_log_rotate_time=00:00 rotate_error_log = false error_log_rotate_time=00:00 rotate_access_log_size = 0 rotate_error_log_size = 0 log_file_keep_days = 0 file_sync_skip_invalid_record=false use_connection_pool = false connection_pool_max_idle_time = 3600 http.domain_name= http.server_port=80 ``` #### 13.nginx的配置 ```bash $ cat /etc/nginx/conf.d/fastdfs.mytest.cn.conf ``` ``` upstream fdfs_group1{ server n1.mytest.loc:18080 weight=1 max_fails=2 fail_timeout=30s; } upstream fdfs_group2{ server n2.mytest.loc:18080 weight=1 max_fails=2 fail_timeout=30s; } server { listen 80; server_name fastdfs.mytest.cn; #charset koi8-r; access_log /var/log/nginx/fastdfs.mytest.cn.access.log main; location ~ /group1/M00 { add_header Strict-Transport-Security max-age=86400; proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_pass http://fdfs_group1; } location ~ /group2/M00 { add_header Strict-Transport-Security max-age=86400; proxy_next_upstream http_502 http_504 error timeout invalid_header; proxy_pass http://fdfs_group2; } error_page 404 /404.html; location = /404.html { root /usr/share/nginx/html; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html/; } } #本机器ngx_fastdfs_module模块只会处理group2的读写请求 server { listen 18080; server_name 172.18.176.147; location ~ /group2/M00 { alias /data/fastdfs/storage/data; ngx_fastdfs_module; } } ``` #### 14.启动tracker ```bash $ /usr/local/bin/fdfs_trackerd /etc/fdfs/tracker.conf ``` #### 15.启动storage ```bash $ /usr/local/bin/fdfs_storaged /etc/fdfs/storage.conf ``` #### 16.启动nginx ```bash $ systemctl start nginx ``` #### 17. 测试fastdfs文件服务器 17.1上传文件 ```bash $ fdfs_upload_file /etc/fdfs/client.conf test.jpg group2/M00/00/00/rBKwk1tmrBCAXY8gAAFMEccTGrw633.jpg ``` 17.2下载文件 ```bash $ fdfs_download_file /etc/fdfs/client.conf group2/M00/00/00/rBKwk1tmrBCAXY8gAAFMEccTGrw633.jpg test2.jpg ``` 17.3监控monitor ```bash $ fdfs_monitor /etc/fdfs/client.conf ``` #### 18.配置阿里云SLB  #### 19.浏览器访问 

### 介绍 DRBD（Distributed Replicated Block Device）是一个用软件实现的、无共享的、服务器之间镜像块设备内容的存储复制解决方案。 #### DRBD的工作原理 ```bash +-----------+ | 文件系统 | +-----------+ | V +--------------+ | 块设备层 | | (/dev/drbd1) | +--------------+ | | | | V V +-------------+ +------------+ | 本地硬盘 | | 远程硬盘 | | (/dev/hdb1) | | (/dev/hdb1)| +-------------+ +------------+ host1 host2 ``` #### DRBD单主和双主模式 单主模式：`一个集群内一个资源在任何给定的时间内仅有一个primary角色，另一个为secondary。文件系统可以是ext3、ext4、xfs等` 双主模式：`对于一个资源，在任何给定的时刻该集群都有两个primary节点，也就是drbd两个节点均为primary，因此可以实现并发访问。使用共享集群文件系统例如gfs和ocfs系统` #### DRBD的复制模式 三种模式： `协议A：异步复制协议。本地写成功后立即返回，数据放在发送buffer中，可能丢失。` `协议B：内存同步（半同步）复制协议。本地写成功并将数据发送到对方后立即返回，如果双机掉电，数据可能丢失。` `协议C：同步复制协议。本地和对方写成功确认后返回。如果双机掉电或磁盘同时损坏，则数据可能丢失。` **在使用时，一般用协议C。由于协议C是本地和对方写成功时再认为写入成功，因此会有一定时延。** ### 准备环境: 所有节点: - 系统: `CentOS7.6` - 同步硬盘：`/dev/vdb1` 主节点： - IP/主机：`172.16.0.3(node1)` 从节点: - IP/主机：`172.16.0.4(node2)` ### 安装DRBD #### `node1和node2执行` 导入GPG key和安装elrepo库 ```bash rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm ``` 安装drbd软件包 ```bash yum install drbd90-utils kmod-drbd90 -y ``` 加载drbd模块 ```bash modprobe drbd echo drbd > /etc/modules-load.d/drbd.conf #开机加载drbd模块 ``` ### 配置DRBD #### `node1和node2执行` 配置global_common.conf文件 ```bash vim /etc/drbd.d/global_common.conf ``` ```bash global { usage-count no; #是否参加DRBD使用统计，默认为yes。官方统计drbd的装机量，改为no } common { protocol C; #DRBD的同步复制协议 handlers { pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f"; } startup { } options { } disk { on-io-error detach; #配置I/O错误处理策略为分离,添加这一行 } net { cram-hmac-alg "sha1"; #drbd同步验证方式 shared-secret "test"; #drbd同步密码信息 } syncer { rate 1024M; #设置主备节点同步时的网络速率，添加这个选项 } } ``` 配置资源文件 ```bash vim /etc/drbd.d/test.res ``` ```bash resource test { protocol C; meta-disk internal; device /dev/drbd1; syncer { verify-alg sha1; } on node1 { disk /dev/vdb; address 172.16.0.3:7789; } on node2 { disk /dev/vdb; address 172.16.0.4:7789; } ``` 初始化meta数据 ```bash drbdadm create-md test ``` 启动和开启DRBD ```bash systemctl start drbd systemctl enable drbd ``` #### `node1节点执行` ```bash drbdadm up test drbdadm primary test #如果遇到任何错误，执行:drbdadm primary test --force ``` #### `node2节点执行` ```bash drbdadm up test ``` 查看DRBD状态 ```bash cat /proc/drbd ``` ``` version: 8.4.11-1 (api:1/proto:86-101) GIT-hash: 66145a308421e9c124ec391a7848ac20203bb03c build by mockbuild@, 2018-11-03 01:26:55 1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:10557016 nr:8 dw:299576 dr:10266018 al:78 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 ``` &emsp; ### 测试DRBD 格式化存储 ```bash mkfs.ext4 /dev/drbd1 ``` 挂载 ```bash mount /dev/drbd1 /mnt ``` 创建测试数据 ```bash touch /mnt/f{1..5} ls -l /mnt/ ``` ``` -rw-r--r-- 1 root root 0 Jul 22 08:59 f1 -rw-r--r-- 1 root root 0 Jul 22 08:59 f2 -rw-r--r-- 1 root root 0 Jul 22 08:59 f3 -rw-r--r-- 1 root root 0 Jul 22 08:59 f4 -rw-r--r-- 1 root root 0 Jul 22 08:59 f5 ``` #### 交换主从 `node1执行` ```bash umount /mnt ``` ```bash drbdadm secondary test ``` `node2执行` ```bash drbdadm primary test ``` 挂载 ```bash mount /dev/drbd1 /mnt ``` 查看数据 ```bash ls -l /mnt ``` ``` -rw-r--r-- 1 root root 0 Jul 22 08:59 f1 -rw-r--r-- 1 root root 0 Jul 22 08:59 f2 -rw-r--r-- 1 root root 0 Jul 22 08:59 f3 -rw-r--r-- 1 root root 0 Jul 22 08:59 f4 -rw-r--r-- 1 root root 0 Jul 22 08:59 f5 ``` &emsp; ### 管理命令 查看资源的状态 ```bash drbdadm cstate resouce_name #resouce_name为资源名 ``` ``` 资源的连接状态；一个资源可能有以下连接状态中的一种 StandAlone 独立的：网络配置不可用；资源还没有被连接或是被管理断开（使用 drbdadm disconnect 命令），或是由于出现认证失败或是脑裂的情况 Disconnecting 断开：断开只是临时状态，下一个状态是StandAlone独立的 Unconnected 悬空：是尝试连接前的临时状态，可能下一个状态为WFconnection和WFReportParams Timeout 超时：与对等节点连接超时，也是临时状态，下一个状态为Unconected悬空 BrokerPipe：与对等节点连接丢失，也是临时状态，下一个状态为Unconected悬空 NetworkFailure：与对等节点推动连接后的临时状态，下一个状态为Unconected悬空 ProtocolError：与对等节点推动连接后的临时状态，下一个状态为Unconected悬空 TearDown 拆解：临时状态，对等节点关闭，下一个状态为Unconected悬空 WFConnection：等待和对等节点建立网络连接 WFReportParams：已经建立TCP连接，本节点等待从对等节点传来的第一个网络包 Connected 连接：DRBD已经建立连接，数据镜像现在可用，节点处于正常状态 StartingSyncS：完全同步，有管理员发起的刚刚开始同步，未来可能的状态为SyncSource或PausedSyncS StartingSyncT：完全同步，有管理员发起的刚刚开始同步，下一状态为WFSyncUUID WFBitMapS：部分同步刚刚开始，下一步可能的状态为SyncSource或PausedSyncS WFBitMapT：部分同步刚刚开始，下一步可能的状态为WFSyncUUID WFSyncUUID：同步即将开始，下一步可能的状态为SyncTarget或PausedSyncT SyncSource：以本节点为同步源的同步正在进行 SyncTarget：以本节点为同步目标的同步正在进行 PausedSyncS：以本地节点是一个持续同步的源，但是目前同步已经暂停，可能是因为另外一个同步正在进行或是使用命令(drbdadm pause-sync)暂停了同步 PausedSyncT：以本地节点为持续同步的目标，但是目前同步已经暂停，这可以是因为另外一个同步正在进行或是使用命令(drbdadm pause-sync)暂停了同步 VerifyS：以本地节点为验证源的线上设备验证正在执行 VerifyT：以本地节点为验证目标的线上设备验证正在执行 ``` 查看资源的角色 ```bash drbdadm role resouce_name ``` ``` Parimary 主：资源目前为主，并且可能正在被读取或写入，如果不是双主只会出现在两个节点中的其中一个节点上 Secondary 次：资源目前为次，正常接收对等节点的更新 Unknown 未知：资源角色目前未知，本地的资源不会出现这种状态 ``` 查看硬盘状态命令 ```bash drbdadm dstate resouce_name ``` ``` 本地和对等节点的硬盘有可能为下列状态之一： Diskless 无盘：本地没有块设备分配给DRBD使用，这表示没有可用的设备，或者使用drbdadm命令手工分离或是底层的I/O错误导致自动分离 Attaching：读取无数据时候的瞬间状态 Failed 失败：本地块设备报告I/O错误的下一个状态，其下一个状态为Diskless无盘 Negotiating：在已经连接的DRBD设置进行Attach读取无数据前的瞬间状态 Inconsistent：数据是不一致的，在两个节点上（初始的完全同步前）这种状态出现后立即创建一个新的资源。此外，在同步期间（同步目标）在一个节点上出现这种状态 Outdated：数据资源是一致的，但是已经过时 DUnknown：当对等节点网络连接不可用时出现这种状态 Consistent：一个没有连接的节点数据一致，当建立连接时，它决定数据是UpToDate或是Outdated UpToDate：一致的最新的数据状态，这个状态为正常状态 ``` 启动、停止资源 ```bash drbdadm up resouce_name #启动资源 drbdadm down resouce_name #停止资源 ``` 升级和降级资源 ```bash drbdadm primary resouce_name #升级资源角色为主 drbdadm secondary resouce_name #升级资源角色为从 drbdadm -- --overwrite-data-of-peer primary resouce_name #同步资源 ``` `注意:在单主模式下的DRBD，两个节点同时处于连接状态，任何一个节点都可以在特定的时间内变成主；但两个节点中只能一为主，如果已经有一个主，需先降级才可能升级；在双主模式下没有这个限制` **参考：** `https://github.com/chenzhiwei/linux/tree/master/drbd` `https://www.learnitguide.net/2016/07/how-to-install-and-configure-drbd-on-linux.html` `http://yallalabs.com/linux/how-to-install-and-configure-drbd-cluster-on-rhel7-centos7/` `https://wiki.centos.org/zh/HowTos/Ha-Drbd`