兜兜    2018-07-09 17:35:10    2018-11-14 14:34:52   

HAProxy 高可用
### HAProxy安装 #### yum安装 ```bash yum install haproxy -y ``` 启动、停止和重启 ```bash systemctl start haproxy systemctl stop haproxy systemctl restart haproxy ```   #### 源码安装 下载 ```bash wget http://www.haproxy.org/download/2.0/src/haproxy-2.0.1.tar.gz tar -xzf haproxy-2.0.1.tar.gz ``` 编译并安装 ```bash make PREFIX=/opt/haproxy TARGET=linux2628 make install PREFIX=/opt/haproxy ``` ``` - linux22 for Linux 2.2 - linux24 for Linux 2.4 and above (default) - linux24e for Linux 2.4 with support for a working epoll (> 0.21) - linux26 for Linux 2.6 and above - linux2628 for Linux 2.6.28, 3.x, and above (enables splice and tproxy) ``` 创建配置文件 ```bash mkdir -p /opt/haproxy/conf vi /opt/haproxy/conf/haproxy.cfg ``` ```ini global #全局属性 daemon #以daemon方式在后台运行 maxconn 256 #最大同时256连接 pidfile /opt/haproxy/conf/haproxy.pid #指定保存HAProxy进程号的文件 defaults #默认参数 mode http #http模式 timeout connect 5000ms #连接server端超时5s timeout client 50000ms #客户端响应超时50s timeout server 50000ms #server端响应超时50s frontend http-in #前端服务http-in bind *:8080 #监听8080端口 default_backend servers backend servers #后端服务servers server server1 127.0.0.1:8000 maxconn 32 ``` 创建启动脚本 ```bash vi /etc/init.d/haproxy ``` ```bash #!/bin/sh set -e PATH=/sbin:/bin:/usr/sbin:/usr/bin:/opt/haproxy/sbin PROGDIR=/opt/haproxy PROGNAME=haproxy DAEMON=$PROGDIR/sbin/$PROGNAME CONFIG=$PROGDIR/conf/$PROGNAME.cfg PIDFILE=$PROGDIR/conf/$PROGNAME.pid DESC="HAProxy daemon" SCRIPTNAME=/etc/init.d/$PROGNAME # Gracefully exit if the package has been removed. test -x $DAEMON || exit 0 start() { echo -e "Starting $DESC: $PROGNAME\n" $DAEMON -f $CONFIG echo "." } stop() { echo -e "Stopping $DESC: $PROGNAME\n" haproxy_pid="$(cat $PIDFILE)" kill $haproxy_pid echo "." } restart() { echo -e "Restarting $DESC: $PROGNAME\n" $DAEMON -f $CONFIG -p $PIDFILE -sf $(cat $PIDFILE) echo "." } case "$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo "Usage: $SCRIPTNAME {start|stop|restart}" >&2 exit 1 ;; esac exit 0 ``` 添加执行权限 ```bash chmod +x /etc/init.d/haproxy ``` 启动、停止和重启 ```bash service haproxy start service haproxy stop service haproxy restart ```   ### HAProxy的配置介绍 ```ini 总览 HAProxy的配置文件共有5个域 global:用于配置全局参数 default:用于配置所有frontend和backend的默认属性 frontend:用于配置前端服务(即HAProxy自身提供的服务)实例 backend:用于配置后端服务(即HAProxy后面接的服务)实例组 listen:frontend+backend的组合配置,可以理解成更简洁的配置方法 global域的关键配置 daemon:指定HAProxy以后台模式运行,通常情况下都应该使用这一配置 user [username] :指定HAProxy进程所属的用户 group [groupname] :指定HAProxy进程所属的用户组 log [address] [device] [maxlevel] [minlevel]:日志输出配置,如log 127.0.0.1 local0 info warning,即向本机rsyslog或syslog的local0输出info到warning级别的日志。其中[minlevel]可以省略。HAProxy的日志共有8个级别,从高到低为emerg/alert/crit/err/warning/notice/info/debug pidfile :指定记录HAProxy进程号的文件绝对路径。主要用于HAProxy进程的停止和重启动作。 maxconn :HAProxy进程同时处理的连接数,当连接数达到这一数值时,HAProxy将停止接收连接请求 frontend域的关键配置 acl [name] [criterion] [flags] [operator] [value]:定义一条ACL,ACL是根据数据包的指定属性以指定表达式计算出的true/false值。如"acl url_ms1 path_beg -i /ms1/"定义了名为url_ms1的ACL,该ACL在请求uri以/ms1/开头(忽略大小写)时为true bind [ip]:[port]:frontend服务监听的端口 default_backend [name]:frontend对应的默认backend disabled:禁用此frontend http-request [operation] [condition]:对所有到达此frontend的HTTP请求应用的策略,例如可以拒绝、要求认证、添加header、替换header、定义ACL等等。 http-response [operation] [condition]:对所有从此frontend返回的HTTP响应应用的策略,大体同上 log:同global域的log配置,仅应用于此frontend。如果要沿用global域的log配置,则此处配置为log global maxconn:同global域的maxconn,仅应用于此frontend mode:此frontend的工作模式,主要有http和tcp两种,对应L7和L4两种负载均衡模式 option forwardfor:在请求中添加X-Forwarded-For Header,记录客户端ip option http-keep-alive:以KeepAlive模式提供服务 option httpclose:与http-keep-alive对应,关闭KeepAlive模式,如果HAProxy主要提供的是接口类型的服务,可以考虑采用httpclose模式,以节省连接数资源。但如果这样做了,接口的调用端将不能使用HTTP连接池 option httplog:开启httplog,HAProxy将会以类似Apache HTTP或Nginx的格式来记录请求日志 option tcplog:开启tcplog,HAProxy将会在日志中记录数据包在传输层的更多属性 stats uri [uri]:在此frontend上开启监控页面,通过[uri]访问 stats refresh [time]:监控数据刷新周期 stats auth [user]:[password]:监控页面的认证用户名密码 timeout client [time]:指连接创建后,客户端持续不发送数据的超时时间 timeout http-request [time]:指连接创建后,客户端没能发送完整HTTP请求的超时时间,主要用于防止DoS类攻击,即创建连接后,以非常缓慢的速度发送请求包,导致HAProxy连接被长时间占用 use_backend [backend] if|unless [acl]:与ACL搭配使用,在满足/不满足ACL时转发至指定的backend backend域的关键配置 acl:同frontend域 balance [algorithm]:在此backend下所有server间的负载均衡算法,常用的有roundrobin和source,完整的算法说明见官方文档configuration.html#4.2-balance cookie:在backend server间启用基于cookie的会话保持策略,最常用的是insert方式,如cookie HA_STICKY_ms1 insert indirect nocache,指HAProxy将在响应中插入名为HA_STICKY_ms1的cookie,其值为对应的server定义中指定的值,并根据请求中此cookie的值决定转发至哪个server。indirect代表如果请求中已经带有合法的HA_STICK_ms1 cookie,则HAProxy不会在响应中再次插入此cookie,nocache则代表禁止链路上的所有网关和缓存服务器缓存带有Set-Cookie头的响应。 default-server:用于指定此backend下所有server的默认设置。具体见下面的server配置。 disabled:禁用此backend http-request/http-response:同frontend域 log:同frontend域 mode:同frontend域 option forwardfor:同frontend域 option http-keep-alive:同frontend域 option httpclose:同frontend域 option httpchk [METHOD] [URL] [VERSION]:定义以http方式进行的健康检查策略。如option httpchk GET /healthCheck.html HTTP/1.1 option httplog:同frontend域 option tcplog:同frontend域 server [name] [ip]:[port] [params]:定义backend中的一个后端server,[params]用于指定这个server的参数,常用的包括有: check:指定此参数时,HAProxy将会对此server执行健康检查,检查方法在option httpchk中配置。同时还可以在check后指定inter, rise, fall三个参数,分别代表健康检查的周期、连续几次成功认为server UP,连续几次失败认为server DOWN,默认值是inter 2000ms rise 2 fall 3 cookie [value]:用于配合基于cookie的会话保持,如cookie ms1.srv1代表交由此server处理的请求会在响应中写入值为ms1.srv1的cookie(具体的cookie名则在backend域中的cookie设置中指定) maxconn:指HAProxy最多同时向此server发起的连接数,当连接数到达maxconn后,向此server发起的新连接会进入等待队列。默认为0,即无限 maxqueue:等待队列的长度,当队列已满后,后续请求将会发至此backend下的其他server,默认为0,即无限 weight:server的权重,0-256,权重越大,分给这个server的请求就越多。weight为0的server将不会被分配任何新的连接。所有server默认weight为1 timeout connect [time]:指HAProxy尝试与backend server创建连接的超时时间 timeout check [time]:默认情况下,健康检查的连接+响应超时时间为server命令中指定的inter值,如果配置了timeout check,HAProxy会以inter作为健康检查请求的连接超时时间,并以timeout check的值作为健康检查请求的响应超时时间 timeout server [time]:指backend server响应HAProxy请求的超时时间 default域 上文所属的frontend和backend域关键配置中,除acl、bind、http-request、http-response、use_backend外,其余的均可以配置在default域中。default域中配置了的项目,如果在frontend或backend域中没有配置,将会使用default域中的配置。 listen域 listen域是frontend域和backend域的组合,frontend域和backend域中所有的配置都可以配置在listen域下 ```   ### HAProxy配置代理(7层)web服务 后端两台web服务器192.168.50.251和192.168.50.252,url匹配/app1/访问192.168.50.251,url匹配/app2/访问192.168.50.252 ```bash vim /etc/haproxy/haproxy.cfg ``` ```ini global log 127.0.0.1 local0 info log 127.0.0.1 local1 warning chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 40000 user haproxy group haproxy daemon stats socket /var/lib/haproxy/stats defaults mode tcp log global option tcplog option dontlognull option http-server-close timeout connect 10s timeout client 1m timeout server 1m maxconn 3000 frontend public mode tcp bind :80 log-format %ft\ %b/%s acl url_app1 path_beg -i /app1/ acl url_app2 path_beg -i /app2/ use_backend app1 if url_app1 default_backend default_servers backend app1 mode http server web1 192.168.50.251:5555 maxconn 300 check backend app2 mode http server web2 192.168.50.252:5555 maxconn 300 check backend default_servers mode http balance roundrobin cookie SERVER insert indirect #开启会话粘滞 server web1 192.168.50.251:5555 maxconn 300 cookie server1 check server web2 192.168.50.252:5555 maxconn 300 cookie server2 check listen status bind *:7777 mode http stats enable stats refresh 10s stats uri /haproxy stats realm Haproxy\ Statistics stats auth admin:admin stats hide-version ``` 配置rsyslog日志 ```bash vim /etc/rsyslog.conf ``` ```ini local0.* /var/log/haproxy_info.log local1.* /var/log/haproxy_warning.log ```   ### HAProxy配置复用8080 haproxy使用8080端口分别代理ssh和web,原理是通过获取请求数据包前三个字节的二进制值,匹配代理到不同的后端服务。 ```bash vim /etc/haproxy/haproxy.cfg ``` ```ini global log 127.0.0.1 local0 info log 127.0.0.1 local1 warning chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 40000 user haproxy group haproxy daemon stats socket /var/lib/haproxy/stats defaults mode tcp log global option tcplog option dontlognull timeout connect 10s timeout client 1m timeout server 1m maxconn 3000 frontend public mode tcp bind :8080 log global option tcplog log-format %ft\ %b/%s tcp-request inspect-delay 5s acl is_ssh req.payload(0,3) -m bin 535348 #数据包前三个字节,SSH acl is_http req.payload(0,3) -m bin 474554 504f53 505554 44454c 4f5054 484541 434f4e 545241 #数据包前三个字节做匹配,GET,POST,DELETE,PUT,OPTIONS,HEAD,TRACE,CONNECT tcp-request content accept if is_http tcp-request content accept if is_ssh use_backend web if is_http use_backend ssh if is_ssh backend web mode http balance roundrobin server web1 192.168.50.251:5555 check server web2 192.168.50.252:5555 check backend ssh mode tcp timeout server 3h server ssh 192.168.50.252:22 ```
阅读 562 评论 0 收藏 0
阅读 562
评论 0
收藏 0

兜兜    2018-07-02 11:31:47    2018-11-14 14:35:02   

HAProxy Keepalived 高可用 负载均衡 LoadBalance
### 准备工作 ```bash HAProxy/Keepalived 192.168.50.250 (Master) 192.168.50.253 (Backup) web服务器 192.168.50.251 192.168.50.252 VIP地址 192.168.50.240 ``` ### HAProxy(Master) #### 安装HAProxy ```bash yum install haproxy -y ``` &emsp; #### 开启IP转发 ```bash echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf sysctl -p ``` ``` net.ipv4.ip_forward = 1 net.ipv4.ip_nonlocal_bind = 1 ``` &emsp; #### 配置HAProxy ```bash cat /etc/haproxy/haproxy.cfg ``` ```ini global log 127.0.0.1 local2 #日志输出配置,所有日志都记录在本机,通过local0输出 chroot /var/lib/haproxy #改变工作目录 pidfile /var/run/haproxy.pid maxconn 80000 #限制单个进程的最大连接数 user haproxy #所属运行用户 group haproxy #所属运行用户组 daemon #后台运行 nbproc 1 #指定作为守护进程运行时的进程数 stats socket /var/lib/haproxy/stats defaults mode http #mode {http|tcp|health},http是七层模式,tcp是四层模式,health是健康检测返回OK log global option httplog #http 日志格式 option dontlognull #不记录空连接 option http-server-close option forwardfor except 127.0.0.0/8 option redispatch #在连接失败或断开的情况下,允许当前会话被重新分发 retries 3 #设置在一个服务器上链接失败后的重连次数 timeout http-request 10s timeout queue 1m timeout connect 10s #连接超时 timeout client 1m #客户端超时 timeout server 1m #服务器超时 timeout http-keep-alive 10s timeout check 10s #心跳检测超时 maxconn 80000 #限制单个进程的最大连接数 #前端代理web frontend web bind *:5555 #acl www hdr(host) -i www.ynotes.cn #acl规则,-i是访问的域名,如果访问的是www.ynotes.cn,分发到后端www #acl image hdr(host) -i files.ynotes.cn #use_backend www if www #use_backend image if image default_backend web #backend www # mode http # balance roundrobin # server web2 192.168.50.252:5555 check #backend image # mode http # balance roundrobin # server web1 192.168.50.251:5555 check backend web balance roundrobin server web1 192.168.50.251:5555 check inter 2000 fall 3 server web2 192.168.50.252:5555 check inter 2000 fall 3 listen status #启动统计页面 bind *:7777 mode http stats enable stats refresh 10s stats uri /haproxy stats realm Haproxy\ Statistics stats auth admin:admin stats hide-version ``` #### 开启HAProxy日志 修改rsyslog配置文件 ```bash vim /etc/rsyslog.conf ``` ```ini #启用在udp 514端口接收日志消息 $ModLoad imudp $UDPServerRun 514 #在rules(规则)节中添加如下信息 local2.* /var/log/haproxy.log #表示将发往facility local2的消息写入haproxy.log文件中,"local2.* "前面的local2表示facility,预定义的。*表示所有等级的消息 ``` 重启rsyslog ```bash systemctl restart rsyslog ``` &emsp; #### 配置两台nginx 192.168.50.251/192.168.50.252 ```bash cat /etc/nginx.conf ``` ```ini ... server { listen 5555; location / { root /var/www/haproxy/node; } } ... ``` 192.168.50.251 ```bash echo 192.168.50.251 >/var/www/haproxy/node/index.html ``` 192.168.50.252 ```bash echo 192.168.50.252 >/var/www/haproxy/node/index.html ``` &emsp; #### HAProxy启动关闭与开机启动 启动/关闭 ```bash systemctl start haproxy systemctl stop haproxy ``` 开机启动/禁用 ```bash systemctl enable haproxy systemctl disable haproxy ``` &emsp; #### 防火墙开启访问HAProxy代理的服务 iptable ```bash iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 5555 -j ACCEPT iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 7777 -j ACCEPT ``` firewalld ```bash firewall-cmd --zone=<zone> --add-port=5555/tcp --permanent#zone指定网卡接口应用的区域,可通过firewall-cmd --get-zone-of-interface=<interface> 查看网卡所在区域,添加网卡到指定区域firewall-cmd --permanent --zone=<zone> --change-interface=<interface> firewall-cmd --zone=<zone> --add-port=7777/tcp --permanent firewall-cmd --reload ``` &emsp; #### 测试访问HAProxy代理 ```bash while true; do curl http://192.168.50.253:5555; sleep 1; done ``` ``` 192.168.50.252 192.168.50.251 192.168.50.252 192.168.50.251 192.168.50.252 ^C ``` &emsp; #### 访问统计页面 http://192.168.50.253:7777/haproxy ![](https://files.ynotes.cn/haproxy_statistics.png) &emsp; #### 配置HAProxy会话粘滞 开启会话粘滞,使用cookie参数SERVER的值做匹配 ```bash cat /etc/haproxy/haproxy.cfg ``` ```ini #balance roundrobin #注释改行 cookie SERVER insert server web1 192.168.50.251:5555 cookie 1 check server web2 192.168.50.252:5555 cookie 2 check ``` 测试 ```bash while true; do curl http://192.168.50.253:5555 --cookie "SERVER=1"; sleep 1; done ``` ``` 192.168.50.251 192.168.50.251 192.168.50.251 ^C ``` ```bash while true; do curl http://192.168.50.253:5555 --cookie "SERVER=2"; sleep 1; done ``` ``` 192.168.50.252 192.168.50.252 192.168.50.252 ^C ``` 开启会话粘滞,使用cookie参数前缀名做匹配,使用"\~"做分隔符,以SESSIONID为例,格式如:set-Cookie: SESSIONID=N\~Session_ID; ```bash cat /etc/haproxy/haproxy.cfg ``` ```ini #balance roundrobin #注释改行 cookie SESSIONID prefix server web1 192.168.50.251:5555 cookie 1 check server web2 192.168.50.252:5555 cookie 2 check ``` 测试 ```bash while true; do curl http://192.168.50.253:5555 --cookie "SESSIONID=1~AAA"; sleep 1; done ``` ``` 192.168.50.251 192.168.50.251 192.168.50.251 ^C ``` ```bash while true; do curl http://192.168.50.253:5555 --cookie "SESSIONID=2~AAA"; sleep 1; done ``` ``` 192.168.50.252 192.168.50.252 192.168.50.252 ^C ``` &emsp; ### HAProxy(Backup) `同Master` &emsp; ### keepalived(Master) #### 安装keepalived ```bash yum install keepalived -y ``` #### 配置Keepalived ```bash vim /etc/keepalived/keepalived.conf ``` ```bash global_defs { notification_email { test01@ynotes.cn } notification_email_from haproxy1@ynotes.cn smtp_server localhost smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0 } vrrp_script chk_haproxy { script "/etc/keepalived/check_haproxy.sh" interval 5 weight -4 } vrrp_instance VI_1 { state MASTER interface enp0s3 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.50.240 } track_script { chk_haproxy } } ``` ```bash cat /etc/keepalived/check_haproxy.sh ``` ```bash #!/bin/bash if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then systemctl start haproxy sleep 2 #睡眠时间少于vrrp_script 中的interval 5参数值 if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then systemctl stop keepalived fi fi ``` #### 开启路由转发(前面已开启,如果单独配置keepalived需开启) ```bash echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf sysctl -p ``` ``` net.ipv4.ip_forward = 1 ``` #### Keepalived启动关闭与开机启动 启动/关闭 ```bash systemctl start keepalived systemctl stop keepalived ``` 开机启动/禁用 ```bash systemctl enable keepalived systemctl disable keepalived ``` &emsp; ### keepalived(Backup) #### 安装keepalived ```bash yum install keepalived -y ``` #### 配置Keepalived ```bash vim /etc/keepalived/keepalived.conf ``` ```bash global_defs { notification_email { test01@ynotes.cn } notification_email_from haproxy1@ynotes.cn smtp_server localhost smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0 } vrrp_script chk_haproxy { script "/etc/keepalived/check_haproxy.sh" interval 5 weight -4 } vrrp_instance VI_1 { state BACKUP interface enp0s3 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.50.240 } track_script { chk_haproxy } } ``` `其他同Master` &emsp; #### 测试 停止192.168.50.253的keeaplived ```bash systemctl stop keepalived ``` 查看192.168.50.253的vip ```bash ip a|grep 192.168.50.240 #执行无输出 ``` 查看192.168.50.250的vip ```bash ip a|grep 192.168.50.240 #输出VIP ``` ```bash inet 192.168.50.240/32 scope global enp0s3 ``` 访问192.168.50.240:5555 ```bash curl http://192.168.50.240:5555 #看到192.168.50.250成功接管VIP,并且能访问页面 ``` ``` 192.168.50.252 ```
阅读 847 评论 0 收藏 0
阅读 847
评论 0
收藏 0