YNOTES笔记

### HAProxy安装 #### yum安装 ```bash yum install haproxy -y ``` 启动、停止和重启 ```bash systemctl start haproxy systemctl stop haproxy systemctl restart haproxy ``` &emsp; #### 源码安装下载 ```bash wget http://www.haproxy.org/download/2.0/src/haproxy-2.0.1.tar.gz tar -xzf haproxy-2.0.1.tar.gz ``` 编译并安装 ```bash make PREFIX=/opt/haproxy TARGET=linux2628 make install PREFIX=/opt/haproxy ``` ``` - linux22 for Linux 2.2 - linux24 for Linux 2.4 and above (default) - linux24e for Linux 2.4 with support for a working epoll (> 0.21) - linux26 for Linux 2.6 and above - linux2628 for Linux 2.6.28, 3.x, and above (enables splice and tproxy) ``` 创建配置文件 ```bash mkdir -p /opt/haproxy/conf vi /opt/haproxy/conf/haproxy.cfg ``` ```ini global #全局属性 daemon #以daemon方式在后台运行 maxconn 256 #最大同时256连接 pidfile /opt/haproxy/conf/haproxy.pid #指定保存HAProxy进程号的文件 defaults #默认参数 mode http #http模式 timeout connect 5000ms #连接server端超时5s timeout client 50000ms #客户端响应超时50s timeout server 50000ms #server端响应超时50s frontend http-in #前端服务http-in bind *:8080 #监听8080端口 default_backend servers backend servers #后端服务servers server server1 127.0.0.1:8000 maxconn 32 ``` 创建启动脚本 ```bash vi /etc/init.d/haproxy ``` ```bash #!/bin/sh set -e PATH=/sbin:/bin:/usr/sbin:/usr/bin:/opt/haproxy/sbin PROGDIR=/opt/haproxy PROGNAME=haproxy DAEMON=$PROGDIR/sbin/$PROGNAME CONFIG=$PROGDIR/conf/$PROGNAME.cfg PIDFILE=$PROGDIR/conf/$PROGNAME.pid DESC="HAProxy daemon" SCRIPTNAME=/etc/init.d/$PROGNAME # Gracefully exit if the package has been removed. test -x $DAEMON || exit 0 start() { echo -e "Starting $DESC: $PROGNAME\n" $DAEMON -f $CONFIG echo "." } stop() { echo -e "Stopping $DESC: $PROGNAME\n" haproxy_pid="$(cat $PIDFILE)" kill $haproxy_pid echo "." } restart() { echo -e "Restarting $DESC: $PROGNAME\n" $DAEMON -f $CONFIG -p $PIDFILE -sf $(cat $PIDFILE) echo "." } case "$1" in start) start ;; stop) stop ;; restart) restart ;; *) echo "Usage: $SCRIPTNAME {start|stop|restart}" >&2 exit 1 ;; esac exit 0 ``` 添加执行权限 ```bash chmod +x /etc/init.d/haproxy ``` 启动、停止和重启 ```bash service haproxy start service haproxy stop service haproxy restart ``` &emsp; ### HAProxy的配置介绍 ```ini 总览 HAProxy的配置文件共有5个域 global：用于配置全局参数 default：用于配置所有frontend和backend的默认属性 frontend：用于配置前端服务（即HAProxy自身提供的服务）实例 backend：用于配置后端服务（即HAProxy后面接的服务）实例组 listen：frontend+backend的组合配置，可以理解成更简洁的配置方法 global域的关键配置 daemon：指定HAProxy以后台模式运行，通常情况下都应该使用这一配置 user [username] ：指定HAProxy进程所属的用户 group [groupname] ：指定HAProxy进程所属的用户组 log [address] [device] [maxlevel] [minlevel]：日志输出配置，如log 127.0.0.1 local0 info warning，即向本机rsyslog或syslog的local0输出info到warning级别的日志。其中[minlevel]可以省略。HAProxy的日志共有8个级别，从高到低为emerg/alert/crit/err/warning/notice/info/debug pidfile ：指定记录HAProxy进程号的文件绝对路径。主要用于HAProxy进程的停止和重启动作。 maxconn ：HAProxy进程同时处理的连接数，当连接数达到这一数值时，HAProxy将停止接收连接请求 frontend域的关键配置 acl [name] [criterion] [flags] [operator] [value]：定义一条ACL，ACL是根据数据包的指定属性以指定表达式计算出的true/false值。如"acl url_ms1 path_beg -i /ms1/"定义了名为url_ms1的ACL，该ACL在请求uri以/ms1/开头（忽略大小写）时为true bind [ip]:[port]：frontend服务监听的端口 default_backend [name]：frontend对应的默认backend disabled：禁用此frontend http-request [operation] [condition]：对所有到达此frontend的HTTP请求应用的策略，例如可以拒绝、要求认证、添加header、替换header、定义ACL等等。 http-response [operation] [condition]：对所有从此frontend返回的HTTP响应应用的策略，大体同上 log：同global域的log配置，仅应用于此frontend。如果要沿用global域的log配置，则此处配置为log global maxconn：同global域的maxconn，仅应用于此frontend mode：此frontend的工作模式，主要有http和tcp两种，对应L7和L4两种负载均衡模式 option forwardfor：在请求中添加X-Forwarded-For Header，记录客户端ip option http-keep-alive：以KeepAlive模式提供服务 option httpclose：与http-keep-alive对应，关闭KeepAlive模式，如果HAProxy主要提供的是接口类型的服务，可以考虑采用httpclose模式，以节省连接数资源。但如果这样做了，接口的调用端将不能使用HTTP连接池 option httplog：开启httplog，HAProxy将会以类似Apache HTTP或Nginx的格式来记录请求日志 option tcplog：开启tcplog，HAProxy将会在日志中记录数据包在传输层的更多属性 stats uri [uri]：在此frontend上开启监控页面，通过[uri]访问 stats refresh [time]：监控数据刷新周期 stats auth [user]:[password]：监控页面的认证用户名密码 timeout client [time]：指连接创建后，客户端持续不发送数据的超时时间 timeout http-request [time]：指连接创建后，客户端没能发送完整HTTP请求的超时时间，主要用于防止DoS类攻击，即创建连接后，以非常缓慢的速度发送请求包，导致HAProxy连接被长时间占用 use_backend [backend] if|unless [acl]：与ACL搭配使用，在满足/不满足ACL时转发至指定的backend backend域的关键配置 acl：同frontend域 balance [algorithm]：在此backend下所有server间的负载均衡算法，常用的有roundrobin和source，完整的算法说明见官方文档configuration.html#4.2-balance cookie：在backend server间启用基于cookie的会话保持策略，最常用的是insert方式，如cookie HA_STICKY_ms1 insert indirect nocache，指HAProxy将在响应中插入名为HA_STICKY_ms1的cookie，其值为对应的server定义中指定的值，并根据请求中此cookie的值决定转发至哪个server。indirect代表如果请求中已经带有合法的HA_STICK_ms1 cookie，则HAProxy不会在响应中再次插入此cookie，nocache则代表禁止链路上的所有网关和缓存服务器缓存带有Set-Cookie头的响应。 default-server：用于指定此backend下所有server的默认设置。具体见下面的server配置。 disabled：禁用此backend http-request/http-response：同frontend域 log：同frontend域 mode：同frontend域 option forwardfor：同frontend域 option http-keep-alive：同frontend域 option httpclose：同frontend域 option httpchk [METHOD] [URL] [VERSION]：定义以http方式进行的健康检查策略。如option httpchk GET /healthCheck.html HTTP/1.1 option httplog：同frontend域 option tcplog：同frontend域 server [name] [ip]:[port] [params]：定义backend中的一个后端server，[params]用于指定这个server的参数，常用的包括有： check：指定此参数时，HAProxy将会对此server执行健康检查，检查方法在option httpchk中配置。同时还可以在check后指定inter, rise, fall三个参数，分别代表健康检查的周期、连续几次成功认为server UP，连续几次失败认为server DOWN，默认值是inter 2000ms rise 2 fall 3 cookie [value]：用于配合基于cookie的会话保持，如cookie ms1.srv1代表交由此server处理的请求会在响应中写入值为ms1.srv1的cookie（具体的cookie名则在backend域中的cookie设置中指定） maxconn：指HAProxy最多同时向此server发起的连接数，当连接数到达maxconn后，向此server发起的新连接会进入等待队列。默认为0，即无限 maxqueue：等待队列的长度，当队列已满后，后续请求将会发至此backend下的其他server，默认为0，即无限 weight：server的权重，0-256，权重越大，分给这个server的请求就越多。weight为0的server将不会被分配任何新的连接。所有server默认weight为1 timeout connect [time]：指HAProxy尝试与backend server创建连接的超时时间 timeout check [time]：默认情况下，健康检查的连接+响应超时时间为server命令中指定的inter值，如果配置了timeout check，HAProxy会以inter作为健康检查请求的连接超时时间，并以timeout check的值作为健康检查请求的响应超时时间 timeout server [time]：指backend server响应HAProxy请求的超时时间 default域上文所属的frontend和backend域关键配置中，除acl、bind、http-request、http-response、use_backend外，其余的均可以配置在default域中。default域中配置了的项目，如果在frontend或backend域中没有配置，将会使用default域中的配置。 listen域 listen域是frontend域和backend域的组合，frontend域和backend域中所有的配置都可以配置在listen域下 ``` &emsp; ### HAProxy配置代理(7层)web服务后端两台web服务器192.168.50.251和192.168.50.252，url匹配/app1/访问192.168.50.251，url匹配/app2/访问192.168.50.252 ```bash vim /etc/haproxy/haproxy.cfg ``` ```ini global log 127.0.0.1 local0 info log 127.0.0.1 local1 warning chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 40000 user haproxy group haproxy daemon stats socket /var/lib/haproxy/stats defaults mode tcp log global option tcplog option dontlognull option http-server-close timeout connect 10s timeout client 1m timeout server 1m maxconn 3000 frontend public mode tcp bind :80 log-format %ft\ %b/%s acl url_app1 path_beg -i /app1/ acl url_app2 path_beg -i /app2/ use_backend app1 if url_app1 default_backend default_servers backend app1 mode http server web1 192.168.50.251:5555 maxconn 300 check backend app2 mode http server web2 192.168.50.252:5555 maxconn 300 check backend default_servers mode http balance roundrobin cookie SERVER insert indirect #开启会话粘滞 server web1 192.168.50.251:5555 maxconn 300 cookie server1 check server web2 192.168.50.252:5555 maxconn 300 cookie server2 check listen status bind *:7777 mode http stats enable stats refresh 10s stats uri /haproxy stats realm Haproxy\ Statistics stats auth admin:admin stats hide-version ``` 配置rsyslog日志 ```bash vim /etc/rsyslog.conf ``` ```ini local0.* /var/log/haproxy_info.log local1.* /var/log/haproxy_warning.log ``` &emsp; ### HAProxy配置复用8080 haproxy使用8080端口分别代理ssh和web,原理是通过获取请求数据包前三个字节的二进制值，匹配代理到不同的后端服务。 ```bash vim /etc/haproxy/haproxy.cfg ``` ```ini global log 127.0.0.1 local0 info log 127.0.0.1 local1 warning chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 40000 user haproxy group haproxy daemon stats socket /var/lib/haproxy/stats defaults mode tcp log global option tcplog option dontlognull timeout connect 10s timeout client 1m timeout server 1m maxconn 3000 frontend public mode tcp bind :8080 log global option tcplog log-format %ft\ %b/%s tcp-request inspect-delay 5s acl is_ssh req.payload(0,3) -m bin 535348 #数据包前三个字节,SSH acl is_http req.payload(0,3) -m bin 474554 504f53 505554 44454c 4f5054 484541 434f4e 545241 #数据包前三个字节做匹配，GET，POST，DELETE，PUT，OPTIONS，HEAD，TRACE，CONNECT tcp-request content accept if is_http tcp-request content accept if is_ssh use_backend web if is_http use_backend ssh if is_ssh backend web mode http balance roundrobin server web1 192.168.50.251:5555 check server web2 192.168.50.252:5555 check backend ssh mode tcp timeout server 3h server ssh 192.168.50.252:22 ```

```bash pip uninstall requests yum reinstall python-requests pip uninstall six yum reinstall python-six pip uninstall urllib3 yum reinstall python-urllib3 ```

```nginx proxy_intercept_errors on; server { error_page 404 /404.html; location = /404.html { root /usr/share/nginx/html; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } } ```

1.下载数据库二进制包 ```bash cd /root wget https://cdn.mysql.com//Downloads/MySQL-5.7/mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz ``` 2.解压到安装目录 ```bash cd /root tar xvf mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz mv mysql-5.7.22-linux-glibc2.12-x86_64 /opt/mysql5.7 ln -s /opt/mysql5.7 /opt/mysql ``` 3.增加mysql用户/修改数据库目录权限创建mysql组： ```bash groupadd mysql ``` 创建mysql用户： ```bash useradd -M -g mysql -s /sbin/nologin mysql ``` 4.初始化数据库(注意:记住初始化生成的密码) ```bash /opt/mysql/bin/mysqld --initialize --user=mysql --basedir=/opt/mysql/ --datadir=/home/data/mysql/data_dev/ /opt/mysql/bin/mysqld --initialize --user=mysql --basedir=/opt/mysql/ --datadir=/home/data/mysql/data_test/ ``` 5.增加数据库配置文件 ```bash cd /opt/mysql/support-files vim my_test.cnf ``` ``` [mysqld] innodb_buffer_pool_size = 128M cp my_test.cnf my_dev.cnf ``` 6.增加systemctl启动mysql脚本 ```bash cd /lib/systemd/system ``` 测试库脚本 ```bash cat mysql_test.service ``` ``` [Unit] Description=MySQL TEST ENV PORT 3306 [Service] Type=simple PIDFile=/home/data/mysql/data_test/mysqld_test.pid ExecStart=/opt/mysql/bin/mysqld --defaults-file=/opt/mysql/support-files/my_test.cnf --basedir=/opt/mysql/ --datadir=/home/data/mysql/data_test/ --user=mysql --log-error=/home/data/mysql/data_test/error_test.log --pid-file=/home/data/mysql/data_test/mysqld_test.pid --socket=/home/data/mysql/data_test/mysqld_test.sock --port=3306 [Install] WantedBy=multi-user.target ``` 研发库脚本 ```bash cat mysql_dev.service ``` ``` [Unit] Description=MySQL DEV ENV PORT 3307 [Service] Type=simple PIDFile=/home/data/mysql/data_dev/mysqld_dev.pid ExecStart=/opt/mysql/bin/mysqld --defaults-file=/opt/mysql/support-files/my_dev.cnf --basedir=/opt/mysql/ --datadir=/home/data/mysql/data_dev/ --user=mysql --log-error=/home/data/mysql/data_dev/error_dev.log --pid-file=/home/data/mysql/data_dev/mysqld_dev.pid --socket=/home/data/mysql/data_dev/mysqld_dev.sock --port=3307 [Install] WantedBy=multi-user.target ``` 7.修改数据库默认密码测试库脚本 ```bash /opt/mysql/bin/mysqladmin -P3306 -h127.0.0.1 -u root -p password ``` 研发库脚本 ```bash /opt/mysql/bin/mysqladmin -P3307 -h127.0.0.1 -u root -p password ``` 8.增加数据库用户 ```mysql mysql>grant all privileges on *.* to dev@'%' identified by '123456'; mysql>flush privileges; ``` 9.测试 ``` mysql -h127.0.0.1 -udev -p123456 ```

### 安装Keepalived ```bash yum install keepalived -y ``` &emsp; ### 使用简单的failover 参考:https://docs.oracle.com/cd/E37670_01/E41138/html/section_uxg_lzh_nr.html 典型的Keepalived高可用性配置包括一个主服务器和一个或多个备份服务器。一个或多个虚拟IP地址（定义为VRRP实例）被分配给主服务器的网络接口，以便它可以为网络客户端提供服务。备份服务器侦听主服务器定期发送的组播VRRP通告报文。默认广告时间间隔为一秒。如果备份节点无法接收三个连续的VRRP通告，则具有最高分配优先级的备份服务器将接管主服务器，并将虚拟IP地址分配给其自己的网络接口。如果多个备份服务器具有相同的优先级，则具有最高IP地址值的备份服务器将成为主服务器以下示例使用Keepalived在两台服务器上实现简单的故障转移配置。一台服务器充当主服务器，另一台服务器充当备份服务器，主服务器的优先级高于备份服务器。显示虚拟IP地址10.0.0.100最初如何分配给主服务器（10.0.0.71）。当主服务器发生故障时，备份服务器（10.0.0.72）将成为新的主服务器，并被分配虚拟IP地址10.0.0.100。 &emsp; ![](https://docs.oracle.com/cd/E37670_01/E41138/html/images/keepalived1.png) &emsp; #### master配置 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini global_defs { notification_email { root@mydomain.com } notification_email_from svr1@mydomain.com smtp_server localhost smtp_connect_timeout 30 } vrrp_instance VRRP1 { state MASTER # Specify the network interface to which the virtual address is assigned interface eth0 # The virtual router ID must be unique to each VRRP instance that you define virtual_router_id 41 # Set the value of priority higher on the master server than on a backup server priority 200 advert_int 1 authentication { auth_type PASS auth_pass 1066 } virtual_ipaddress { 10.0.0.100/24 } } ``` &emsp; #### backup配置 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini global_defs { notification_email { root@mydomain.com } notification_email_from svr2@mydomain.com smtp_server localhost smtp_connect_timeout 30 } vrrp_instance VRRP1 { state BACKUP # Specify the network interface to which the virtual address is assigned interface eth0 virtual_router_id 41 # Set the value of priority lower on the backup server than on the master server priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1066 } virtual_ipaddress { 10.0.0.100/24 } } ``` 如果主服务器（svr1）发生故障，keepalived会将虚拟IP地址10.0.0.100/24分配给备份服务器（svr2）上的eth0接口，该服务器将成为主服务器。 #### 查看master网卡分配的VIP ```bash ip addr list eth0 ``` ``` 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:cb:a6:8d brd ff:ff:ff:ff:ff:ff inet 10.0.0.72/24 brd 10.0.0.255 scope global eth0 inet 10.0.0.100/24 scope global eth0 inet6 fe80::a00:27ff:fecb:a68d/64 scope link valid_lft forever preferred_lft forever ``` #### 查看backup日志 ```bash tail -f /var/log/messages ``` ``` ...51:55 ... VRRP_Instance(VRRP1) Entering BACKUP STATE ... ...53:08 ... VRRP_Instance(VRRP1) Transition to MASTER STATE ...53:09 ... VRRP_Instance(VRRP1) Entering MASTER STATE ...53:09 ... VRRP_Instance(VRRP1) setting protocol VIPs. ...53:09 ... VRRP_Instance(VRRP1) Sending gratuitous ARPs on eth0 for 10.0.0.100 ``` &emsp; ### Keepalived负载均衡之NAT模式(双网卡场景) 以下示例在NAT模式下使用Keepalived在两台服务器上实现简单的故障转移和负载平衡配置。一台服务器充当主服务器，另一台服务器充当备份服务器，主服务器的优先级高于备份服务器。每个服务器都有两个网络接口，其中一个接口连接到面向外部网络（192.168.1.0/24）的一侧，另一个接口连接到内部网络（10.0.0.0/24），其中两个Web服务器可以访问。 &emsp; ![](https://docs.oracle.com/cd/E37670_01/E41138/html/images/keepalived2.png) &emsp; #### master配置此配置类似“使用简单的failover”，增加了vrrp_sync_group部分，以便在故障转移时将网络接口一起分配，并使用virtual_server部分定义真实的后备Keepalived用于负载平衡的终端服务器。 lb_kind的值设置为NAT（网络地址转换），这意味着Keepalived服务器代表后端服务器处理来自客户端的入站和出站网络流量。 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini global_defs { notification_email { root@mydomain.com } notification_email_from svr1@mydomain.com smtp_server localhost smtp_connect_timeout 30 } vrrp_sync_group VRRP1 { # Group the external and internal VRRP instances so they fail over together group { external internal } } vrrp_instance external { state MASTER interface eth0 virtual_router_id 91 priority 200 advert_int 1 authentication { auth_type PASS auth_pass 1215 } # Define the virtual IP address for the external network interface virtual_ipaddress { 192.168.1.1/24 } } vrrp_instance internal { state MASTER interface eth1 virtual_router_id 92 priority 200 advert_int 1 authentication { auth_type PASS auth_pass 1215 } # Define the virtual IP address for the internal network interface virtual_ipaddress { 10.0.0.100/24 } } # Define a virtual HTTP server on the virtual IP address 192.168.1.1 virtual_server 192.168.1.1 80 { delay_loop 10 protocol TCP # Use round-robin scheduling in this example lb_algo rr # Use NAT to hide the back-end servers lb_kind NAT # Persistence of client sessions times out after 2 hours persistence_timeout 7200 real_server 10.0.0.71 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } real_server 10.0.0.72 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } } ``` &emsp; #### backup配置 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini global_defs { notification_email { root@mydomain.com } notification_email_from svr2@mydomain.com smtp_server localhost smtp_connect_timeout 30 } vrrp_sync_group VRRP1 { # Group the external and internal VRRP instances so they fail over together group { external internal } } vrrp_instance external { state BACKUP interface eth0 virtual_router_id 91 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1215 } # Define the virtual IP address for the external network interface virtual_ipaddress { 192.168.1.1/24 } } vrrp_instance internal { state BACKUP interface eth1 virtual_router_id 92 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1215 } # Define the virtual IP address for the internal network interface virtual_ipaddress { 10.0.0.100/24 } } # Define a virtual HTTP server on the virtual IP address 192.168.1.1 virtual_server 192.168.1.1 80 { delay_loop 10 protocol TCP # Use round-robin scheduling in this example lb_algo rr # Use NAT to hide the back-end servers lb_kind NAT # Persistence of client sessions times out after 2 hours persistence_timeout 7200 real_server 10.0.0.71 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } real_server 10.0.0.72 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } } ``` &emsp; #### 防火墙配置如果将Keepalived配置为使用NAT模式与内部网络上的服务器进行负载平衡，则Keepalived服务器将处理所有入站和出站网络流量，并通过重写真实服务器的源IP地址来隐藏后端服务器，使用外部网络接口的虚拟IP地址在传出数据包。开启IP伪装（master/backup） ```bash iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE service iptables save ``` 允许网卡之间的转发(master/backup) ```bash iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited service iptables save ``` 开放端口（以HTTP为例） ```bash iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT service iptables save ``` &emsp; #### 创建VIP的路由(后端服务器) ```bash ip route add default via 10.0.0.100 dev eth0 ip route show ``` ``` default via 10.0.0.100 dev eth0 10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.71 ``` 永久生效 ```bash echo "default via 10.0.0.100 dev eth0" > /etc/sysconfig/network-scripts/route-eth0 ``` &emsp; ### Keepalived负载均衡之NAT模式(单网卡场景) `注意:不能在同一网段直接访问虚拟IP,因为真实服务针对客户端发来的数据包，应答的时候不会经过keepalived(数据包不需要路由)，改不了数据包，所以访问不了。如果keepalived和真实主机处于同一内网推荐使用DR模式,代理性能更高,此处仅为实验，为了更深入理解。` 此配置类似“Keepalived负载均衡之NAT模式(双网卡场景)” ![](https://files.ynotes.cn/keepalived_single_network.png) &emsp; #### master配置 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini ! Configuration File for keepalived global_defs { notification_email { test01@ynotes.cn } notification_email_from haproxy1@ynotes.cn smtp_server localhost smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr #vrrp_strict #严格执行VRRP协议规范，此模式不支持节点单播,注销该参数 vrrp_garp_interval 0 vrrp_gna_interval 0 } vrrp_instance VI_1 { state MASTER interface enp0s3 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.50.240 } } virtual_server 192.168.50.240 5555 { delay_loop 6 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP real_server 192.168.50.251 5555 { weight 1 TCP_CHECK { connect_port 5555 connect_timeout 3 } } real_server 192.168.50.252 5555 { weight 1 TCP_CHECK { connect_port 5555 connect_timeout 3 } } } ``` &emsp; #### backup配置 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini ! Configuration File for keepalived global_defs { notification_email { test01@ynotes.cn } notification_email_from haproxy2@ynotes.cn smtp_server localhost smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr #vrrp_strict #严格执行VRRP协议规范，此模式不支持节点单播,注销该参数 vrrp_garp_interval 0 vrrp_gna_interval 0 } vrrp_instance VI_1 { state BACKUP interface enp0s3 virtual_router_id 51 priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.50.240 } } virtual_server 192.168.50.240 5555 { delay_loop 6 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP real_server 192.168.50.251 5555 { weight 1 TCP_CHECK { connect_port 5555 connect_timeout 3 } } real_server 192.168.50.252 5555 { weight 1 TCP_CHECK { connect_port 5555 connect_timeout 3 } } } ``` &emsp; #### 开启端口转发（master/backup） ```bash echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf sysctl -p ``` ``` net.ipv4.ip_forward = 1 ``` &emsp; #### 防火墙配置对后端服务器的流量进行转发开启SNAT（master/backup） ```bash iptables -t nat -A POSTROUTING -o enp0s3 -s 192.168.50.251/32 -j SNAT --to-source 192.168.50.240 #对源地址为192.168.50.251的数据包替换源地址为192.168.50.240 iptables -t nat -A POSTROUTING -o enp0s3 -s 192.168.50.252/32 -j SNAT --to-source 192.168.50.240 service iptables save #永久保存配置 ``` &emsp; #### 修改默认网关(后端服务器) ```bash vim /etc/sysconfig/network-scripts/ifcfg-enp0s3 #修改默认网关 ``` ```bash GATEWAY0=192.168.50.240 #这里配置成VIP地址 ``` ```bash systemctl restart network #重启网卡 ip route #查看路由信息 ``` ``` default via 192.168.50.240 dev enp0s3 proto static metric 100 172.10.0.0/24 dev br-975d989973b4 proto kernel scope link src 172.10.0.1 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 172.18.0.0/16 dev br-b64efd3fb71e proto kernel scope link src 172.18.0.1 172.20.0.0/16 dev docker_gwbridge proto kernel scope link src 172.20.0.1 192.168.50.0/24 dev enp0s3 proto kernel scope link src 192.168.50.252 metric 100 192.168.100.0/24 dev br-ea58d9f6ef1f proto kernel scope link src 192.168.100.1 ``` &emsp; ### Keepalived负载均衡之DR模式以下示例在直接路由（DR）模式下使用Keepalived在两台服务器上实现简单的故障转移和负载平衡配置。一台服务器充当主服务器，另一台服务器充当备份服务器，主服务器的优先级高于备份服务器。每个Keepalived服务器都有一个网络接口，服务器连接到同一网段（10.0.0.0/24），可以访问两个Web服务器。下图显示Keepalived主服务器具有网络地址10.0.0.11和10.0.0.1（虚拟）。 Keepalived备份服务器的网络地址为10.0.0.12。 Web服务器websvr1和websvr2分别具有网络地址10.0.0.71和10.0.0.72。此外，两个Web服务器都配置了虚拟IP地址10.0.0.1，以使它们接受具有该目标地址的数据包。主服务器接收传入请求并重定向到Web服务器，Web服务器直接响应。 &emsp; ![](https://docs.oracle.com/cd/E37670_01/E41138/html/images/keepalived3.png) #### master配置类似“Keepalived负载均衡之NAT模式”的配置，但lb_kind的值设置为DR（直接路由），这意味着Keepalived服务器处理来自的所有入站网络包，出站网络包由后端服务器直接回复客户端，绕过Keepalived服务器。此配置减少了Keepalived服务器上的负载，但安全性较低，因为每个后端服务器都需要外部访问。某些实现使用额外的网络接口，每个Web服务器都有一个专用网关来处理响应网络包。 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini global_defs { notification_email { root@mydomain.com } notification_email_from svr1@mydomain.com smtp_server localhost smtp_connect_timeout 30 } vrrp_instance external { state MASTER interface eth0 virtual_router_id 91 priority 200 advert_int 1 authentication { auth_type PASS auth_pass 1215 } virtual_ipaddress { 10.0.0.1/24 } } virtual_server 10.0.0.1 80 { delay_loop 10 protocol TCP lb_algo rr # Use direct routing lb_kind DR persistence_timeout 7200 real_server 10.0.0.71 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } real_server 10.0.0.72 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } } ``` &emsp; #### backup配置 ```bash vim /etc/keepalived/keepalived.conf ``` ```ini global_defs { notification_email { root@mydomain.com } notification_email_from svr2@mydomain.com smtp_server localhost smtp_connect_timeout 30 } vrrp_instance external { state BACKUP interface eth0 virtual_router_id 91 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1215 } virtual_ipaddress { 10.0.0.1/24 } } virtual_server 10.0.0.1 80 { delay_loop 10 protocol TCP lb_algo rr # Use direct routing lb_kind DR persistence_timeout 7200 real_server 10.0.0.71 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } real_server 10.0.0.72 80 { weight 1 TCP_CHECK { connect_timeout 5 connect_port 80 } } } ``` &emsp; #### 防火墙配置开启服务端口，以http服务为例(master/backup) ```bash iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT service iptables save ``` &emsp; #### 后端服务器(RS)配置 ```bash echo "net.ipv4.conf.lo.arp_ignore = 1" >> /etc/sysctl.conf #只响应目的IP地址为接收网卡上的本地地址的arp请求 echo "net.ipv4.conf.lo.arp_announce = 2" >> /etc/sysctl.conf echo "net.ipv4.conf.all.arp_ignore = 1" >> /etc/sysctl.conf echo "net.ipv4.conf.all.arp_announce = 2" >> /etc/sysctl.conf sysctl -p ``` ``` net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 ``` ```bash echo "ifconfig lo:0 10.0.0.1 broadcast 10.0.0.255 netmask 255.255.255.255 up" >> /etc/rc.local ifconfig lo:0 10.0.0.1 broadcast 10.0.0.255 netmask 255.255.255.255 up ip addr show lo ``` ``` 2: lo: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:cb:a6:8d brd ff:ff:ff:ff:ff:ff inet 10.0.0.72/24 brd 10.0.0.255 scope global eth0 inet 10.0.0.1/24 brd 10.0.0.255 scope global secondary eth0 inet6 fe80::a00:27ff:fecb:a68d/64 scope link valid_lft forever preferred_lft forever ```