基于Keepalived+Haproxy搭建四层负载均衡器

r囧r小猫 2022-08-07 16:44 474阅读 0赞

一、前言  
 Haproxy是稳定、高性能、高可用性的负载均衡解决方案，支持HTTP及TCP代理后端服务器池，因支持强大灵活的7层acl规则，广泛作为HTTP反向代理。本文则详细介绍如何利用它的四层交换与Keepalived实现一个负载均衡器，适用于Socket、ICE、Mail、Mysql、私有通讯等任意TCP服务。系统架构图如下：  
  
[![点击在新窗口中浏览此图片][attachment.php_fid_288]][attachment.php_fid_288]   
  
二、平台环境

引用

OS:Centos5.4(64X)  
MASTER:192.168.0.20  
BACKUP:192.168.0.21  
VIP:192.168.0.100  
Serivce Port:11231

三、平台安装配置  
1、添加非本机IP邦定支持

引用

\#vi /etc/sysctl.conf  
net.ipv4.ip\_nonlocal\_bind=1  
\#sysctl –p

2、配置平台日志支持

引用

\#vi /etc/syslog.conf  
添加：  
local3.\* /var/log/haproxy.log  
local0.\* /var/log/haproxy.log  
  
\#vi /etc/sysconfig/syslog  
修改：  
SYSLOGD\_OPTIONS="-r -m 0"  
\#/etc/init.d/syslog restart

3、关闭SELINUX

引用

vi /etc/sysconfig/selinux  
修改：  
SELINUX=disabled  
\#setenforce 0

4、配置iptables，添加VRRP通讯支持

引用

iptables -A INPUT -d 224.0.0.18 -j ACCEPT

5、Keepalived的安装、配置

引用

\#mkdir -p /home/install/keepalivedha  
\#cd /home/install/keepalivedha  
\#wget http://www.keepalived.org/software/keepalived-1.2.2.tar.gz  
\#tar zxvf keepalived-1.2.2.tar.gz  
\#cd keepalived-1.2.2  
\#./configure  
\#make && make install

引用

\#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/  
\#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/  
\#mkdir /etc/keepalived  
\#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/  
\#cp /usr/local/sbin/keepalived /usr/sbin/

\#vi /etc/keepalived/keepalived.conf

[view plain][] [print][view plain] [?][view plain]

1.  ! Configuration File **for** keepalived 
2.  
3.  global\_defs \{ 
4.   notification\_email \{ 
5.   liutiansi@gmail.com 
6.   \} 
7.   notification\_email\_from liutiansi@gmail.com 
8.   smtp\_connect\_timeout 3 
9.   smtp\_server 127.0.0.1 
10.  router\_id LVS\_DEVEL 
11. \} 
12. vrrp\_script chk\_haproxy \{ 
13.  script "killall -0 haproxy"
14.  interval 2 
15.  weight 2 
16. \} 
17. vrrp\_instance VI\_1 \{ 
18.  interface eth1 
19.  state MASTER \# 从为BACKUP 
20.  priority 101 \# 从为100 
21.  virtual\_router\_id 50 \#路由ID，可通过\#tcpdump vrrp查看。 
22.  garp\_master\_delay 1 \#主从切换时间，单位为秒。 
23. 
24.  authentication \{ 
25.  auth\_type PASS 
26.  auth\_pass KJj23576hYgu23IP 
27.  \} 
28.  track\_interface \{ 
29.  eth0 
30.  eth1 
31.  \} 
32.  virtual\_ipaddress \{ 
33.  192.168.0.100 
34.  \} 
35.  track\_script \{ 
36.  chk\_haproxy 
37.  \} 
38. 
39.  \#状态通知 
40.  notify\_master "/etc/keepalived/Mailnotify.py master"
41.  notify\_backup "/etc/keepalived/Mailnotify.py backup"
42.  notify\_fault "/etc/keepalived/Mailnotify.py fault"
43. \}

6、Haproxy的安装与配置

引用

\#cd /home/install/keepalivedha  
\#wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.11.tar.gz  
\#tar -zxvf haproxy-1.4.11.tar.gz  
\#cd haproxy-1.4.11  
\#make install  
\#mkdir -p /usr/local/haproxy/etc  
\#mkdir -p /usr/local/haproxy/sbin  
\#cp examples/haproxy.cfg /usr/local/haproxy/etc  
\#ln -s /usr/local/sbin/haproxy /usr/local/haproxy/sbin/haproxy

\#vi /usr/local/haproxy/etc/haproxy.cfg

[view plain][] [print][view plain] [?][view plain]

1.  **\# this config needs haproxy-1.1.28 or haproxy-1.2.1**
2.  
3.  global 
4.  **\# log 127.0.0.1 local0**
5.   log 127.0.0.1 local1 notice 
6.   maxconn 5000 
7.   uid 99 
8.   gid 99 
9.   daemon 
10.  pidfile /usr/local/haproxy/haproxy.pid 
11. 
12. 
13. defaults 
14.  log global 
15.  mode http 
16.  \#option httplog 
17.  option dontlognull 
18.  retries 3 
19.  option redispatch 
20.  maxconn 2000 
21.  contimeout 5000 
22.  clitimeout 50000 
23.  srvtimeout 50000 
24. 
25. listen ICE01 192.168.0.100:11231 
26.  mode tcp \#配置TCP模式 
27.  maxconn 2000 
28.  balance roundrobin 
29.  server ice-192.168.0.128 192.168.0.128:11231 check inter 5000 fall 1 rise 2 
30.  server ice-192.168.0.129 192.168.0.129:11231 check inter 5000 fall 1 rise 2 
31.  server ice-192.168.0.130 192.168.0.130:11231 check inter 5000 fall 1 rise 2 
32.  server ice-192.168.0.131 192.168.0.131:11231 check inter 5000 fall 1 rise 2 
33.  server ice-192.168.0.132 192.168.0.132:11231 check inter 5000 fall 1 rise 2 
34.  server ice-192.168.0.34 192.168.0.34:11231 check inter 5000 fall 1 rise 2 
35.  srvtimeout 20000 
36. 
37. listen stats\_auth 192.168.0.20:80 
38. **\# listen stats\_auth 192.168.0.21:80 \# backup config**
39.  stats enable 
40.  stats uri /admin-status \#管理地址 
41.  stats auth admin:123456 \#管理帐号:管理密码 
42.  stats admin **if** TRUE

7、邮件通知程序(python实现)  
\#vi /etc/keepalived/Mailnotify.py

[view plain][] [print][view plain] [?][view plain]

1.  \#!/usr/local/bin/python
2.  \#coding: utf-8
3.  **from** *email*.MIMEMultipart **import** MIMEMultipart 
4.  **from** *email*.MIMEText **import** MIMEText 
5.  **from** *email*.MIMEImage **import** MIMEImage 
6.  **from** *email*.header **import** Header 
7.  **import** *sys*
8.  **import** *smtplib*
9.  
10. \#---------------------------------------------------------------
11. \# Name: Mailnotify.py
12. \# Purpose: Mail notify to SA
13. \# Author: Liutiansi
14. \# Email: liutiansi@gamil.com
15. \# Created: 2011/03/09
16. \# Copyright: (c) 2011
17. \#--------------------------------------------------------------
18. strFrom = 'admin@domain.com' 
19. strTo = 'liutiansi@gmail.com' 
20. smtp\_server='smtp.domain.com' 
21. smtp\_pass='123456' 
22. 
23. **if** *sys*.argv\[1\]!="master" **and** *sys*.argv\[1\]!="backup" **and** *sys*.argv\[1\]!="fault": 
24. *sys*.exit() 
25. **else**: 
26.  notify\_type=*sys*.argv\[1\] 
27. 
28. 
29. mail\_title='\[紧急\]负载均衡器邮件通知' 
30. mail\_body\_plain=notify\_type+'被激活，请做好应急处理。' 
31. mail\_body\_html='<b><font color=red>'+notify\_type+'被激活，请做好应急处理。</font></b>' 
32. 
33. msgRoot = MIMEMultipart('related') 
34. msgRoot\['Subject'\] =Header(mail\_title,'utf-8') 
35. msgRoot\['From'\] = strFrom 
36. msgRoot\['To'\] = strTo 
37. 
38. msgAlternative = MIMEMultipart('alternative') 
39. msgRoot.attach(msgAlternative) 
40. 
41. msgText = MIMEText(mail\_body\_plain, 'plain', 'utf-8') 
42. msgAlternative.attach(msgText) 
43. 
44. 
45. msgText = MIMEText(mail\_body\_html, 'html','utf-8') 
46. msgAlternative.attach(msgText) 
47. 
48. 
49. smtp = *smtplib*.SMTP() 
50. smtp.connect(smtp\_server) 
51. smtp.login(smtp\_user,smtp\_pass) 
52. smtp.sendmail(strFrom, strTo, msgRoot.as\_string()) 
53. smtp.quit()

注：修改成系统python实际路径“\#!/usr/local/bin/python”(第一行)  
\#chmod +x /etc/keepalived/Mailnotify.py  
\#/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/etc/haproxy.cfg  
\#service keepalived start  
  
8、查看VRRP通讯记录  
\#tcpdump vrrp

引用

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode  
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes  
15:49:05.270017 IP 192.168.0.20 > VRRP.MCAST.NET: VRRPv2, Advertisement, vrid 50, prio 100, authtype simple, intvl 1s, length 20

四、Haproxy界面  
访问http://192.168.0.20/admin-status，输入帐号admin密码123456进入管理监控平台。  
[![点击在新窗口中浏览此图片][attachment.php_fid_290]][attachment.php_fid_290]  
haproxy-1.4.9以后版本最大的亮点是添加了手工启用/禁用功能，对升级变更应用时非常有用。  
  
五、邮件通知  
[![点击在新窗口中浏览此图片][attachment.php_fid_291]][attachment.php_fid_291]

[attachment.php_fid_288]: /images/20220805/f0710ecf3e1b434b88aecfa5ecf3df39.png
[view plain]: http://blog.liuts.com/#
[attachment.php_fid_290]: /images/20220805/17baca3d35da415c90520bd6fd5750ef.png
[attachment.php_fid_291]: /images/20220805/8612d72a7c724454a512ada890c986a8.png