折雨的天空

新浪微薄腾讯微薄

最新碎语:最近霉到了住。哎。。。

您的位置:折雨的天空 >其他技术> 用nginx和keepalived配置了一个简单的负载均衡

用nginx和keepalived配置了一个简单的负载均衡

参考文章:


http://www.cnblogs.com/holbrook/archive/2012/10/25/2738475.html


http://blog.zhuyin.org/866.html


2 安装

两台接入服务器分别安装NginX和keepalived:

  • 准备依赖包:
yum -y install gcc pcre-devel zlib-devel openssl-devel
  • 下载
wget http://nginx.org/download/nginx-1.2.4.tar.gz  wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz
  • 安装NginX
复制代码
tar zxvf nginx-1.2.4.tar.gz

cd nginx-1.2.4 ./configure --with-http_stub_status_module make && make install
复制代码

 

  • 安装keepalived
复制代码
tar zxvf keepalived-1.2.7.tar.gz
cd keepalived-1.2.7 ./configure make make install cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/ cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ mkdir /etc/keepalived cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/ cp /usr/local/sbin/keepalived /usr/sbin/
复制代码

 

  • 加入启动服务

echo "/usr/local/nginx/sbin/nginx" >> /etc/rc.local
echo "/etc/init.d/keepalived start" >> /etc/rc.local

 

3 配置

 

3.1 配置NginX

两台接入服务器的NginX的配置完全一样,主要是配置/usr/local/nginx/conf/nginx.conf的http。其中多域名指向是通过虚拟主机(配置http下面的server)实现;同一域名的不同虚拟目录通过每个server下面的不同location实现;到后端的服务器在http下面配置upstream,然后在server或location中通过proxypass引用。要实现前面规划的接入方式,http的配置如下:

复制代码
http {
    include       mime.types; default_type  application/octet-stream; sendfile on; upstream dev.hysec.com {
        server 50.1.1.21:80; }


    upstream www.hysec.com {
      ip_hash; server 50.1.1.10:80; server 50.1.1.11:80; server 50.1.1.12:80; }

    upstream oa.hysec.com {
      ip_hash; server 50.1.1.13:8080; server 50.1.1.14:8080; server {
        listen 80; server_name dev.hysec.com; location /svn {
            proxy_pass http://dev.hysec.com; }

        location /submin {
            proxy_pass http://dev.hysec.com; }
    }

    server {
        listen 80; server_name  www.hysec.com; location / {
            proxy_pass http://www.hysec.com; }
    server {
        listen 80; server_name  oa.hysec.com; location / {
            proxy_pass http://oa.hysec.com; }
}
复制代码

 

验证方法:

  • 首先用IP访问前表中各个应用服务器的url
  • 再用域名和路径访问前表中各个应用系统的域名/虚拟路径

3.2 配置keepalived

按照上面的安装方法,keepalived的配置文件在/etc/keepalived/keepalived.conf。主、从服务器的配置相关联但有所不同。如下:

Master:

复制代码
! Configuration File for keepalived

global_defs {
notification_email {
        wanghaikuo@hysec.com
        wanghaikuo@gmail.com
   }

   notification_email_from wanghaikuo@hysec.com
   smtp_server smtp.hysec.com
   smtp_connect_timeout 30 router_id nginx_master

}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51 priority 101 advert_int 1 authentication {
        auth_type PASS
        auth_pass 1111 }
    virtual_ipaddress { 50.1.1.2 }
}
复制代码

Backup:

复制代码
! Configuration File for keepalived

global_defs {
notification_email {
        wanghaikuo@hysec.com
        wanghaikuo@gmail.com
   }

   notification_email_from wanghaikuo@hysec.com
   smtp_server smtp.hysec.com
   smtp_connect_timeout 30 router_id nginx_backup

} vrrp_instance VI_1 {
    state BACKUP interface eth0
    virtual_router_id 51 priority 99 advert_int 1 authentication {
        auth_type PASS
        auth_pass 1111 }
    virtual_ipaddress { 50.1.1.2 }
}
复制代码

验证:

  • 先后在主、从服务器上启动keepalived: /etc/init.d/keepalived start
  • 在主服务器上查看是否已经绑定了虚拟IP: ip addr
  • 停止主服务器上的keepalived: /etc/init.d/keepalived stop 然后在从服务器上查看是否已经绑定了虚拟IP:
  • 启动主服务器上的keepalived,看看主服务器能否重新接管虚拟IP

3.3 让keepalived监控NginX的状态

经过前面的配置,如果主服务器的keepalived停止服务,从服务器会自动接管VIP对外服务;一旦主服务器的keepalived恢复,会重新接管VIP。 但这并不是我们需要的,我们需要的是当NginX停止服务的时候能够自动切换。

keepalived支持配置监控脚本,我们可以通过脚本监控NginX的状态,如果状态不正常则进行一系列的操作,最终仍不能恢复NginX则杀掉keepalived,使得从服务器能够接管服务。

  • 如何监控NginX的状态

最简单的做法是监控NginX进程,更靠谱的做法是检查NginX端口,最靠谱的做法是检查多个url能否获取到页面。

  • 如何尝试恢复服务

如果发现NginX不正常,重启之。等待3秒再次校验,仍然失败则不再尝试。

根据上述策略很容易写出监控脚本。这里使用nmap检查nginx端口来判断nginx的状态,记得要首先安装nmap。监控脚本如下:

按 Ctrl+C 复制代码
按 Ctrl+C 复制代码

 

不要忘了设置脚本的执行权限,否则不起作用。

假设上述脚本放在/opt/chk_nginx.sh,则keepalived.conf中增加如下配置:

复制代码
vrrp_script chk_http_port {
    script "/opt/chk_nginx.sh" interval 2 weight 2 }

track_script {
    chk_http_port
}
复制代码

 

更进一步,为了避免启动keepalived之前没有启动nginx , 可以在/etc/init.d/keepalived的start中首先启动nginx:

复制代码
start() { /usr/local/nginx/sbin/nginx
    sleep 3 echo -n $"Starting $prog: " daemon keepalived ${KEEPALIVED_OPTIONS}
    RETVAL=$? echo [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
}
复制代码

 


主nginx负载均衡器:172.26.11.99  (通过keepalived配置了VIP:172.26.11.101供外使用)
副nginx负载均衡器:172.26.11.100 (通过keepalived配置了VIP:172.26.11.101供外使用)

后端web服务器:
172.26.11.73
172.26.11.74

一、172.26.11.99 以及 172.26.11.100的关键nginx配置如下:
vim /etc/nginx/nginx.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#################
....
upstream  www.xxx.com  {
server   172.26.11.73:8080 max_fails=1;#max_fails 表示健康检查失败的次数,这里表示次数为一次,即标记该服务器down了
server   172.26.11.74:8080 max_fails=1;
}
 
server
{
listen  80;
server_name  www.xxx.com;
 
location / {
proxy_next_upstream error timeout http_500 http_502 http_504;  #这里表示健康检查涉及到的情形,有这些情形的,都切换到另外的web服务器访问
proxy_read_timeout 10s;   #这里表示程序返回的时间,请参考php.ini的max_exe_time来设置。
proxy_pass        http://www.xxx.com;
proxy_set_header   Host             $host;
proxy_set_header   X-Real-IP        $remote_addr;
proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
 
}
 
#access_log  /var/log/nginx/xxx.log;
}
 
##########################

二、安装keepalive (centos)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#安装 popt
yum -y install popt popt-devel
 
cd /data/software
wget http://www.keepalived.org/software/keepalived-1.2.8.tar.gz
cd /data/src
tar zxf ../software/keepalived-1.2.8.tar.gz
cd keepalived-1.2.8
./configure --prefix=/usr/local/keepalived --sysconf=/etc
make && make install
 
cp /usr/local/keepalived/sbin/keepalived  /bin/
chkconfig --add keepalived
#设置开机启动
chkconfig keepalived on
#启动keepalive服务
/etc/init.d/keepalived start

如果是ubuntu 直接 apt-get install keepalived 吧….

三、keepalive设置
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf_bak

MASTER
vim /etc/keepalived/keepalived.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
! Configuration File for keepalived
global_defs {
    notification_email {
        admin@test.com
    }
    notification_email_from admin@test.com
    smtp_server xxx.smtp.com
    smtp_connect_timeout 30
    router_id LVS_DEVEL
}
vrrp_script Monitor_Nginx {
     script "/root/monitor_nginx.sh"
     interval 2
     weight 2
}
vrrp_instance VI_1 {
    state MASTER    #(主机为MASTER,备用机为BACKUP)
    interface eth0  #(HA监测网络接口)
 
    virtual_router_id 61 #(主、备机的virtual_router_id必须相同)
    #mcast_src_ip 172.26.11.99 #(多播的源IP,设置为本机外网IP,与VIP同一网卡)此项可不设置
    priority 90 #(主、备机取不同的优先级,主机值较大,备份机值较小,值越大优先级越高)
    advert_int 1 #(VRRP Multicast广播周期秒数)
    authentication {
        auth_type PASS #(VRRP认证方式)
        auth_pass 1234 #(密码)
    }
 
    track_script {
        Monitor_Nginx #(调用nginx进程检测脚本)
    }
    virtual_ipaddress {
        172.26.11.101 #(VRRP HA虚拟地址)
    }
}

BACKUP方面只需要修改state为BACKUP , priority比MASTER稍低即可

四、监控nginx进程的脚本:monitor_nginx.sh 内容如下:
vim /root/monitor_nginx.sh
当检测到nginx进程不存在的时候,就干掉所有的keepalived,这时候,请求将会由keepalived的backup接管!!

1
2
3
4
5
#!/bin/bash
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]
 then
 killall keepalived
fi

chmod +x /root/monitor_nginx.sh

172.26.11.99 172.26.11.100都重新启动keepalived:
service keepalived restart

这里请注意,当keepalived启动后,我们可以用命令:

ip add show eth0 来看我们的eth0网卡确实被添加了虚拟IP,如下图:

22222222222 nginx+keepalive 实现高可用负载均衡方案

完毕,可以测试了!

---

转载请注明本文标题和链接:《用nginx和keepalived配置了一个简单的负载均衡

分享到:

发表评论

路人甲 表情
看不清楚?点图切换