用nginx和keepalived配置了一个简单的负载均衡

参考文章：http://www.cnblogs.com/holbrook/archive/2012/10/25/2738475.htmlhttp://blog.zhuyin.org/866.html2 安装两台接入服务器分别安装NginX和keepalived: 准备依赖包： yum -y install gcc pcre-devel zlib-devel ...

参考文章：

http://www.cnblogs.com/holbrook/archive/2012/10/25/2738475.html

http://blog.zhuyin.org/866.html

2 安装

两台接入服务器分别安装NginX和keepalived:

准备依赖包：

yum -y install gcc pcre-devel zlib-devel openssl-devel

下载

wget http://nginx.org/download/nginx-1.2.4.tar.gz wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

安装NginX

tar zxvf nginx-1.2.4.tar.gz



cd nginx-1.2.4./configure --with-http_stub_status_modulemake && make install

安装keepalived

tar zxvf keepalived-1.2.7.tar.gz

cd keepalived-1.2.7./configuremake make install cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/ cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ mkdir /etc/keepalivedcp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/ cp /usr/local/sbin/keepalived /usr/sbin/

加入启动服务

echo "/usr/local/nginx/sbin/nginx" >> /etc/rc.local

echo "/etc/init.d/keepalived start" >> /etc/rc.local

3 配置

3.1 配置NginX

两台接入服务器的NginX的配置完全一样,主要是配置/usr/local/nginx/conf/nginx.conf的http。其中多域名指向是通过虚拟主机（配置http下面的server）实现；同一域名的不同虚拟目录通过每个server下面的不同location实现；到后端的服务器在http下面配置upstream,然后在server或location中通过proxy_{pass引用。要实现前面规划的接入方式，http的配置如下：}

http {

    include       mime.types;default_type  application/octet-stream;sendfileon;upstream dev.hysec.com {

        server50.1.1.21:80;}





    upstream www.hysec.com {

      ip_hash;server50.1.1.10:80;server50.1.1.11:80;server50.1.1.12:80;}



    upstream oa.hysec.com {

      ip_hash;server50.1.1.13:8080;server50.1.1.14:8080;server {

        listen80;server_name dev.hysec.com;location/svn {

            proxy_pass http://dev.hysec.com;}



        location/submin {

            proxy_pass http://dev.hysec.com;}

    }



    server {

        listen80;server_name  www.hysec.com;location/{

            proxy_pass http://www.hysec.com;}

    server {

        listen80;server_name  oa.hysec.com;location/{

            proxy_pass http://oa.hysec.com;}

}

验证方法：

首先用IP访问前表中各个应用服务器的url
再用域名和路径访问前表中各个应用系统的域名/虚拟路径

3.2 配置keepalived

按照上面的安装方法，keepalived的配置文件在/etc/keepalived/keepalived.conf。主、从服务器的配置相关联但有所不同。如下：

Master:

! Configuration File forkeepalived



global_defs {

notification_email {

        wanghaikuo@hysec.com

        wanghaikuo@gmail.com

   }



   notification_email_from wanghaikuo@hysec.com

   smtp_server smtp.hysec.com

   smtp_connect_timeout30router_id nginx_master



}



vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id51 priority 101advert_int1authentication {

        auth_type PASS

        auth_pass1111}

    virtual_ipaddress {50.1.1.2}

}

Backup:

! Configuration File forkeepalived



global_defs {

notification_email {

        wanghaikuo@hysec.com

        wanghaikuo@gmail.com

   }



   notification_email_from wanghaikuo@hysec.com

   smtp_server smtp.hysec.com

   smtp_connect_timeout30router_id nginx_backup



}vrrp_instance VI_1 {

    stateBACKUPinterface eth0

    virtual_router_id51 priority 99advert_int1authentication {

        auth_type PASS

        auth_pass1111}

    virtual_ipaddress {50.1.1.2}

}

验证：

先后在主、从服务器上启动keepalived: /etc/init.d/keepalived start
在主服务器上查看是否已经绑定了虚拟IP： ip addr
停止主服务器上的keepalived: /etc/init.d/keepalived stop 然后在从服务器上查看是否已经绑定了虚拟IP：
启动主服务器上的keepalived，看看主服务器能否重新接管虚拟IP

3.3 让keepalived监控NginX的状态

经过前面的配置，如果主服务器的keepalived停止服务，从服务器会自动接管VIP对外服务；一旦主服务器的keepalived恢复，会重新接管VIP。但这并不是我们需要的，我们需要的是当NginX停止服务的时候能够自动切换。

keepalived支持配置监控脚本，我们可以通过脚本监控NginX的状态，如果状态不正常则进行一系列的操作，最终仍不能恢复NginX则杀掉keepalived，使得从服务器能够接管服务。

如何监控NginX的状态

最简单的做法是监控NginX进程，更靠谱的做法是检查NginX端口，最靠谱的做法是检查多个url能否获取到页面。

如何尝试恢复服务

如果发现NginX不正常，重启之。等待3秒再次校验，仍然失败则不再尝试。

根据上述策略很容易写出监控脚本。这里使用nmap检查nginx端口来判断nginx的状态，记得要首先安装nmap。监控脚本如下:

按 Ctrl+C 复制代码
按 Ctrl+C 复制代码

不要忘了设置脚本的执行权限，否则不起作用。

假设上述脚本放在/opt/chk_nginx.sh，则keepalived.conf中增加如下配置：

vrrp_script chk_http_port {

    script"/opt/chk_nginx.sh"interval2weight2}



track_script {

    chk_http_port

}

更进一步，为了避免启动keepalived之前没有启动nginx , 可以在/etc/init.d/keepalived的start中首先启动nginx:

start(){/usr/local/nginx/sbin/nginx

    sleep3 echo -n $"Starting $prog: "daemon keepalived${KEEPALIVED_OPTIONS}

    RETVAL=$?echo[$RETVAL -eq 0 ] && touch /var/lock/subsys/$prog

}

主nginx负载均衡器：172.26.11.99 （通过keepalived配置了VIP：172.26.11.101供外使用）

副nginx负载均衡器：172.26.11.100 （通过keepalived配置了VIP：172.26.11.101供外使用）

后端web服务器：

172.26.11.73

172.26.11.74

一、172.26.11.99 以及 172.26.11.100的关键nginx配置如下：

vim /etc/nginx/nginx.conf

				
						
						#################
....
upstream  www.xxx.com  {
server   172.26.11.73:8080 max_fails=1;#max_fails 表示健康检查失败的次数，这里表示次数为一次，即标记该服务器down了
server   172.26.11.74:8080 max_fails=1;
}
 
server
{
listen  80;
server_name  www.xxx.com;
 
location / {
proxy_next_upstream error timeout http_500 http_502 http_504;  #这里表示健康检查涉及到的情形，有这些情形的，都切换到另外的web服务器访问
proxy_read_timeout 10s;   #这里表示程序返回的时间，请参考php.ini的max_exe_time来设置。
proxy_pass        http://www.xxx.com;
proxy_set_header   Host             $host;
proxy_set_header   X-Real-IP        $remote_addr;
proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
 
}
 
#access_log  /var/log/nginx/xxx.log;
}
 
##########################

					

			

二、安装keepalive (centos)

				
						
						#安装 popt
yum -y install popt popt-devel
 
cd /data/software
wget http://www.keepalived.org/software/keepalived-1.2.8.tar.gz
cd /data/src
tar zxf ../software/keepalived-1.2.8.tar.gz
cd keepalived-1.2.8
./configure --prefix=/usr/local/keepalived --sysconf=/etc
make && make install
 
cp /usr/local/keepalived/sbin/keepalived  /bin/
chkconfig --add keepalived
#设置开机启动
chkconfig keepalived on
#启动keepalive服务
/etc/init.d/keepalived start

					

			

如果是ubuntu 直接 apt-get install keepalived 吧….

三、keepalive设置

cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf_bak

MASTER

vim /etc/keepalived/keepalived.conf

				
						
						! Configuration File for keepalived
global_defs {
    notification_email {
        admin@test.com
    }
    notification_email_from admin@test.com
    smtp_server xxx.smtp.com
    smtp_connect_timeout 30
    router_id LVS_DEVEL
}
vrrp_script Monitor_Nginx {
     script "/root/monitor_nginx.sh"
     interval 2
     weight 2
}
vrrp_instance VI_1 {
    state MASTER    #(主机为MASTER，备用机为BACKUP)
    interface eth0  #(HA监测网络接口)
 
    virtual_router_id 61 #(主、备机的virtual_router_id必须相同)
    #mcast_src_ip 172.26.11.99 #(多播的源IP，设置为本机外网IP，与VIP同一网卡)此项可不设置
    priority 90 #(主、备机取不同的优先级，主机值较大，备份机值较小,值越大优先级越高)
    advert_int 1 #(VRRP Multicast广播周期秒数)
    authentication {
        auth_type PASS #(VRRP认证方式)
        auth_pass 1234 #(密码)
    }
 
    track_script {
        Monitor_Nginx #(调用nginx进程检测脚本)
    }
    virtual_ipaddress {
        172.26.11.101 #(VRRP HA虚拟地址)
    }
}

					

			

BACKUP方面只需要修改state为BACKUP , priority比MASTER稍低即可

四、监控nginx进程的脚本：monitor_nginx.sh 内容如下：

vim /root/monitor_nginx.sh

当检测到nginx进程不存在的时候，就干掉所有的keepalived，这时候，请求将会由keepalived的backup接管！！

				
						#!/bin/bash
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]
 then
 killall keepalived
fi

chmod +x /root/monitor_nginx.sh

172.26.11.99 172.26.11.100都重新启动keepalived：

service keepalived restart

这里请注意，当keepalived启动后，我们可以用命令：

ip add show eth0 来看我们的eth0网卡确实被添加了虚拟IP，如下图：

完毕，可以测试了！