案例

Nginx + Keepalived 案例

使用 Nginx + Keepalived 实现一个 web 服务高可用方案

架构图如下:

Nginx 的安装与配置

安装软件 # 安装nginx以及拓展源

yum install epel-release -y

yum install -y nginx

Nginx 配置 web 站点 #web1

[root@nginx1 ~]# vi /etc/nginx/conf.d/web.conf

server{

listen 8080;

root /usr/share/nginx/html;

index test.html;

}

[root@nginx1 ~]# echo "

This is web1

" > /usr/share/nginx/html/test.html

#web2

[root@nginx2 ~]# vi /etc/nginx/conf.d/web.conf

server{

listen 8080;

root /usr/share/nginx/html;

index test.html;

}

[root@nginx2 ~]# echo "

This is web2

" > /usr/share/nginx/html/test.html

启动 Nginx nginx -t

nginx

Keepalived 的安装与配置

安装 Keepalived yum install -y keepalived

配置 Keepalived(不同角色有些选项不一样:state 和 priority 配置项) web1:master vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

router_id LVS_DEVEL_1

}

vrrp_script nginx_check {

script "/tools/nginx_check.sh"

interval 1

}

vrrp_instance VI_1 {

state MASTER

#nopreempt

interface ens33

virtual_router_id 52

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass test

}

virtual_ipaddress {

192.168.149.100

}

track_script {

nginx_check

}

notify_master /tools/master.sh

notify_backup /tools/backup.sh

notify_fault /tools/fault.sh

notify_stop /tools/stop.sh

}

web2:slave vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

router_id LVS_DEVEL_2

}

vrrp_script nginx_check {

script "/tools/nginx_check.sh"

interval 1

}

vrrp_instance VI_1 {

state BACKUP

#nopreempt

interface ens33

virtual_router_id 52

priority 90

advert_int 1

authentication {

auth_type PASS

auth_pass test

}

virtual_ipaddress {

192.168.149.100

}

track_script {

nginx_check

}

notify_master /tools/master.sh

notify_backup /tools/backup.sh

notify_fault /tools/fault.sh

notify_stop /tools/stop.sh

}

编写相关脚本 编写 Keepalived 日志脚本(将每次工作过程输出到日志上) # 将所有脚本放在同一个目录下,方便集中管理

mkdir /tools && cd /tools

cat > master.sh << 'EOF'

ip=$(hostname -I | awk '{print $1}')

dt=$(date +'%Y%m%d %H:%M:%S')

echo "$0--${ip}--${dt}" >> /tmp/kp.log

EOF

cat > backup.sh << 'EOF'

ip=$(hostname -I | awk '{print $1}')

dt=$(date +'%Y%m%d %H:%M:%S')

echo "$0--${ip}--${dt}" >> /tmp/kp.log

EOF

cat > fault.sh << 'EOF'

ip=$(ip addr|grep inet| grep 192.168 |awk '{print $2}')

dt=$(date +'%Y%m%d %H:%M:%S')

echo "$0--${ip}--${dt}" >> /tmp/kp.log

EOF

cat > stop.sh << 'EOF'

ip=$(ip addr|grep inet| grep 192.168| awk '{print $2}')

dt=$(date +'%Y%m%d %H:%M:%S')

echo "$0--${ip}--${dt}" >> /tmp/kp.log

EOF

编写健康检查脚本 该脚本对 Nginx 进程进行检测,如果发现 Nginx 进程没了则返回失败(1) cat > nginx_check.sh << 'EOF'

#!/bin/bash

result=`pidof nginx`

if [ ! -z "${result}" ];

then exit 0

else exit 1

fi

EOF

编写完脚本后对这些脚本加 x 权限,并启动 Keepalived cd /tools/ && chmod +x *.sh

systemctl restart keepalived.service

验证

访问 VIP 地址 现在 web1(192.168.149.130)是 master 访问 192.168.149.100:8080,可以看到访问成功了 切换主备 关闭 web1 服务 [root@nginx1 ~]# nginx -s stop

然后再 web2 服务的主机可以看到 web2(192.168.149.131)获得了 VIP,升级成了主 访问 VIP 地址(192.168.149.100:8080),可以发现访问到了 web2 服务器上 之后重启 web1 看一下 可以看到 web1 主机又变成了 master,因为没有配置抢占模式/非抢占模式,所以选举机制是优先级,web1 的优先级比 web2 要高,所以 web1 启动后就会把 master 角色拿过来

问题解决

查看 keepalived 系统日志:less /var/log/messages

Unable to access script /shell/nginx_check.sh selinux 禁止了这个脚本。keepalived 进程的安全上下文与脚本的安全上下文不一致,就算有 rwx 权限也无法执行

最简单的解决方案就是关闭 selinux 功能: # 永久关闭 selinux

sed -i "s\SELINUX=enforcing\SELINUX=disabled\g" /etc/selinux/config

# 重启主机

reboot

# 临时关闭selinux,reboot服务器后失效

etenforce 0

解决方案2:将脚本转移到 /usr/libexec/keepalived 目录中 这个目录的安全上下文是 keepalived_unconfined_script_exec_t,与 keepalived 进程的安全上下文一致 解决方案3:修改 check.sh 的安全上下文为 keepalived 进程的安全上下文 chcon -t keepalived_unconfined_sript_exec_it /etc/keepalived/nginx_check.sh

keepalived_unconfined_script_exec_t 是一个在 SELinux 中用于标识 Keepalived 执行未限制脚本的上下文 这个上下文允许 Keepalived 进程在执行脚本时绕过一些 SELinux 限制,从而可以在需要的情况下执行脚本 # 查看 keepalived 进程的安全上下文

ps -eZ | grep keepalived

#输出:system_u:system_r:keepalived_t:s0 19689 ? 00:00:00 keepalived

# 查看文件/目录的安全上下文

ll -Z /etc/keepalived/check.sh

#输出:-rwr-xr-x. root system_u:object_r:etc_t:s0

Disabling track script chk_nginx since not found 在 VRRP 实例中添加 track_script { chk_nginx } 以启动脚本检测 warning default user ‘keepalived_script’for script execution does not exist - please create 找不到 keepalived_script 这个用户组,开启下面的配置,如果找不到就使用 root 在全局配置 global_defs 中添加 script_user root 注:该配置非必需,日志打印也只是警告而已 SECURITY VIOLATION - scripts are being executed but script_security notenabled 在全局配置 global_defs 中添加 enable_script_security Unknown keyword:’}’、‘track_script{’、'chk_nginx 等等 配置文件格式不正确:使用到大括号 { 的地方,确保正文与大括号间有空格。 Can't open PID file /var/run/nginx.pid (yet?) after start: No such fileor directory 使用 systemctl start nginx 时,先调用的是 nginx.service,启动时 nginx.pid 文件并未生成 在 /usr/lib/systemd/system/nginx.service 中 [service] 模块下添加 ExecStartPost=/bin/sleep 0.1 Can't open PID file /var/run/keepalived.pid (yet?) after start: No suchfile or directory 使用 systemctl start keepalived 时,先调用的是 keepalived.service,启动时 keealived.pid 文件并未生成 在 /usr/lib/systemd/system/keepalived.service 中 [service] 模块下添加 ExecStartPost=/bin/sleep 0.1

好文链接

评论可见,请评论后查看内容,谢谢!!!
 您阅读本篇文章共花了: