当前位置： 首页 > news >正文

使用k3s高可用部署rancher

news 2025/2/13 9:38:48

本次部署采用3节点的etcd服务+2master节点的k3s+使用helm部署的rancher+vip(keepalived)

一、安装etcd服务

# 准备 3 个节点部署 etcd

cd /hskj/tmp
wget  https://github.com/etcd-io/etcd/releases/download/v3.3.15/etcd-v3.3.15-linux-amd64.tar.gz
tar xzvf etcd-v3.3.15-linux-amd64.tar.gz -C /hskj/
cp /hskj/etcd-v3.3.15-linux-amd64/etc* /usr/local/bin

创建 etcd.service 服务托管于 systemd

 vim /usr/lib/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target[Service]
Type=notify
# 指定环境变量所在 
EnvironmentFile=-/etc/etcd/config
ExecStart=/usr/local/bin/etcd \
--name=${ETCD_NAME} \
--data-dir=${ETCD_DATA_DIR} \
--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-state=new 
Restart=on-failure
LimitNOFILE=65536[Install]
WantedBy=multi-user.target

创建 etcd 配置文件
mkdir /etc/etcd 
vim /etc/etcd/config 

#[Member]
##节点名字,每个节点都进行修改
ETCD_NAME="etcd01" 
#数据目录
ETCD_DATA_DIR="/hskj/etcd/"
#当前节点的ip地址,每个节点都进行修改
ETCD_LISTEN_PEER_URLS="http://192.168.113.230:2380" 
ETCD_LISTEN_CLIENT_URLS="http://192.168.113.230:2379"#[Clustering]
#每个节点都要修改
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.113.230:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.113.230:2379"
#集群所有的节点的ip地址
ETCD_INITIAL_CLUSTER="etcd01=http://192.168.113.230:2380,etcd02=http://192.168.113.231:2380,etcd03=http://192.168.113.242:2380" 
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

启动 etcd

systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

检查etcd服务是否正常

# ETCDCTL_API=3 etcdctl --endpoints=http://192.168.113.230:2379,http://192.168.113.231:2379,http://192.168.113.242:2379 endpoint health
http://192.168.113.242:2379 is healthy: successfully committed proposal: took = 8.90272ms
http://192.168.113.231:2379 is healthy: successfully committed proposal: took = 9.948766ms
http://192.168.113.230:2379 is healthy: successfully committed proposal: took = 9.792125ms

二、安装k3s集群，

1、下载软件，并赋予执行权限

cd /hskj/tmp
wget https://rancher-mirror.rancher.cn/k3s/v1.26.15-k3s1/k3s-airgap-images-amd64.tar.gz
wget https://rancher-mirror.rancher.cn/k3s/v1.26.15-k3s1/k3s
chmod +x k3s

2.然后将k3s的二进制文件复制到/usr/local/bin/

cp k3s /usr/local/bin/

3.然后导入镜像

docker load -i k3s-airgap-images-amd64.tar.gz

4.设置环境遍历，

这里要先禁用k3s自带的traefik，不禁用无法搭建2master。等k3s双master节点搭建成功后在手动部署traefix

export INSTALL_K3S_SKIP_DOWNLOAD=true
export INSTALL_K3S_EXEC='--docker --disable=traefik'
export INSTALL_K3S_VERSION=k3s_v1.26.15-k3s1

5、安装k3s,--token手动指定

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -s - server \
--token=5f98ff636dc1b5cad22fedaa46390557 \
--datastore-endpoint="http://192.168.113.231:2379,http://192.168.113.232:2379,http://192.168.113.233:2379"

6、查看k3s服务

[root@jxqzw-kvm-rancher-130-89 tmp]# systemctl status k3s
● k3s.service - Lightweight KubernetesLoaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: disabled)Active: active (running) since 四 2025-02-13 08:46:50 CST; 38min agoDocs: https://k3s.ioMain PID: 3974 (k3s-server)Tasks: 25Memory: 852.4MCGroup: /system.slice/k3s.service└─3974 /usr/local/bin/k3s server

7、查看k3s集群是否ok

[root@jxqzw-kvm-rancher-130-89 tmp]#kubectl get nodes                                                              kubectl get nodes
NAME                       STATUS   ROLES                  AGE   VERSION
jxqzw-kvm-rancher-130-89   Ready    control-plane,master   17h   v1.26.15+k3s1
jxqzw-kvm-rancher-130-90   Ready    control-plane,master   17h   v1.26.15+k3s1
您在 /var/spool/mail/root 中有新邮件

三、安装rancher

1、rancher签证脚本直接复制即可

#!/bin/bash -e
help ()
{echo  ' ================================================================ 'echo  ' --ssl-domain: 生成ssl证书需要的主域名，如不指定则默认为www.rancher.local，如果是ip访问服务，则可忽略；'echo  ' --ssl-trusted-ip: 一般ssl证书只信任域名的访问请求，有时候需要使用ip去访问server，那么需要给ssl证书添加扩展IP，多个IP用逗号隔开；'echo  ' --ssl-trusted-domain: 如果想多个域名访问，则添加扩展域名（SSL_TRUSTED_DOMAIN）,多个扩展域名用逗号隔开；'echo  ' --ssl-size: ssl加密位数，默认2048；'echo  ' --ssl-cn: 国家代码(2个字母的代号),默认CN;'echo  ' 使用示例:'echo  ' ./create_self-signed-cert.sh --ssl-domain=www.test.com --ssl-trusted-domain=www.test2.com \ 'echo  ' --ssl-trusted-ip=1.1.1.1,2.2.2.2,3.3.3.3 --ssl-size=2048 --ssl-date=3650'echo  ' ================================================================'
}case "$1" in-h|--help) help; exit;;
esacif [[ $1 == '' ]];thenhelp;exit;
fiCMDOPTS="$*"
for OPTS in $CMDOPTS;
dokey=$(echo ${OPTS} | awk -F"=" '{print $1}' )value=$(echo ${OPTS} | awk -F"=" '{print $2}' )case "$key" in--ssl-domain) SSL_DOMAIN=$value ;;--ssl-trusted-ip) SSL_TRUSTED_IP=$value ;;--ssl-trusted-domain) SSL_TRUSTED_DOMAIN=$value ;;--ssl-size) SSL_SIZE=$value ;;--ssl-date) SSL_DATE=$value ;;--ca-date) CA_DATE=$value ;;--ssl-cn) CN=$value ;;esac
done# CA相关配置
CA_DATE=${CA_DATE:-3650}
CA_KEY=${CA_KEY:-cakey.pem}
CA_CERT=${CA_CERT:-cacerts.pem}
CA_DOMAIN=cattle-ca# ssl相关配置
SSL_CONFIG=${SSL_CONFIG:-$PWD/openssl.cnf}
SSL_DOMAIN=${SSL_DOMAIN:-'www.rancher.local'}
SSL_DATE=${SSL_DATE:-3650}
SSL_SIZE=${SSL_SIZE:-2048}## 国家代码(2个字母的代号),默认CN;
CN=${CN:-CN}SSL_KEY=$SSL_DOMAIN.key
SSL_CSR=$SSL_DOMAIN.csr
SSL_CERT=$SSL_DOMAIN.crtecho -e "\033[32m ---------------------------- \033[0m"
echo -e "\033[32m       | 生成 SSL Cert |       \033[0m"
echo -e "\033[32m ---------------------------- \033[0m"if [[ -e ./${CA_KEY} ]]; thenecho -e "\033[32m ====> 1. 发现已存在CA私钥，备份"${CA_KEY}"为"${CA_KEY}"-bak，然后重新创建 \033[0m"mv ${CA_KEY} "${CA_KEY}"-bakopenssl genrsa -out ${CA_KEY} ${SSL_SIZE}
elseecho -e "\033[32m ====> 1. 生成新的CA私钥 ${CA_KEY} \033[0m"openssl genrsa -out ${CA_KEY} ${SSL_SIZE}
fiif [[ -e ./${CA_CERT} ]]; thenecho -e "\033[32m ====> 2. 发现已存在CA证书，先备份"${CA_CERT}"为"${CA_CERT}"-bak，然后重新创建 \033[0m"mv ${CA_CERT} "${CA_CERT}"-bakopenssl req -x509 -sha256 -new -nodes -key ${CA_KEY} -days ${CA_DATE} -out ${CA_CERT} -subj "/C=${CN}/CN=${CA_DOMAIN}"
elseecho -e "\033[32m ====> 2. 生成新的CA证书 ${CA_CERT} \033[0m"openssl req -x509 -sha256 -new -nodes -key ${CA_KEY} -days ${CA_DATE} -out ${CA_CERT} -subj "/C=${CN}/CN=${CA_DOMAIN}"
fiecho -e "\033[32m ====> 3. 生成Openssl配置文件 ${SSL_CONFIG} \033[0m"
cat > ${SSL_CONFIG} <<EOM
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
EOMif [[ -n ${SSL_TRUSTED_IP} || -n ${SSL_TRUSTED_DOMAIN} ]]; thencat >> ${SSL_CONFIG} <<EOM
subjectAltName = @alt_names
[alt_names]
EOMIFS=","dns=(${SSL_TRUSTED_DOMAIN})dns+=(${SSL_DOMAIN})for i in "${!dns[@]}"; doecho DNS.$((i+1)) = ${dns[$i]} >> ${SSL_CONFIG}doneif [[ -n ${SSL_TRUSTED_IP} ]]; thenip=(${SSL_TRUSTED_IP})for i in "${!ip[@]}"; doecho IP.$((i+1)) = ${ip[$i]} >> ${SSL_CONFIG}donefi
fiecho -e "\033[32m ====> 4. 生成服务SSL KEY ${SSL_KEY} \033[0m"
openssl genrsa -out ${SSL_KEY} ${SSL_SIZE}echo -e "\033[32m ====> 5. 生成服务SSL CSR ${SSL_CSR} \033[0m"
openssl req -sha256 -new -key ${SSL_KEY} -out ${SSL_CSR} -subj "/C=${CN}/CN=${SSL_DOMAIN}" -config ${SSL_CONFIG}echo -e "\033[32m ====> 6. 生成服务SSL CERT ${SSL_CERT} \033[0m"
openssl x509 -sha256 -req -in ${SSL_CSR} -CA ${CA_CERT} \-CAkey ${CA_KEY} -CAcreateserial -out ${SSL_CERT} \-days ${SSL_DATE} -extensions v3_req \-extfile ${SSL_CONFIG}echo -e "\033[32m ====> 7. 证书制作完成 \033[0m"
echo
echo -e "\033[32m ====> 8. 以YAML格式输出结果 \033[0m"
echo "----------------------------------------------------------"
echo "ca_key: |"
cat $CA_KEY | sed 's/^/  /'
echo
echo "ca_cert: |"
cat $CA_CERT | sed 's/^/  /'
echo
echo "ssl_key: |"
cat $SSL_KEY | sed 's/^/  /'
echo
echo "ssl_csr: |"
cat $SSL_CSR | sed 's/^/  /'
echo
echo "ssl_cert: |"
cat $SSL_CERT | sed 's/^/  /'
echoecho -e "\033[32m ====> 9. 附加CA证书到Cert文件 \033[0m"
cat ${CA_CERT} >> ${SSL_CERT}
echo "ssl_cert: |"
cat $SSL_CERT | sed 's/^/  /'
echoecho -e "\033[32m ====> 10. 重命名服务证书 \033[0m"
echo "cp ${SSL_DOMAIN}.key tls.key"
cp ${SSL_DOMAIN}.key tls.key
echo "cp ${SSL_DOMAIN}.crt tls.crt"
cp ${SSL_DOMAIN}.crt tls.crt

2、执行自签证脚本,(将vip脚本也加入)

sh key-create.sh --ssl-domain=www.rancher.baidutest.com --ssl-trusted-ip=192.168.113.200，192.168.113.231,192.168.113.230,192.168.113.242 --ssl-date=36500 --ssl-size=2048

3、 验证是否成功

openssl verify -CAfile cacerts.pem tls.cr

openssl x509 -in tls.crt -noout -text

4、安装helm

# 安装 helm 
wget https://rancher-mirror.rancher.cn/helm/v3.17.0/helm-v3.17.0-linux-amd64.tar.gz
tar zxvf helm-v3.17.0-linux-amd64.tar.gz -C /hskj/
cp /hskj/linux-amd64/helm /usr/local/bin/ 
# 创建命名空间
kubectl  create namespace cattle-system
# ca证书密文
kubectl -n cattle-system create secret tls tls-rancher-ingress --cert=tls.crt --key=tls.key 
kubectl -n cattle-system create secret generic tls-ca --from-file=cacerts.pem
# 添加 chat 仓库，推荐添加 stable 稳定版仓库，如果需要固定版本可以去 http://mirror.cnrancher.com/ 上面下载
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
# helm install rancher 
helm install rancher rancher-stable/rancher \
-n cattle-system \
# 使用自签证
--set ingress.tls.source=secret \
# 是否使用自签证
--set privateCA=true \
--set additionalTrustedCAs=true \
--set hostname=rancher.baidutest.com
# 验证是否安装成功
helm list -n cattle-system 
# 检查 rancher 服务运行是否正常
kubectl -n cattle-system get pods
# 查看 ingress 地址
kubectl -n cattle-system get ingress

选择 Rancher 版本 | Rancher

5、使用helm安装traefik

# 使用helm命令行安装它：
helm repo add traefik https://traefik.github.io/charts
helm repo update
kubectl create ns traefik
helm install -n traefik traefik traefik/traefik
kubectl get all  -n  traefik

6、在2台部署了k3s的节点上，安装keepalived

yum install -y keepalived

vi /etc/keepalived/keepalived.conf

! Configuration File for keepalivedglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 192.168.200.1smtp_connect_timeout 30router_id LVS_DEVELvrrp_skip_check_adv_addrvrrp_strictvrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.130.200}
}

systemctl start keepalived

systemctl enable keepalived

7、配置本地hosts，进行访问设置。高可用验证，可随机关闭一台k3s服务器，rancher也可正常访问

---------------------------------------------------问题集----------------------------------------------------------------

1、使用helm安装报错：

helm安装com Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "http://localhost:8080/version": dial tcp [::1]:8080: connect: connection refused

如果文件不存在，可能需要重新生成 Kubeconfig 文件。对于 K3s 集群，Kubeconfig 文件通常位于 /etc/rancher/k3s/k3s.yaml。你可以将其复制到 ~/.kube/config 并设置正确的权限：

mkdir -p ~/.kube

sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

sudo chown $(id -u):$(id -g) ~/.kube/config

2、安装过程中会缺失一些镜像，需要根据实际缺失的进行导入，由于墙的原因，很多镜像都是一言难尽，

参考链接：k3s安装指定版本以及离线安装（docker)_k3s离线安装-CSDN博客

Rancher Releases Mirror
Rancher On K3s 高可用架构部署 - 尚墨 - 博客园

单机安装Rancher2.8.5-CSDN博客


http://www.mrgr.cn/news/90472.html

相关文章：

  • 名词解释：npm，cnpm，yarn，vite，vue，electron
  • vue不是内部或外部命令？
  • 【Stable Diffusion模型测试】测试ControlNet，没有线稿图？
  • 【安全靶场】信息收集靶场
  • 深入理解Java对接DeepSeek
  • 超详细的数据结构3（初阶C语言版）栈和队列。
  • YOLO自定义数据集实现K折交叉验证——K-Fold Cross Validation
  • 使用grafana v11 建立k线(蜡烛图)仪表板
  • CF Round 997 记录 题解 （div. 2 A - E）
  • PyQt学习记录03——批量设置水印
  • 算法很美笔记（Java）——树
  • package.json 文件配置
  • 华为云kubernetes基于keda自动伸缩deployment副本（监听redis队列长度）
  • python 获取smpl身高 fbx身高
  • 如何使用Java语言在Idea和Android中分别建立服务端和客户端实现局域网聊天
  • Android 14.0 Launcher3单层模式workspace中app列表页排序功能实现
  • @synchronized的使用
  • 使用 Express 写接口
  • 自己部署DeepSeek 助力 Vue 开发：打造丝滑的标签页（Tabs）
  • 通过钉钉创建个人AI助理：无需官网即可使用DeepSeek满血版全攻略
  • [极客大挑战 2019]PHP
  • 【竞技宝】LOL-LPL:EDG3-0零封LNG
  • 图神经网络怎么和LLM结合
  • 前端如何判断浏览器 AdBlock/AdBlock Plus（最新版）广告屏蔽插件已开启拦截
  • 使用EVE-NG-锐捷实现ACL访问控制
  • C++ 设计模式-桥接模式