kubernetes中的ingress-nginx
华子目录
- ingress-nginx
- ingress-nginx功能
- 部署ingress及使用
- 注意
- `ingress`的`高级用法`
- 1.基于路径的访问
- 2.基于域名的访问
- 3.建立tls加密
- 4.建立auth认证
- 5.rewrite重定向
ingress-nginx
官网:https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal-clusters
ingress-nginx功能
- 一种
全局
的、为了代理
不同后端Service
而设置的负载均衡
服务,支持7层
Ingress
由两部分
组成:Ingress controller
和Ingress服务
Ingress Controller
会根据你定义的Ingress对象
,提供对应的代理能力
。- 业界常用的各种
反向代理
项目,比如Nginx、HAProxy、Envoy、Traefik
等,都已经为Kubernetes
专门维护了对应
的Ingress Controller
[root@k8s-master services]# kubectl get pods
No resources found in default namespace.[root@k8s-master services]# kubectl create deployment huazi --image myapp:v1 --dry-run=client -o yaml > huazi-dp.yml[root@k8s-master services]# vim huazi-dp.yml
[root@k8s-master services]# cat huazi-dp.yml
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: huaziname: huazi
spec:replicas: 1selector:matchLabels:app: huazitemplate:metadata:labels:app: huazispec:containers:- image: myapp:v1name: myapp#在克隆一份
[root@k8s-master services]# cp huazi-dp.yml huazi-dp1.yml
[root@k8s-master services]# vim huazi-dp1.yml
[root@k8s-master services]# cat huazi-dp1.yml
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: huaname: hua
spec:replicas: 1selector:matchLabels:app: huatemplate:metadata:labels:app: huaspec:containers:- image: myapp:v2name: myapp
[root@k8s-master services]# kubectl apply -f huazi-dp.yml
deployment.apps/huazi created
[root@k8s-master services]# kubectl apply -f huazi-dp1.yml
deployment.apps/hua created
[root@k8s-master services]# kubectl expose deployment huazi --port 8080 --target-port 80 --dry-run=client -o yaml >> huazi-dp.yml
[root@k8s-master services]# kubectl expose deployment hua --port 8080 --target-port 80 --dry-run=client -o yaml >> huazi-dp1.yml
[root@k8s-master services]# vim huazi-dp.yml
[root@k8s-master services]# vim huazi-dp1.yml
[root@k8s-master services]# kubectl apply -f huazi-dp.yml
deployment.apps/huazi unchanged
service/huazi created
[root@k8s-master services]# kubectl apply -f huazi-dp1.yml
deployment.apps/hua unchanged
service/hua created
[root@k8s-master services]# kubectl get pods -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
hua-69554ffb96-b7z6t 1/1 Running 0 8m33s 10.244.2.6 k8s-node2.org <none> <none> app=hua,pod-template-hash=69554ffb96
huazi-646d7864fd-w7rrz 1/1 Running 0 8m39s 10.244.2.5 k8s-node2.org <none> <none> app=huazi,pod-template-hash=646d7864fd[root@k8s-master services]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hua ClusterIP 10.96.56.127 <none> 8080/TCP 4m12s
huazi ClusterIP 10.102.139.29 <none> 8080/TCP 4m25s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20d[root@k8s-master services]# kubectl describe svc huazi
Name: huazi
Namespace: default
Labels: app=huazi
Annotations: <none>
Selector: app=huazi
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.102.139.29
IPs: 10.102.139.29
Port: <unset> 8080/TCP
TargetPort: 80/TCP
Endpoints: 10.244.2.5:80
Session Affinity: None
Events: <none>[root@k8s-master services]# kubectl describe svc hua
Name: hua
Namespace: default
Labels: app=hua
Annotations: <none>
Selector: app=hua
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.96.56.127
IPs: 10.96.56.127
Port: <unset> 8080/TCP
TargetPort: 80/TCP
Endpoints: 10.244.2.6:80
Session Affinity: None
Events: <none>
[root@k8s-master services]# curl 10.102.139.29:8080
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>[root@k8s-master services]# curl 10.96.56.127:8080
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
现在只能在集群内部
访问,可以利用ingress-nginx
部署ingress及使用
deploy.yaml
上传到master
中
[root@harbor ingress]# ls
deploy.yaml ingress-nginx-1.11.2.tag.gz
[root@harbor ingress]# docker load -i ingress-nginx-1.11.2.tag.gz
[root@harbor ingress]# docker tag reg.timinglee.org/ingress-nginx/controller:v1.11.2 harbor.huazi.org/ingress-nginx/controller:v1.11.2
[root@harbor ingress]# docker tag reg.timinglee.org/ingress-nginx/kube-webhook-certgen:v1.4.3 harbor.huazi.org/ingress-nginx/kube-webhook-certgen:v1.4.3
上传镜像
[root@harbor ingress]# docker push harbor.huazi.org/ingress-nginx/controller:v1.11.2[root@harbor ingress]# docker push harbor.huazi.org/ingress-nginx/kube-webhook-certgen:v1.4.3
#修改配置文件中的image位置
[root@k8s-master services]# vim deploy.yaml
[root@k8s-master services]# kubectl apply -f deploy.yaml
[root@k8s-master services]# kubectl get namespaces
[root@k8s-master services]# kubectl -n ingress-nginx get pods
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-8bhdh 0/1 Completed 0 2m11s
ingress-nginx-admission-patch-8lwwn 0/1 Completed 2 2m11s
ingress-nginx-controller-bb7d8f97c-d7psv 1/1 Running 0 2m11s[root@k8s-master services]# kubectl -n ingress-nginx get all
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-8bhdh 0/1 Completed 0 2m36s
pod/ingress-nginx-admission-patch-8lwwn 0/1 Completed 2 2m36s
pod/ingress-nginx-controller-bb7d8f97c-d7psv 1/1 Running 0 2m36sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller NodePort 10.102.168.161 <none> 80:32882/TCP,443:39565/TCP 2m36s
service/ingress-nginx-controller-admission ClusterIP 10.97.9.224 <none> 443/TCP 2m36sNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 2m36sNAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-bb7d8f97c 1 1 1 2m36sNAME STATUS COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create Complete 1/1 5s 2m36s
job.batch/ingress-nginx-admission-patch Complete 1/1 20s 2m36s
[root@k8s-master services]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hua ClusterIP 10.96.56.127 <none> 8080/TCP 28m
huazi ClusterIP 10.102.139.29 <none> 8080/TCP 28m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20d
[root@k8s-master services]# kubectl create ingress huazi --class nginx --rule='/=huazi:8080' --dry-run=client -o yaml > ingress1.yml[root@k8s-master services]# vim ingress1.yml
[root@k8s-master services]# cat ingress1.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: huazi
spec:ingressClassName: nginxrules:- http:paths:- backend:service:name: huazi #port:number: 8080 #path: / #所有对于根路径的请求,将被转发到名为 `huazi` 的服务(`Service`)的 `8080` 端口上。pathType: Prefix
Exact
(精确匹配
),ImplementationSpecific
(特定实现
),Prefix
(前缀匹配
),Regular expression
(正则表达式匹配
)
-
kubectl create ingress huazi
:这部分是命令的主体,表示要创建一个名为huazi
的Ingress
资源。 -
--class nginx
:这个参数指定了Ingress
控制器的类为nginx
。在Kubernetes
中,Ingress
控制器可以有多种实现,如nginx、traefik
等,这个参数就是用来指定使用哪种Ingress控制器
的。 -
--rule='/=huazi:8080'
:这个参数定义了Ingress规则
。- 这里的
规则表示
,对于所有匹配根路径
(/
)的请求
,都将被转发到名为huazi
的服务(Service
)的8080
端口上。 - 注意,这里的语法可能略有不同,通常我们看到的规则格式可能更接近于
--rule='/' - -service=huazi:8080
或者在YAML
文件中以更详细的方式配置。
- 这里的
[root@k8s-master services]# kubectl -n ingress-nginx get all
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-8bhdh 0/1 Completed 0 8m30s
pod/ingress-nginx-admission-patch-8lwwn 0/1 Completed 2 8m30s
pod/ingress-nginx-controller-bb7d8f97c-d7psv 1/1 Running 0 8m30sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller NodePort 10.102.168.161 <none> 80:32882/TCP,443:39565/TCP 8m30s
service/ingress-nginx-controller-admission ClusterIP 10.97.9.224 <none> 443/TCP 8m30sNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 8m30sNAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-bb7d8f97c 1 1 1 8m30sNAME STATUS COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create Complete 1/1 5s 8m30s
job.batch/ingress-nginx-admission-patch Complete 1/1 20s 8m30s
- 修改
微服务
为LoadBalancer
[root@k8s-master services]# kubectl -n ingress-nginx edit svc ingress-nginx-controller
[root@k8s-master services]# kubectl -n ingress-nginx get all
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-8bhdh 0/1 Completed 0 11m
pod/ingress-nginx-admission-patch-8lwwn 0/1 Completed 2 11m
pod/ingress-nginx-controller-bb7d8f97c-d7psv 1/1 Running 0 11mNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller LoadBalancer 10.102.168.161 172.25.254.50 80:32882/TCP,443:39565/TCP 11m
service/ingress-nginx-controller-admission ClusterIP 10.97.9.224 <none> 443/TCP 11mNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 11mNAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-bb7d8f97c 1 1 1 11mNAME STATUS COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create Complete 1/1 5s 11m
job.batch/ingress-nginx-admission-patch Complete 1/1 20s 11m
- 在
ingress-nginx-controller
中看到的对外IP
就是ingress最终对外开放的ip
[root@k8s-master services]# kubectl apply -f ingress1.yml
[root@k8s-master services]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
huazi nginx * 172.25.254.10 80 18m
[root@k8s-master services]# curl 172.25.254.50
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
注意
ingress
必须和输出的service
资源处于同一namespace
ingress
的高级用法
1.基于路径的访问
- 建立用于
测试
的控制器myapp
[root@k8s-master services]# kubectl create deployment myapp-v1 --image myapp:v1 --dry-run=client -o yaml > myapp-v1.yaml
[root@k8s-master services]# kubectl create deployment myapp-v2 --image myapp:v2 --dry-run=client -o yaml > myapp-v2.yaml
[root@k8s-master services]# vim myapp-v1.yaml
[root@k8s-master services]# cat myapp-v1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: myapp-v1name: myapp-v1
spec:replicas: 1selector:matchLabels:app: myapp-v1template:metadata:labels:app: myapp-v1spec:containers:- image: myapp:v1name: myapp
[root@k8s-master services]# vim myapp-v2.yaml
[root@k8s-master services]# cat myapp-v2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: myapp-v2name: myapp-v2
spec:replicas: 1selector:matchLabels:app: myapp-v2template:metadata:labels:app: myapp-v2spec:containers:- image: myapp:v2name: myapp
[root@k8s-master services]# kubectl apply -f myapp-v1.yaml
deployment.apps/myapp-v1 created
[root@k8s-master services]# kubectl apply -f myapp-v2.yaml
deployment.apps/myapp-v2 created
[root@k8s-master services]# kubectl expose deployment myapp-v1 --port 80 --target-port 80 --dry-run=client -o yaml >> myapp-v1.yaml
[root@k8s-master services]# kubectl expose deployment myapp-v2 --port 80 --target-port 80 --dry-run=client -o yaml >> myapp-v2.yaml
[root@k8s-master services]# vim myapp-v1.yaml
[root@k8s-master services]# cat myapp-v1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: myapp-v1name: myapp-v1
spec:replicas: 1selector:matchLabels:app: myapp-v1template:metadata:labels:app: myapp-v1spec:containers:- image: myapp:v1name: myapp---
apiVersion: v1
kind: Service
metadata:labels:app: myapp-v1name: myapp-v1
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: myapp-v1
[root@k8s-master services]# vim myapp-v2.yaml
[root@k8s-master services]# cat myapp-v2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: myapp-v2name: myapp-v2
spec:replicas: 1selector:matchLabels:app: myapp-v2template:metadata:labels:app: myapp-v2spec:containers:- image: myapp:v2name: myapp---
apiVersion: v1
kind: Service
metadata:labels:app: myapp-v2name: myapp-v2
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: myapp-v2
[root@k8s-master services]# kubectl apply -f myapp-v1.yaml
deployment.apps/myapp-v1 unchanged
service/myapp-v1 created
[root@k8s-master services]# kubectl apply -f myapp-v2.yaml
deployment.apps/myapp-v2 unchanged
service/myapp-v2 created
[root@k8s-master services]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
myapp-v1-7479d6c54d-m7tcj 1/1 Running 0 6m19s 10.244.2.4 k8s-node2.org <none> <none>
myapp-v2-7cd6d597d-t2sd5 1/1 Running 0 6m12s 10.244.2.5 k8s-node2.org <none> <none>[root@k8s-master services]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23d
myapp-v1 ClusterIP 10.111.141.57 <none> 80/TCP 85s
myapp-v2 ClusterIP 10.103.170.42 <none> 80/TCP 79s
- 建立
ingress1.yaml
文件
[root@k8s-master services]# vim ingress1.yml
[root@k8s-master services]# cat ingress1.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:annotations:nginx.ingress.kubernetes.io/rewrite-target: / #访问路径后加任何内容都被定向到/name: ingress1
spec:ingressClassName: nginxrules:- host: www.huazi.comhttp:paths:- backend: # ingress只能在一个文件中写,这里我们写两个backendservice:name: myapp-v1port:number: 80path: /v1pathType: Prefix- backend:service:name: myapp-v2port:number: 80path: /v2pathType: Prefix
[root@k8s-master services]# kubectl apply -f ingress1.yml
ingress.networking.k8s.io/ingress1 created
[root@k8s-master services]# vim /etc/hosts
[root@k8s-master services]# kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.102.168.161 172.25.254.50 80:32882/TCP,443:39565/TCP 2d3h
ingress-nginx-controller-admission ClusterIP 10.97.9.224 <none> 443/TCP 2d3h
[root@k8s-master services]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress1 nginx www.huazi.com 80 18s
- 测试
[root@k8s-master services]# curl www.huazi.com/v1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8s-master services]# curl www.huazi.com/v2
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
#nginx.ingress.kubernetes.io/rewrite-target: / 的功能实现
[root@k8s-master services]# curl www.huazi.com/v1/gagdasghg
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8s-master services]# curl www.huazi.com/v2/gagdasghg
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
2.基于域名的访问
- 在测试主机中设定解析
[root@k8s-master services]# vim /etc/hosts
- 建立基于
域名
的yaml文件
[root@k8s-master services]# vim ingress2.yml
[root@k8s-master services]# cat ingress2.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:annotations:nginx.ingress.kubernetes.io/rewrite-target: /name: ingress2
spec:ingressClassName: nginxrules:- host: myappv1.huazi.com #第一个域名http:paths:- backend:service:name: myapp-v1port:number: 80path: /pathType: Prefix- host: myappv2.huazi.com #第二个域名http:paths:- backend:service:name: myapp-v2port:number: 80path: /pathType: Prefix
[root@k8s-master services]# kubectl apply -f ingress2.yml
ingress.networking.k8s.io/ingress2 created
[root@k8s-master services]# kubectl describe ingress ingress2
Name: ingress2
Labels: <none>
Namespace: default
Address: 172.25.254.10
Ingress Class: nginx
Default backend: <default>
Rules:Host Path Backends---- ---- --------myappv1.huazi.com/ myapp-v1:80 (10.244.2.4:80)myappv2.huazi.com/ myapp-v2:80 (10.244.2.5:80)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Sync 31s (x2 over 37s) nginx-ingress-controller Scheduled for sync[root@k8s-master services]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress1 nginx www.huazi.com 172.25.254.10 80 17m
ingress2 nginx myappv1.huazi.com,myappv2.huazi.com 172.25.254.10 80 98s
- 测试
[root@k8s-master services]# curl myappv1.huazi.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8s-master services]# curl myappv2.huazi.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
3.建立tls加密
- 建立证书
[root@k8s-master services]# openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -subj "/CN=nginxsvc/O=nginxsvc" -out tls.crt
- 建立加密资源类型
secret
secret
通常在kubernetes
中存放敏感数据
,他并不是一种加密方式
[root@k8s-master services]# kubectl create secret tls web-tls-secret --key tls.key --cert tls.crt
secret/web-tls-secret created
- 建立基于
tls
认证的ingress3.yml
文件
[root@k8s-master services]# vim ingress3.yml
[root@k8s-master services]# cat ingress3.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:annotations:nginx.ingress.kubernetes.io/rewrite-target: /name: ingress3
spec:tls:- hosts:- myapp-tls.huazi.comsecretName: web-tls-secretingressClassName: nginxrules:- host: myapp-tls.huazi.comhttp:paths:- backend:service:name: myapp-v1port:number: 80path: /pathType: Prefix
[root@k8s-master services]# kubectl apply -f ingress3.yml
ingress.networking.k8s.io/ingress3 created
- 测试
[root@k8s-master services]# kubectl describe ingress ingress3
Name: ingress3
Labels: <none>
Namespace: default
Address: 172.25.254.10
Ingress Class: nginx
Default backend: <default>
TLS:web-tls-secret terminates myapp-tls.huazi.com
Rules:Host Path Backends---- ---- --------myapp-tls.huazi.com/ myapp-v1:80 (10.244.2.4:80)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Sync 4m18s (x2 over 4m46s) nginx-ingress-controller Scheduled for sync
[root@k8s-master services]# curl -k https://myapp-tls.huazi.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>