k8s☞13-1 ingress-nginx的各种用法

阅读量: zyh 2020-09-22 18:30:04

Categories： > Tags：

基本

官方文档

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/

关键须知

ingress 的注解可能会在当前所有的 path 里生效。例如在 ingress-nginx 中，下列两个就会遵循这个规则：

inginx.ingress.kubernetes.io/rewrite-target
nginx.ingress.kubernetes.io/configuration-snippet

💥这里列举的注解并非是所有遵循这个规则的注解，仅是一个例子。

✨如果你只想针对某个 path 来添加，则需要重新创建一个新的ingress对象，并保持 ingress 对象中的 host 一致。

💥ingress对象名不可以相同。

✨不同的ingress对象，但拥有相同的host和不同的path，则两个ingress对象规则会合并。

两个ingress的host和path不能完全一样，一样的时候，第二个ingress应用的时候会被拒绝，并提示已定义。

Error from server (BadRequest): error when creating "1-ingress-err.yaml": admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "ingress.xxx.com" and path "/foo" is already defined in ingress default/test-ingress

如果你发现某个注解没有在 ingress-controller 里的 nginx.conf 里生效，则看下注解的key是否写的正确。

👙可能的最佳方式：

一个域名一个yaml，一个yaml里添加多个 ingress，一个ingress配置通用，其它ingress对应特殊的path（额外注解），并确保所有的ingress保持相同的host和不同的name

基本配置示例1

后端服务

✨registry.cn-hangzhou.aliyuncs.com/yilong/ingress-test:web1 可以接收 /foo 和 /bar ，并输出 web1: /foo 和 web1: /bar

0-dep-web1service.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-web1
  labels:
    app: test-web1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-web1
  template:
    metadata:
      labels:
        app: test-web1
    spec:
      containers:
      - name: test-web1
        imagePullPolicy: IfNotPresent
        image: registry.cn-hangzhou.aliyuncs.com/yilong/ingress-test:web1
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: web1-service
spec:
  type: ClusterIP
  selector:
    app: test-web1
  ports:
    - port: 8080
      targetPort: 8080

ingress 配置

1-ingress-web1service.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: ImplementationSpecific
      - path: /bar
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: ImplementationSpecific

访问

curl -H "host:foo.bar.com" http://39.103.219.253/foo
web1: /foo
curl -H "host:foo.bar.com" http://39.103.219.253/bar
web1: /bar

基本重定向

使用nginx.ingress.kubernetes.io/rewrite-target注解支持基本的Rewrite配置。

下列配置实现效果： http://foo.bar.com/aoo/xxx => http://foo.bar.com/xxx

💥这个基本重定向的 rewrite 会写在当前 ingress 对象里的所有 path 下，等同于

location /foo {
rewrite "(?i)/foo" /$1 break;
}
location /bar {
rewrite "(?i)/bar" /$1 break;
}
location /aoo/(.*) {
rewrite "(?i)/aoo/(.*)" /$1 break;
}

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
  annotations:
    #URL重定向。
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: ImplementationSpecific
      - path: /bar
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: ImplementationSpecific
      - path: /aoo/(.*)
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: ImplementationSpecific

复杂重定向

nginx.ingress.kubernetes.io/server-snippet：添加配置到nginx的server配置上下文。
nginx.ingress.kubernetes.io/configuration-snippet：添加配置到nginx的【所有】location上下文。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |
      rewrite ^/v4/(.*)/card/query http://foo.bar.com/v5/#!/card/query permanent;
    nginx.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^/v6/(.*)/card/query http://foo.bar.com/v7/#!/card/query permanent;

对应的nginx配置如下（configuration-snippet的配置根据location来动态添加）

## start server foo.bar.com
    server {
        server_name foo.bar.com ;
        listen 80;
        listen [::]:80;
        set $proxy_upstream_name "-";
        ### server-snippet配置。
        rewrite ^/v4/(.*)/card/query http://foo.bar.com/v5/#!/card/query permanent;
        ...
        location / {
          ### configuration-snippet配置。
          rewrite ^/v6/(.*)/card/query http://foo.bar.com/v7/#!/card/query permanent;
          ...
      }
    }
    ## end server foo.bar.com

基本重定向和复杂重定向混合使用

实现需求：

➜   curl ingress.pengwin.com/foo
web1: /foo
➜   curl ingress.pengwin.com/bar
web1: /bar
➜   curl ingress.pengwin.com/aoo/foo
web1: /foo
➜   curl ingress.pengwin.com/aoo/bar
web1: /bar

# 满足 curl ingress.pengwin.com/foo 和 curl ingress.pengwin.com/bar
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-pengwin-com-01
  annotations:
    nginx.ingress.kubernetes.io/use-regex: "true"     # 启用正则，否则 path 无法支持正则
spec:
  tls:
  - hosts:
      - ingress.pengwin.com
    secretName: pengwin-com-tls
  rules:
  - host: ingress.pengwin.com
    http:
      paths:
      - path: /(foo|bar)
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: Prefix
---
# 满足  curl ingress.pengwin.com/aoo/foo 和 curl ingress.pengwin.com/aoo/bar
# 仅针对 path: /aoo/ 启用特殊的重定向规则.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-pengwin-com-02
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
        rewrite "(?i)/aoo/(.*)" /$1 break; 
spec:
  tls:
  - hosts:
      - ingress.pengwin.com
    secretName: pengwin-com-tls
  rules:
  - host: ingress.pengwin.com
    http:
      paths:
      - path: /aoo/
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: Prefix

SSL配置

假设你需要创建一个foo.bar.com域名的私有证书

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout foo.bar.com.key -out foo.bar.com.crt -subj "/CN=foo.bar.com/O=foo.bar.com"

通过k8s的secret对象导入证书

命令方式：

kubectl create secret tls foo-bar-com --key foo.bar.com.key --cert foo.bar.com.crt

声明方式：

apiVersion: v1
kind: Secret
metadata:
  name: foo-bar-com
  namespace: default
data:
  tls.crt: base64 encoded cert # 这里需要填写 base64 编码后的 cert 内容. tls.crt 不可更名
  tls.key: base64 encoded key  # 这里需要填写 base64 编码后的 key 内容. tls.key 不可更名
type: kubernetes.io/tls

ingress调用证书

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo-bar-com-foo
  namespace: default
spec:
  tls:
  - hosts:
    - foo.bar.com
    secretName: foo-bar-com
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          service: 
            name: web1-service
            port: 
              number: 80
        pathType: ImplementationSpecific

ingress设置tls之后，会自动创建443的配置

➜   kubectl get ingress
NAME           CLASS   HOSTS                 ADDRESS          PORTS     AGE
foo-bar-com-foo   nginx   foo.bar.com  		 xxx.xxx.xxx.xxx   80, 443   10m

双向SSL配置

创建ca、server、client的私钥和证书

✨命令严格执行

✨server 请求文件 /CN=foo.bar.com 替换成自有域名

#ca
openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=Fern CA'
#server
openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=foo.bar.com'
openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
#client
openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes -subj '/CN=Fern'
openssl x509 -req -sha256 -days 3650 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

创建ca、server的secret对象

kubectl create secret generic ca-secret --from-file=ca.crt=ca.crt
kubectl create secret generic tls-secret --from-file=tls.crt=server.crt --from-file=tls.key=server.key

ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
    nginx.ingress.kubernetes.io/auth-tls-secret: "default/ca-secret"
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
  name: nginx-test
  namespace: default
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          service:
            name: web1-service
            port: 
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - foo.bar.com
    secretName: tls-secret

测试

 curl --cacert ./ca.crt --cert ./client.crt --key ./client.key https://foo.bar.com

ingress转发到https服务

通过添加注解：nginx.ingress.kubernetes.io/backend-protocol: “HTTPS”

追加多个域名到server_name

nginx.ingress.kubernetes.io/server-alias: ‘~^\d+.pengwin.com$’

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-pengwin-com-foo
  annotations:
    nginx.ingress.kubernetes.io/server-alias: '~^\d+\.bar\.com$'
spec:
  tls:
  - hosts:
    - foo.bar.com
    secretName: foo-bar-com
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          service:
            name: web1-service
            port:
              number: 8080
        pathType: ImplementationSpecific

-------------------------------------------------------------------------------------

基本配置示例2

后端服务

0-dep-oldnginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: old-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      run: old-nginx
  template:
    metadata:
      labels:
        run: old-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx
        imagePullPolicy: Always
        name: old-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: old-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: old-nginx
  sessionAffinity: None
  type: NodePort

ingress 配置

1-ingress-oldnginx.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # 新版本服务。
      - path: /
        backend:
          service:
            name: old-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

测试

➜ kubectl get ingress
➜ curl -H "host:www.example.com" http://<ingress_ip>
old

蓝绿发布

蓝绿发布的基本逻辑：

构建新版本的yaml配置清单。
- 通过 label 来区分新旧程序。例如这里通过：run: old-nginx 和 run: new-nginx 来区分新旧版本
- 通过 service.metadata.name 来区分新旧程序的服务。例如这里是： old-nginx 和 new-nginx

apiVersion: apps/v1
kind: Deployment
metadata:
  name: new-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      run: new-nginx
  template:
    metadata:
      labels:
        run: new-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx
        imagePullPolicy: Always
        name: new-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: new-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: new-nginx
  sessionAffinity: None
  type: NodePort

更新旧版本的ingress

监控☞monit

k8s☞13高级版负载均衡ingress控制器