molesignal.example.com 在内网 DNS 和公网 DNS 里分别解析到这两个负载均衡。MoleSignal 的 Service 在命名空间 molesignal 里,名字是 molesignal、端口 5080;分角色部署时换成 router 的 Service。
两个控制器
两个控制器的 IngressClass 和controllerValue 都要不同,否则会去处理对方的 Ingress。内网控制器的 Service 加上云厂商的内网负载均衡注解:
# ingress-internal-values.yaml
controller:
ingressClassResource:
name: nginx-internal
controllerValue: k8s.io/ingress-nginx-internal
ingressClass: nginx-internal
electionID: ingress-nginx-internal-leader
service:
annotations:
# 按云厂商填写,例如阿里云 ACK:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: intranet
# ingress-public-values.yaml
controller:
ingressClassResource:
name: nginx-public
controllerValue: k8s.io/ingress-nginx-public
ingressClass: nginx-public
electionID: ingress-nginx-public-leader
service:
externalTrafficPolicy: Local
config:
http-snippet: |
map $uri $molesignal_log_uri {
~^(/api/v1/(?:intake/heartbeats|public/calendar)/) $1<token>;
default $uri;
}
log-format-upstream: '$remote_addr [$time_local] "$request_method $molesignal_log_uri $server_protocol" $status $body_bytes_sent "$http_user_agent" $request_time'
helm upgrade --install ingress-internal ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-internal --create-namespace -f ingress-internal-values.yaml
helm upgrade --install ingress-public ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-public --create-namespace -f ingress-public-values.yaml
内网 Ingress
内网 Ingress 转发全部路径。实时查询是流式响应,关掉缓冲并放长超时:apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: molesignal-internal
namespace: molesignal
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
nginx.ingress.kubernetes.io/proxy-buffering: "off"
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
spec:
ingressClassName: nginx-internal
tls:
- hosts: [molesignal.example.com]
secretName: molesignal-tls
rules:
- host: molesignal.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: molesignal
port:
number: 5080
公网 Ingress
公网 Ingress 只列出要开放的路径,没列出的路径由控制器的默认后端返回 404。只保留实际用到的路径:apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: molesignal-public
namespace: molesignal
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
nginx.ingress.kubernetes.io/limit-rps: "20"
spec:
ingressClassName: nginx-public
tls:
- hosts: [molesignal.example.com]
secretName: molesignal-tls
rules:
- host: molesignal.example.com
http:
paths:
# 外部告警
- { path: /api/v1/intake/alerts, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 心跳打卡、Jira 的工单 Webhook、值班日历订阅
- { path: /api/v1/intake/heartbeats, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/tickets/jira, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/calendar, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# Teams 里的 Mole Agent、自动化的 API 触发、回复短信与电话按键
- { path: /api/v1/public/im/teams, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/automations, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/notify, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 推送来源
- { path: /api/v1/_kinesis_firehose, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/_cloudflare, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/_heroku, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 用户体验
- { path: /api/v1/rum/events, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/rum/replay, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 拨测 Probe Agent 上传产物
- { path: /api/v1/synthetics/artifacts, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 状态页
- { path: /status, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/status-pages, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 资源共享
- { path: /s, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /shared, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/share, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
# 状态页和分享页的网页文件
- { path: /assets, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /favicon.ico, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /favicon.svg, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
Prefix 按路径段匹配:/s 匹配 /s/<分享 Token>,不匹配 /shared;/api/v1/public/share 匹配 /api/v1/public/share/query。
Ingress 不能按方法过滤,这一层交给 MoleSignal:上面每个路径只接受它用到的方法,其他方法返回 405 或 404。注解作用于整个 Ingress 对象:要给某些路径单独设请求体上限或来源地址,把它们拆到另一个 Ingress,例如只允许 Azure 服务标记 AzureBotService 里的地址段访问 Teams 的回调:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: molesignal-public-teams
namespace: molesignal
annotations:
nginx.ingress.kubernetes.io/whitelist-source-range: "203.0.113.0/24,198.51.100.0/24"
spec:
ingressClassName: nginx-public
tls:
- hosts: [molesignal.example.com]
secretName: molesignal-tls
rules:
- host: molesignal.example.com
http:
paths:
- { path: /api/v1/public/im/teams, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
molesignal-public 里删掉,同一个路径不要出现在两个 Ingress 里。
状态页的自定义域名
状态页绑定了status.example.org 时,在公网控制器上为它加一条规则,只放行状态页的根路径、接口和网页文件:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: molesignal-status-domain
namespace: molesignal
annotations:
nginx.ingress.kubernetes.io/limit-rps: "20"
spec:
ingressClassName: nginx-public
tls:
- hosts: [status.example.org]
secretName: status-example-org-tls
rules:
- host: status.example.org
http:
paths:
- { path: /, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /history, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /uptime, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /incidents, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /api/v1/public/status-pages, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /assets, pathType: Prefix, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /favicon.ico, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
- { path: /favicon.svg, pathType: Exact, backend: { service: { name: molesignal, port: { number: 5080 } } } }
检查
确认两个 Ingress 分别被两个控制器接管,并拿到各自负载均衡的地址:kubectl -n molesignal get ingress
404,不带 Token 请求告警接入应返回 401:
curl -s -o /dev/null -w '%{http_code}\n' https://molesignal.example.com/api/v1/auth/signin
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://molesignal.example.com/api/v1/intake/alerts
_sys 组织的 设置 → 客户端 IP 识别 里,把公网控制器 Pod 所在的网段列为受信任的代理,见客户端 IP 识别。