kubernetes cluster 19 part 2

19th attempt to get a k8s setup working

Goals

• ingress
• google sign in protected endpoints

tldr

config: seankhliao/kluster @ v0.19.0

use with:

• make create-cluster
• make decrypt // or create appropriate secret files
• kubectl apply -k . // maybe repeat a few times if things fail

ingress / sign in

• traefik ingress controller
• pomerium auth handler

ingress

traefik still easier to setup than ambassador or contour

• hostPort because DNS is already pointed at nodes
• consider running as daemonset
• KubernetesCRD as the only provider

set default TLS cert

1apiVersion: traefik.containo.us/v1alpha1
2kind: TLSStore
3metadata:
4  name: default
5spec:
6  defaultCertificate:
7    secretName: api-seankhliao-com-tls

auth

Running pomerium because I can't find anything else in the same space?

• all in one mode: does the services mode even work with forward auth?
• maybe config should be in a ConfigMap and inject idp_* and *_secret through the env?

pomerium config

 1insecure_server: true
 2grpc_insecure: true
 3address: :80
 4
 5authenticate_service_url: https://auth.api.seankhliao.com
 6forward_auth_url: http://pomerium.networking.svc.cluster.local
 7
 8idp_provider: "google"
 9idp_client_id: CHANGE_ME
10idp_client_secret: CHANGE_ME
11
12shared_secret: CHANGE_ME
13cookie_secret: CHANGE_ME
14
15metrics_address: ":9090"
16
17tracing_provider: jaeger
18tracing_debug: true
19tracing_jaeger_agent_endpoint: jaeger-agent.monitor.svc.cluster.local:6831
20
21policy:
22  - from: https://traefik.api.seankhliao.com
23    to: http://example.com
24    allowed_users:
25      - admin@example.com

IngressRoute config

• route both the authenticate service and catch all the /.pomerium/ paths (css assets)

 1apiVersion: traefik.containo.us/v1alpha1
 2kind: IngressRoute
 3metadata:
 4  name: pomerium
 5spec:
 6  entryPoints:
 7    - https
 8  routes:
 9    - kind: Rule
10      match: Host(`auth.api.seankhliao.com`)
11      services:
12        - kind: Service
13          name: pomerium
14          port: 80
15    - kind: Rule
16      match: PathPrefix(`/.pomerium/`)
17      priority: 100
18      services:
19        - kind: Service
20          name: pomerium
21          port: 80
22  tls: {}

Use with Middleware, ex for traefik's dashboard

 1apiVersion: traefik.containo.us/v1alpha1
 2kind: IngressRoute
 3metadata:
 4  name: traefik
 5spec:
 6  entryPoints:
 7    - https
 8  routes:
 9    - kind: Rule
10      match: Host(`traefik.api.seankhliao.com`)
11      middlewares:
12        - name: auth-traefik
13      services:
14        - kind: Service
15          name: traefik
16          port: 9000
17  tls: {}
18---
19apiVersion: traefik.containo.us/v1alpha1
20kind: Middleware
21metadata:
22  name: auth-traefik
23spec:
24  forwardAuth:
25    address: http://pomerium.networking.svc.cluster.local/?uri=https://traefik.api.seankhliao.com