1 files changed, 77 insertions(+), 6 deletions(-)

M README.md

@@ 86,6 86,54 @@ that a node's hostname is not mapped to localhost. Example:
 
 ## Network
 
+### Service network
+
+Network:
+
+- 10.32.0.0./24
+
+### Public virtual service IPs
+
+We have to handle SSH traffic to multiple destinations. SSH is inherently hard
+to route (no SNI, host header, or such). Hence, we will need dedicated IPs for
+certain services (git, hg?, build runner). By far the simples solution is to
+maintain a mapping by hand.
+
+Each service should get at least two IPs for redundancy. Each IP is manually
+assigned to a cluster memeber. The range and numbering scheme is to be
+determined, but for example:
+
+- git.sr.ht
+  - 46.23.81.200 (assigned to sakuya2)
+  - 46.23.81.201 (assigned to sakuya3)
+- k8s.runners.sr.ht
+  - 46.23.81.202 (assigned to sakuya2)
+  - 46.23.81.203 (assigned to sakuya3)
+
+DNS has to be configured manually. Each IP has to be brought up on the host's
+main network interface, in addition to their host IP. A Kubernetes service can
+then be declared like this:
+
+```
+apiVersion: v1
+kind: Service
+metadata:
+  name: buildsrht-ssh
+spec:
+  selector:
+    app: buildsrht-ssh
+  ports:
+    - protocol: TCP
+      port: 22
+      targetPort: 22
+  externalIPs:
+    - 46.23.81.202
+    - 46.23.81.203
+```
+
+This will cause kube-proxy to intercept and handle traffic destined for the
+specified IP/port pairs.
+
 ### Pod network
 
 Network: 


@@ 100,17 148,40 @@ Subnets:
 
 ### Routing
 
-Add routes on each node for the service network and the pod network parts of
-the other nodes. E.g. on sakuya2, in `/etc/network/interfaces`:
+Each node must have the following configured:
+
+- Its host address
+- Any virtual service IPs the host should handle (see above)
+- A route to the service network via the cni0 interface (.1 of the host's pod
+  network subnet)
+- A route to other hosts' pod network subnets via the hosts' main addresses
+
+E.g. on sakuya2, in `/etc/network/interfaces` (assuming the virtual service IPs
+provided in the example above):
 
 ```
+auto eth0
+iface eth0 inet static
+    hostname sakuya2
+    address 46.23.81.134
+    netmask 255.255.255.128
+    gateway 46.23.81.129
+
+iface eth0 inet static
+    address 46.23.81.200
+    netmask 255.255.255.128
+
+iface eth0 inet static
+    address 46.23.81.202
+    netmask 255.255.255.128
+
 auto eth1
 iface eth1 inet static
-	address 10.0.0.134
-	netmask 255.255.255.0
-	up ip route add 10.200.132.0/24 via 10.0.0.132
-	up ip route add 10.200.135.0/24 via 10.0.0.135
+    address 10.0.0.134
+    netmask 255.255.255.0
     up ip route add 10.32.0.0/24 via 10.200.134.1
+    up ip route add 10.200.132.0/24 via 10.0.0.132
+    up ip route add 10.200.135.0/24 via 10.0.0.135
 ```
 
 ## Installation