Prepare k3s and GPUs | Geo-MLOps Manual

Training, image builds and serving run as k3s pods. On this page you install k3s, let pods use the GPU, and let the app count GPUs as "slots". Run every command on the server; the source is assumed to be in /opt/geo-mlops.

1. NVIDIA driver and Container Toolkit

k3s checks for nvidia-container-runtime when it starts and registers the nvidia runtime with containerd. So install the driver and the NVIDIA Container Toolkit before k3s (follow NVIDIA's official documentation). If k3s is already running, run sudo systemctl restart k3s after installing the Toolkit so it picks it up.

nvidia-smi                       # check the driver
which nvidia-container-runtime   # check the Toolkit

2. Install k3s

Install k3s (the official install script).
```
curl -sfL https://get.k3s.io | sh -
```
Check that the node is Ready and that the kubeconfig exists. The app container reads this file as is.
```
sudo k3s kubectl get nodes
ls -l /etc/rancher/k3s/k3s.yaml
```

Check whether RuntimeClass nvidia exists. If not, create it.

sudo k3s kubectl get runtimeclass nvidia || \
cat <<'EOF' | sudo k3s kubectl apply -f -
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: nvidia
handler: nvidia
EOF

3. NVIDIA device plugin

A RuntimeClass alone can put a GPU into a pod, but then the cluster does not know GPU occupancy. Two pods land on one GPU and both die from out-of-memory errors. The device plugin makes the node advertise GPUs as the nvidia.com/gpu resource, so pods wait in Pending when there is no room. The app's GPU slots screen also counts this advertised value.

There are two manifests. Apply only one of them (they use the same DaemonSet name).

File	When
`scripts/serving/nvidia-device-plugin.yml`	One GPU = one slot. When you have several GPUs and giving each pod one GPU is enough
`scripts/serving/nvidia-device-plugin-timeSlicing.yml`	Advertises one GPU as several slots (8 by default). When you have one or two GPUs but need to run several training and serving workloads together

Apply one of them. All you need is the kubeconfig.

sudo k3s kubectl apply -f /opt/geo-mlops/scripts/serving/nvidia-device-plugin-timeSlicing.yml

Check the number of GPUs advertised per node (with time-slicing, number of GPUs × 8).

sudo k3s kubectl get nodes -o custom-columns='NAME:.metadata.name,ALLOC:.status.allocatable.nvidia\.com/gpu'

Sign in to the app as the global administrator and open the GPU slots (GPU 슬롯) screen in the system console. You are done when the device plugin is Ready and the total slots match the number above.

GPU slots — GPUs advertised by each node and the device plugin state

To change replicas (default 8), edit the ConfigMap, apply it again, and always roll out the DaemonSet. Otherwise the change does not take effect.

sudo k3s kubectl apply -f /opt/geo-mlops/scripts/serving/nvidia-device-plugin-timeSlicing.yml
sudo k3s kubectl -n kube-system rollout restart ds/nvidia-device-plugin-daemonset

4. Let k3s pull from the container registry

The app's embedded registry (/v2) is plain HTTP. Images travel two ways.

Direction	Who	Default address	Setting
push	BuildKit inside the build pod	`10.42.0.1:10000`	insecure is already on for BuildKit (`GEO_MLOPS_SERVING_REGISTRY_INSECURE=true`)
pull	The node's containerd	`localhost:10000`	containerd treats `localhost` as insecure — nothing to do on a single node

So a one-server setup does not need registries.yaml. Only if you changed the pull address to something other than localhost (for example, you have several nodes and set GEO_MLOPS_SERVING_REGISTRY_PULL to host:port), register that address as insecure in /etc/rancher/k3s/registries.yaml on each node.

# /etc/rancher/k3s/registries.yaml — replace <registry-host>:10000 with the real pull address
mirrors:
  "<registry-host>:10000":
    endpoint: ["http://<registry-host>:10000"]
configs:
  "<registry-host>:10000":
    tls:
      insecure_skip_verify: true

sudo systemctl restart k3s    # k3s reads registries.yaml when it starts

Next: Required security settings

1. NVIDIA driver and Container Toolkit#

2. Install k3s#

3. NVIDIA device plugin#

4. Let k3s pull from the container registry#

1. NVIDIA driver and Container Toolkit

2. Install k3s

3. NVIDIA device plugin

4. Let k3s pull from the container registry