7. Serving — image build and API service
Build a model version into a container image, run it as an API service and test it in the inference console
Required permission: DeployOPERATOR · ADMIN OperateOPERATOR · ADMIN
To serve a model on the central server, you must turn it into a container image. API services use the built image. Edge devices receive the model version files, not the image (8. Edge deployment). Use an operator or admin account.
1) Build the container image
-
Go to Model Registry (모델 레지스트리) → model → version to open the version screen. In the Container image (컨테이너 이미지) card, press Build image (이미지 빌드). If you leave Serving builder (서빙 빌더) as it is (Auto (default) (자동 (기본))), the tenant's default variant is used. You can build even if the version is not Production (Staging is fine).
Container image card on a model version — choose the serving builder and build the image -
The status moves Building (빌드 중) → Ready (준비됨). During the build, follow progress with View logs (로그 보기); if it fails, the first error line and a Build logs (빌드 로그) button appear.
A finished container image — Ready
:::
The build is done by a builder Job running in the tenant namespace. It generates a Dockerfile from the model recorded in MLflow (mlflow models generate-dockerfile), builds it with rootless BuildKit and pushes it to the platform's built-in container registry. You get a notification when it finishes.
2) Create an API service
- In the sidebar, open API Services (API 서비스) and press Create serving (서빙 생성). Set the Endpoint name (엔드포인트 이름) (it becomes a Kubernetes object name, so use lowercase letters, digits and hyphens), the Image (이미지) (only images that finished building are listed), Replicas (Replicas) and Resources (실행 자원) (GPU slots per pod; 0 means CPU), then press Create (생성). If GPU slots × replicas exceeds the slots the tenant has left, creation is rejected.
Create serving endpoint — name, image, replicas, resources - On the endpoint screen, the status moves Deploying (배포 중) → Connecting (연결 중) → Healthy (정상). In the Inference console (추론 콘솔) below, drop an image and press invocations (invocations): the detections are overlaid on the photo, and the raw JSON response is shown too. Change inference parameters such as
confwith the sliders.Inference console — a digit image sent to invocations. This is a classifier, so the label and confidence come back as JSON (a detection model would draw boxes over the photo)
The service runs in the tenant namespace as a Deployment + NodePort Service, and the platform server forwards inference requests on your behalf. Browsers and external programs do not need to know the cluster address.
POST/api/v1/serving/endpoints/{name/proxy/invocations}
- If you set Validation dataset (optional) (Validation 데이터셋 (선택)), a KPI smoke check runs once on that dataset after deployment (Validating (검증 중) status). The service is still provided even if it falls short of the threshold.
- When you are not using it, Stop (중단) it (0 pods, port kept), and Restart (재기동) it when you need it again.
- Calling from the inference console on screen requires
operatepermission (operator, admin). The API above can also be called withviewpermission.
Next: 8. Edge deployment