API services
Create central serving endpoints, check their status and performance, and call them directly from the inference console.
Required permission: View
This screen runs a model version's serving image on the central cluster and offers it as an HTTP inference API (endpoint). An endpoint runs as a Kubernetes Deployment, and the platform forwards requests to it. Every member can view; creating and deleting endpoints needs deploy, and stopping, restarting and calling inference need operate.
Before you create an endpoint, build the serving image on the version screen of the Model registry.
Endpoint list
| # | Area | Description |
|---|---|---|
| ① | Create serving (서빙 생성) | Endpoint creation dialog (deploy). |
| ② | Go to model registry (모델 레지스트리로) | Shortcut shown when there are no endpoints. |
Endpoints appear as cards; select one to open its detail. A card shows the name and status, RPS, P95(ms), Error(%), Ready, the model version and the number of GPU slots per pod. A metric that could not be measured shows —. If the validation KPI check finds a success rate below 95% or a p95 latency above 2,000 ms, a KPI not met (KPI 미달) chip is attached. The KPI is only recorded; the service keeps running. The list refreshes every 5 seconds while any endpoint is transitioning, otherwise every 30 seconds.
On an installation with serving turned off, the list shows example data under the notice "Serving is not configured, showing example data (서빙이 설정되지 않아 예시 데이터를 표시합니다)".
Create an endpoint
- Press Create serving (서빙 생성).
- Set the endpoint name (엔드포인트 이름) (lowercase letters, digits and hyphens, at most 63 characters — it becomes a Kubernetes object name), image (이미지) (only model versions whose build has finished are listed), Replicas, resources (실행 자원) (CPU or GPU slots wired to the tenant; each pod takes its own slots, so you need slots × replicas) and Validation dataset (Validation 데이터셋) (optional — if set, a KPI smoke test (a short functional check) with 20 samples runs after deployment), then press Create (생성). A tenant with no GPU slots wired can choose CPU only. If the slots cannot be obtained, the deployment waits (Pending) instead of failing.
Create a serving endpoint — name, built image, replicas, resources and validation dataset - The endpoint detail opens, and the Rollout progress (롤아웃 진행) card shows deploy → connect → validate → run (배포 → 연결 → 검증 → 가동). The card is shown only while transitioning (deploying, connecting, validating), and the screen refreshes every 5 seconds meanwhile.
Endpoint detail and inference console
The detail screen shows the status and the cause of any problem, the model version and creation time, the exposed port (NodePort) and the ready replicas (ready/replicas), the serving image (a link to the container registry), Stop (중단) and, once stopped, Restart (재기동) (operate) and Delete (삭제) (deploy; you must type the name once more), and the inference console (추론 콘솔).
Expected input (예상 입력) at the top of the inference console is the input shape read from the model signature. Enter input that matches the model's input format (image upload, point cloud (PLY) upload, text or table, or a JSON body) and call real inference with invocations (invocations). If the format is unknown, only the JSON body editor appears. ping (ping) checks that it is alive; version (version) shows the version being served. The response appears as-is with its HTTP status code. For a classification model, the predicted class and bars for the top probabilities are also shown. For a detection model, results are drawn over the image as boxes, masks and labels; filter them with the confidence threshold slider. A stopped endpoint cannot be called (press Restart (재기동) first).
A point cloud model takes one PLY file (extract archives and upload files one at a time). The parameters are grid_size (0 uses the training value) and include_ply, plus min_score for an instance segmentation model. Results come with a 3D viewer. Semantic segmentation shows a table of point counts per class; instance segmentation (PointGroup) shows the number of objects found, the number of unassigned points and a list of objects (point count, score), with unassigned points in grey. Download the result with Download result PLY (결과 PLY 받기). If no object is found, lower min_score or check grid_size.
Below the console are RPS and Latency charts for the last hour (only when request metrics exist) and the status code distribution (상태코드 분포) (2xx, 4xx, 5xx). With no requests it shows "No requests aggregated (집계된 요청이 없습니다.)".
Statuses and badges
| Badge | Value | Meaning |
|---|---|---|
| Deploying (배포 중) | DEPLOYING | Creating the Deployment and starting pods |
| Connecting (연결 중) | CONNECTING | Pods are up; the platform is checking the connection |
| Validating (검증 중) | VALIDATING | Running the KPI smoke test with the validation dataset |
| Available (정상) | AVAILABLE | Can accept requests |
| Degraded (성능 저하) | DEGRADED | The platform's periodic check (/ping, every 30 seconds by default) failed 3 times in a row — returns to Available after one success |
| Stopped (중단) | STOPPED | Stopped by a user (0 pods, NodePort kept). Restart (재기동) resumes from Connecting |
| Failed (실패) | FAILED | Failed at some step |
When a pod is unhealthy, the card and the detail show the cause reported by the cluster and the restart count, whatever the status (even while connecting) — for example "The container is restarting repeatedly (컨테이너가 반복 재시작 중)" (CrashLoopBackOff), "The image cannot be pulled (이미지를 받지 못하는 중)" (ImagePullBackOff, ErrImagePull), "Container configuration error (컨테이너 설정 오류)" (CreateContainerConfigError), "Deployment progress has stalled (배포 진행이 멈춤)" (ProgressDeadlineExceeded).