Pretrained weights

The starting weights of a training run come from one of two places. You choose them together with the model in wizard step 3.

Catalog weights — the mirror is an optimization#

The platform mirrors the public checkpoints of catalog models to object storage. This covers the YOLO · RF-DETR · Pointcept variants whose checkpoint is registered in the catalog. Before the training container starts, the checkpoint is placed where the framework looks: the working directory (/geo) for YOLO, RF_HOME (/geo/work/weights) for RF-DETR, and /geo/work/weights for Pointcept. If the file is already there, the framework skips the download, so with a filled mirror training runs even in an air-gapped network.

If the mirror is empty, the first training run downloads from the internet and fills the mirror. The log of the preparation container (stage-dataset) looks like this. This log does not appear on the training detail screen; the operator reads it with kubectl -n tenant-<code> logs <pod> -c stage-dataset.

-> weights: mirror miss (… yolo26n.pt: … Not Found), trying upstream
-> weights: mirrored s3://mlflow/runtime-weights/yolo/…/yolo26n.pt
-> weights: yolo26n.pt 5MB from upstream in 1s

From the next training run on, it reads -> weights: yolo26n.pt 5MB from mirror in 0s.

The system administrator manages the mirror (pre-filling, clearing) — see the Administration guide.

Weights uploaded by the tenant#

Use these to continue training from your own checkpoint. Go to the Pretrained weights (사전학습 가중치) link at the top of the Training (학습) screen (address /training/weights) and upload a .pt/.pth file with Upload weights (가중치 올리기).

• When uploading, choose the framework and model variant. Only variants with a public checkpoint in the catalog can be chosen. A checkpoint is tied to an architecture, so the wizard shows only weights for the chosen variant, and the server checks again at submission.
• After the upload, the weights cannot be chosen while the server assembles and inspects the file. Once the status is ready, they appear under Starting weights (시작 가중치) in wizard step 3.

Trainer rule — always use pretrained.path, and fail if it is missing#

If source is tenant, the trainer must start from the file that pretrained.path in experiment.json points to. If it cannot read that file, it must end in failure. If it silently falls back to public weights, a run trained on weights nobody chose is reported as a success.

import json, os, sys
from pathlib import Path

config = json.loads(Path(os.environ["GEO_CONFIG"]).read_text())
pretrained = config.get("pretrained") or {"source": "catalog"}

if pretrained["source"] == "tenant":
    weights = Path(pretrained["path"])
    if not weights.is_file():
        sys.exit(f"selected pretrained weights not found: {weights}")
    model = load_from_checkpoint(weights)      # must start from this file
else:
    model = load_default_pretrained()          # framework default (from the mirror if present)

If the weight file cannot be placed, the preparation container fails first and the training container never starts. On screen it looks like the known display issue in Common failures. The trainer only ever sees the case "the file is there". The check above is a safeguard that writes the rule down in code.