Slurm and training profiles

Slurm schedules Linux nodes, including supported CPU/NVIDIA/AMD resources. Phones do not run slurmd; they participate through browser task leases instead. This repository does not install Slurm or configure GPU GRES on your cluster.

Approved profiles, not remote shell commands

Administrators install reviewed batch scripts and explicitly map their names:

open-compute worker --backend slurm --name hpc-gateway \
  --credentials slurm-worker.json \
  --training-profile smoke=/shared/open-compute/examples/slurm-smoke.sh

open-compute submit --kind training \
  --payload '{"profile":"smoke","parameters":{"message":"hello"}}'

Use an actual checkout path for the example profile. Run the worker from a shared filesystem visible to the submit host and compute nodes. Slurm must provide sbatch, sacct, and scancel, configured accounting, credentials and an appropriate default partition. The worker account must be allowed to submit and cancel its own jobs.

The adapter submits a profile with input/result file paths, polls accounting, and returns the profile's result only after Slurm reports completion. A lost lease or failed job triggers scancel. A 24-hour worker-side limit bounds polling. Temporary data is cleaned after completion or failure. A hard worker crash can leave a Slurm job running until its cluster time limit; lease retries may duplicate work. Profiles must be idempotent and enforce their own time limit.

Real training

The supplied profile is a CPU transport smoke test, not a model trainer. For PyTorch/LoRA/distributed training, install a separate reviewed profile with appropriate #SBATCH resource requests, pinned environment, shared datasets, checkpoint storage and a controlled training entrypoint. Validate all parameters as data; never interpolate job JSON into shell commands or eval.

vLLM and SGLang remain inference engines. The training profile owns the training framework and launcher. Training results and checkpoints are not independently validated by the broker. The live Slurm path requires testing on your cluster; CI adapter tests do not substitute for it.