REANNZ training environment
Describe the environment a workshop needs, and press the button. The app creator turns it into a pull request and test-builds it; once a maintainer approves it, each interface becomes its own repository with its image built, ready to add to the training environment.
Lower case letters, digits and hyphens. It names the repositories and images, and cannot change later.
Shown on the dashboard, with the interface in brackets.
Each interface becomes its own app and repository, sharing the name, features, data and software.
Emulated inside each session, so learners can practise cluster workflows on the training environment.
NVIDIA cards that nvidia-smi, nvtop, Slurm and PyTorch all see, with real memory limits. There is no physical GPU: everything runs on the CPU, so timings say nothing about GPU performance. Code runs on the cards through PyTorch and Numba below; other programs' GPU (CUDA) builds cannot, and the page points them out in the Software section.
nvidia-smi
nvtop
--gpus-per-node
A small card makes out-of-memory errors cheap to teach.
torch.cuda
A single-node scheduler in the session: sbatch, squeue, scancel, sinfo, sacct, with Slurm's options and output, plus NeSI's own seff and svisit from opt-nesi-bin.
sbatch
squeue
scancel
sinfo
sacct
seff
svisit
Always on with emulated GPUs, which are requested through Slurm. Their node is gpunode001 in partition gpu.
gpunode001
gpu
module avail and module load, in terminals and batch jobs. Each conda package becomes a module rather than being on PATH.
module avail
module load
PATH
Baked into the image, and copied into each learner's home directory when a session starts. Files that already exist are never overwritten, so work survives a restart.
No data yet.
One package per line: start typing a name, and the box suggests packages. JupyterLab, VS Code and the basics (git, Python, curl, vim, nano, rsync, ...) are already there.
Versions as name=1.2. Each command is on PATH, or a module with Lmod.
name=1.2
Lines of a requirements file.
RStudio app only.
VS Code app only, from Open VSX.
A training environment node fits about fifteen 2 CPU, 4 GB sessions.
Bump it to release a change to an existing app.
The cluster can only pull public images.
The versions of Python Mahuika has, from conda-forge, in every interface. Ubuntu's own is quicker to build. The Slurm and GPU emulators need 3.10 or newer.
Added to every image after the software. Reviewers will read these.
Run with bash in the home directory, just before the interface starts.
Paste an app spec, or choose a file. This replaces what is in the form.
Choose an app from apps/. Your request will change it.
apps/