AI Runtime CLI examples

Preview

The following examples are complete, end-to-end workloads you submit from the air CLI with air run -f <config>.yaml. Each includes the workload YAML, any bootstrap scripts, and the full code. Start with the quickstart if you haven't submitted a run before.

- Multi-node LLM fine-tuning with FSDP
- Supervised fine-tuning of Llama-3.1-8B across 16 H100 GPUs (2 nodes) using torchrun and PyTorch Fully Sharded Data Parallel (FSDP). Logs to MLflow and checkpoints to a Unity Catalog volume.
- Ray hello world
- Minimal working examples for Ray Core, Ray Train, Ray Data, and Ray Tune, including the shared bootstrap pattern for running Ray clusters on AI Runtime.
- Distributed training with Ray Train
- Distributed data-parallel fine-tuning with Ray Train's TorchTrainer across 8 H100 GPUs on a single node, with one worker per GPU.
- Batch inference with Ray Data and vLLM
- Offline LLM batch inference with Ray Data and vLLM across 8 H100 GPUs on a single node, running one vLLM replica per GPU and writing results to a Unity Catalog volume as Parquet.
- Hyperparameter search with Ray Tune
- LoRA hyperparameter search for Qwen2.5-0.5B with Ray Tune across 4 1xA10 nodes, running one trial per GPU and stopping underperforming trials early with the ASHA scheduler.