Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Introduction to EXO and Local AI Clustering
- Overview of the EXO framework and the exo-explore ecosystem
- Contrasting centralized cloud inference with distributed local inference
- Architecture breakdown: libp2p device discovery, MLX backend, dashboard, and API layers
- Hardware prerequisites: Apple Silicon (M3 Ultra, M4 Pro/Max), Thunderbolt 5, and shared storage
Installing EXO on macOS
- Configuring Xcode, Metal ToolChain, and essential macOS dependencies
- Installing uv, Node.js, and the Rust nightly toolchain
- Setting up the pinned macmon fork for Apple Silicon monitoring
- Cloning the repository and constructing the dashboard using npm
- Executing EXO from source and confirming the dashboard at localhost:52415
Installing EXO on Linux
- Installing dependencies using apt or Homebrew on Linux systems
- Configuring uv, Node.js 18+, and Rust nightly
- Building the dashboard and operating EXO in CPU-only mode
- Understanding the directory layout: XDG Base Directory paths for configuration, data, cache, and logs
Automatic Device Discovery and Cluster Formation
- Explaining libp2p-based auto-discovery mechanisms across local networks
- Setting up custom namespaces via EXO_LIBP2P_NAMESPACE to ensure cluster isolation
- Confirming node membership through the dashboard’s cluster view
- Addressing discovery failures and network segmentation challenges
Enabling RDMA over Thunderbolt 5
- RDMA architecture and the claim of 99 percent latency reduction
- Activating RDMA in macOS Recovery mode using rdma_ctl
- Cable specifications and port topology constraints on Mac Studio
- Ensuring consistent macOS versions across all cluster nodes
- Troubleshooting RDMA discovery and DHCP configurations
Deploying Frontier Models
- Utilizing the dashboard to load and shard DeepSeek v3.1, Qwen3-235B, and Llama family models
- Previewing instance placements via the /instance/previews API endpoint
- Creating model instances using pipeline or tensor-parallel sharding
- Configuring custom model cards sourced from the HuggingFace hub
Monitoring and Troubleshooting
- Analyzing EXO logs and interpreting distributed tracing
- Assessing cluster health within the dashboard’s cluster view
- Diagnosing worker node failures and reconnection dynamics
- Leveraging EXO_TRACING_ENABLED for performance bottleneck analysis
Cluster Maintenance and Updates
- Updating EXO binaries and following dashboard rebuild procedures
- Migrating model caches and managing pre-downloaded models over NFS
- Gracefully removing nodes and rebalancing workloads
Requirements
- A solid grasp of networking fundamentals, including IP addressing, subnetting, and firewall management
- Practical experience with command-line administration on macOS or Linux
- Knowledge of Python package management via pip or uv, as well as Node.js tools
Target Audience
- System administrators
- DevOps engineers
- AI infrastructure architects tasked with on-premise LLM deployment
