Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Offline Deployment of EXO
- Utilizing EXO_OFFLINE to block runtime internet connectivity
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from verified internal sources
- Validating model weight integrity via SHA-256 checksums and signed model cards
- Executing EXO on air-gapped networks without reliance on HuggingFace services
Dashboard and API Access Management
- Setting up and configuring reverse proxies (nginx, Caddy) for TLS termination
- Establishing role-based access control for the EXO dashboard and REST API
- Storing API authentication secrets using macOS Keychain or Linux pass
- Limiting administrative endpoints to designated IP address ranges
Cluster Segmentation and Network Protection
- Partitioning EXO clusters using EXO_LIBP2P_NAMESPACE and VLANs
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO-specific ports
- Mitigating unauthorized device discovery and rogue node integration
- Encrypting libp2p node-to-node traffic when RDMA is unavailable
Model Governance and Origin Tracking
- Establishing an internal model registry containing approved models and metadata
- Tagging and versioning quantized weights (4-bit, 8-bit) in conjunction with original checkpoints
- Restricting loading to specific HuggingFace repositories or internal artifacts only
- Documenting model lineage, licensing terms, and permissible use guidelines
Audit Logging and Regulatory Compliance
- Directing EXO logs to immutable audit trails (SIEM, WORM storage)
- Linking API call logs with user identity and temporal data
- Recording events related to model instance creation, deletion, and inference requests
- Producing periodic compliance reports for internal and external reviewers
Threat Assessment and Incident Management
- Recognizing risks: data leakage via model outputs, prompt injection, and side-channel exposures
- Deploying prompt monitoring and content filtering systems
- Developing incident response playbooks for cluster compromise situations
- Quarantining impacted nodes, retaining forensic evidence, and restoring clean environments
Physical Security and Hardware Constraints
- Safeguarding Thunderbolt ports from unauthorized RDMA cable attachments
- Leveraging secure enclaves and Apple Silicon hardware verification where relevant
- Regulating physical access to clustered Macs and shared storage units
- Recording hardware lifecycle and disposal procedures
Regulatory Implications
- Aligning EXO deployments with GDPR, HIPAA, and SOC 2 standards
- Ensuring data residency by executing inference locally
- Documenting supply chain risks associated with vendors (MLX, EXO, model weights)
- Preparing for AI governance regulations such as EU AI Act Article 53
Requirements
- Practical experience with EXO or alternative local LLM runtimes
- Knowledge of Unix file system permissions and network access control lists (ACLs)
- Familiarity with TLS/SSL certificate administration and fundamental encryption concepts
Target Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators managing sensitive information
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
