Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Utilize EXO_OFFLINE to prevent runtime internet access.
- Pre-load models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors.
- Verify model weight integrity using SHA-256 checksums and signed model cards.
- Run EXO in air-gapped networks without dependencies on HuggingFace.
Dashboard and API Access Control
- Install and configure reverse proxies (nginx, Caddy) with TLS termination.
- Implement role-based access control for the EXO dashboard and REST API.
- Use macOS keychain or Linux pass to store secrets for API authentication.
- Restrict administrative endpoints to specific source IP ranges.
Cluster Isolation and Network Security
- Segment EXO clusters using EXO_LIBP2P_NAMESPACE and VLANs.
- Configure host firewalls (macOS application firewall, iptables, nftables) for EXO ports.
- Prevent unauthorized device discovery and rogue node injection.
- Encrypt libp2p traffic between nodes when RDMA is unavailable.
Model Governance and Provenance
- Build an internal model registry with approved model lists and metadata.
- Tag and version quantized weights (4-bit, 8-bit) alongside source checkpoints.
- Enforce loading only from specific HuggingFace repositories or internal artifacts.
- Document model lineage, license terms, and acceptable use policies.
Audit Logging and Compliance
- Configure EXO log forwarding to immutable audit trails (SIEM, WORM storage).
- Correlate API call logs with user identity and timestamp.
- Capture events related to model instance creation, deletion, and inference requests.
- Generate periodic compliance reports for internal and external auditors.
Threat Modeling and Incident Response
- Identify threats such as data exfiltration via model outputs, prompt injection, and side-channel leaks.
- Implement prompt monitoring and content filtering pipelines.
- Create incident response runbooks for cluster compromise scenarios.
- Isolate affected nodes, preserve forensic logs, and rebuild clean environments.
Physical Security and Hardware Boundaries
- Secure Thunderbolt ports against unauthorized RDMA cable connections.
- Utilize secure enclaves and Apple Silicon hardware attestation where applicable.
- Control physical access to clustered Macs and shared storage.
- Document hardware lifecycle and decommissioning procedures.
Regulatory Considerations
- Map EXO deployments to GDPR, HIPAA, and SOC 2 requirements.
- Maintain data residency by keeping inference on-premises.
- Document vendor supply-chain risks (MLX, EXO, model weights).
- Prepare for AI governance frameworks such as EU AI Act Article 53.
Requirements
- Experience with EXO or another local LLM runtime.
- Understanding of Unix filesystem permissions and networking ACLs.
- Familiarity with TLS/SSL certificate management and encryption basics.
Audience
- Security engineers.
- Compliance officers.
- AI infrastructure administrators responsible for handling sensitive data.
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
