Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Preparing Machine Learning Models for Production Deployment
- Packaging models using Docker
- Exporting models from TensorFlow and PyTorch
- Considerations for versioning and storage
Serving Models on Kubernetes
- Overview of inference server architectures
- Deploying TensorFlow Serving and TorchServe
- Configuration of model endpoints
Optimizing Inference Performance
- Strategies for batching requests
- Managing concurrent request handling
- Tuning for latency and throughput
Autoscaling Machine Learning Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Setting up GPU-enabled nodes
- Overview of the NVIDIA device plugin
- Defining resource requests and limits for ML tasks
Strategies for Model Rollout and Release
- Blue/green deployment techniques
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for Production ML
- Key metrics for inference workloads
- Best practices for logging and tracing
- Dashboard creation and alerting setups
Security and Reliability Factors
- Securing access to model endpoints
- Implementing network policies and access controls
- Ensuring high availability standards
Summary and Future Directions
Requirements
- A solid understanding of containerized application workflows
- Hands-on experience with Python-based machine learning models
- Familiarity with core Kubernetes concepts
Target Audience
- ML Engineers
- DevOps Engineers
- Platform Engineering Teams
Testimonials (1)
I like the Kubernetes part of the training as I had some tasks with K8s pods and ports and it was nice to get a better understanding now. Thank you!