Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Foundations of Parameter-Efficient Fine-Tuning (PEFT)
- Drivers and constraints associated with full fine-tuning
- PEFT overview: objectives and advantages
- Real-world industrial applications and scenarios
LoRA (Low-Rank Adaptation)
- Core concepts and intuitive understanding of LoRA
- Implementation using Hugging Face and PyTorch
- Practical exercise: Fine-tuning a model via LoRA
Adapter Tuning
- Operational mechanics of adapter modules
- Integration strategies for transformer-based architectures
- Practical exercise: Applying Adapter Tuning to a transformer model
Prefix Tuning
- Leveraging soft prompts for model adaptation
- Comparative strengths and limitations against LoRA and adapters
- Practical exercise: Executing Prefix Tuning on an LLM task
Assessment and Comparison of PEFT Methods
- Key metrics for measuring performance and efficiency
- Balancing training speed, memory consumption, and accuracy
- Conducting benchmarks and interpreting outcomes
Deployment of Fine-Tuned Models
- Processes for saving and retrieving fine-tuned models
- Considerations for deploying PEFT-based models
- Incorporation into production applications and workflows
Advanced Practices and Expansions
- Synergizing PEFT with quantization and distillation techniques
- Application in low-resource and multilingual environments
- Emerging trends and areas of active research
Requirements
- Knowledge of machine learning principles
- Practical experience with large language models (LLMs)
- Proficiency in Python and PyTorch
Target Audience
- Data scientists
- AI engineers
