Déployer l'apprentissage profond : quantification, service et IA en périphérie

Aperçu

"Production Deep Learning:

Inference, Quantization & Edge Deployment is designed for ML engineers and developers who want to master the full deployment lifecycle — from compressing and quantizing models to serving them at scale using vLLM, Triton, ONNX, and Llama.cpp. Module 1 covers model compression fundamentals, including pruning, distillation, and INT8/INT4 quantization using AWQ and GPTQ, with a focus on the accuracy–latency tradeoff.

Module 2 dives into high-throughput serving architectures, exploring vLLM's PagedAttention, NVIDIA Triton, TensorRT, and scaling inference across GPU clusters with autoscaling patterns. Module 3 focuses on CPU and edge deployment using ONNX Runtime, GGUF, and Llama.cpp, plus multimodal inference with CLIP and LLaVA on resource-constrained devices.

Module 4 is a capstone project where you'll quantize a fine-tuned LLM, build a production API with vLLM, benchmark performance, and containerize your model with Docker for cloud and edge deployment. By the end of this course, you will:

- Apply INT4/INT8 quantization techniques (AWQ, GPTQ, GGUF) to compress LLMs for production - Deploy high-throughput inference servers using vLLM, Triton, and ONNX Runtime - Run optimized models on GPU, CPU, and edge devices using Llama.cpp and TensorRT - Build, benchmark, and containerize an end-to-end production-ready inference API" Disclaimer:

This is an independent educational resource created by Board Infinity for informational and educational purposes only.

This course is not affiliated with, endorsed by, sponsored by, or officially associated with any company, organization, or certification body unless explicitly stated. The content provided is based on industry knowledge and best practices but does not constitute official training material for any specific employer or certification program.

All company names, trademarks, service marks, and logos referenced are the property of their respective owners and are used solely for educational identification and comparison purposes.

Programme

• Compression de Modèle, Quantification & Optimisation de Latence
Apprenez les fondamentaux de la compression de modèle, le profilage de mémoire, et les techniques modernes de quantification INT8/INT4 y compris AWQ et GPTQ pour optimiser les modèles pour l'inférence en production.
• Service à Haut Débit - vLLM, PagedAttention & Triton
Maîtrisez les moteurs de service de qualité professionnelle, notamment vLLM avec PagedAttention et NVIDIA Triton pour l'évolutivité de l'inférence à travers les GPU et les nœuds.
• ONNX, Llama.cpp & Déploiement Edge / CPU
Exportez des modèles vers ONNX pour l'interopérabilité, déployez des LLMs sur CPU et dispositifs Edge avec Llama.cpp et GGUF, et construisez des pipelines multimodaux avec CLIP et LLaVA.
• Projet Final - L'API Prête pour l'Edge (Quantification pour Service et Benchmark)
Appliquez tous les concepts du cours dans un projet final pour quantifier un modèle affiné, le servir via vLLM, le mesurer, et l'emballer pour le déploiement cloud et edge.

---

Enseigné par

Board Infinity

---

Matières

Programming