Apprentissage profond : architectures avancées et entraînement GPU efficace

Aperçu

Master advanced deep learning architectures and efficient training techniques using PyTorch Lightning, timm, ConvNeXt, Vision Transformers, RoPE, SwiGLU, RMSNorm, and Weights & Biases. This course equips you to design, train, and benchmark modern backbones on limited GPU hardware for real-world production use.

Module 1 introduces modern backbone architectures, tracing the evolution from ResNets to ConvNeXt and Vision Transformers, covering patch embeddings, multi-head self-attention, and position encodings. Module 2 dives into training dynamics and stabilization techniques including RMSNorm, SwiGLU activations, and Rotary Position Embeddings (RoPE) for stable, scalable training.

Module 3 focuses on efficient training on limited GPUs using mixed precision (FP16/BF16), gradient accumulation, efficient data pipelines, and distributed training with DDP/FSDP in Lightning. Module 4 covers experiment tracking with TensorBoard and W&B, profiling FLOPs and throughput, and a hands-on ViT vs.

CNN Showdown project with fine-tuning in timm. By the end of this course, you will:

- Build and fine-tune ConvNeXt and Vision Transformer backbones using PyTorch Lightning and timm - Apply RMSNorm, SwiGLU, and RoPE to stabilize and scale deep transformer training - Implement mixed precision, gradient accumulation, and DDP/FSDP for efficient multi-GPU training - Design controlled CNN vs.

ViT experiments with W&B tracking and PyTorch profiling Disclaimer:

This is an independent educational resource created by Board Infinity for informational and educational purposes only. This course is not affiliated with, endorsed by, sponsored by, or officially associated with any company, organization, or certification body unless explicitly stated.

The content provided is based on industry knowledge and best practices but does not constitute official training material for any specific employer or certification program. All company names, trademarks, service marks, and logos referenced are the property of their respective owners and are used solely for educational identification and comparison purposes.

Programme

• Architectures Modernes de Backbone (ConvNeXt & Vision Transformers)
Explorez l'évolution des backbones de deep learning des CNN classiques vers ConvNeXt et Vision Transformers, en comprenant leur mécanique, compromis et pertinence dans l'industrie.
• Dynamiques de Formation & Techniques de Stabilisation
Apprenez les techniques modernes de stabilisation et d'efficacité, incluant RMSNorm, activations SwiGLU, et Embeddings de Position Rotative qui alimentent les transformers à la pointe de la technologie.
• Formation Efficace avec des GPUs Limités
Maîtrisez des techniques pratiques pour entraîner de grands modèles sur du matériel limité, y compris la précision mixte, l'accumulation de gradients et les stratégies d'entraînement distribuées.
• Expérimentation, Suivi & Projet ViT vs CNN Showdown
Apprenez à suivre les expériences de manière professionnelle et appliquez tous les concepts du cours dans un projet pratique ViT vs CNN Showdown en utilisant le fine-tuning avec timm et PyTorch Lightning.

---

Enseigné par

Board Infinity

---

Matières

Computer Science