Deep Learning: Advanced Backbones and Efficient GPU Training
Explore modern CNN and Vision Transformer backbones, stabilize transformer training with RMSNorm, SwiGLU and RoPE, and learn techniques for more efficient GPU training - then compare models properly.
Deep Learning: Advanced Backbones and Efficient GPU Training
By Board Infinity · Offered on Coursera
- 39 video lessons · 16 assignments
- ViT vs CNN Showdown - hands-on project
- Shareable certificate on completion
- English - 9 subtitle languages
Modern backbones
ResNet to ConvNeXt and Vision Transformers
Training dynamics
RMSNorm, SwiGLU and RoPE
GPU efficiency
Mixed precision, accumulation, DDP/FSDP
Experiment tracking
TensorBoard, W&B and PyTorch Profiler
Design, train and benchmark modern backbones.
An intermediate course on modern backbones and efficient training on limited GPU hardware.
Deep Learning: Advanced Backbones and Efficient GPU Training traces the move from classical CNNs and ResNets to ConvNeXt and Vision Transformers - patch embeddings, multi-head self-attention and position encodings.
It then covers the components behind stable transformer training - RMSNorm, SwiGLU and RoPE - and how to train on limited hardware with mixed precision, gradient accumulation and DDP/FSDP in PyTorch Lightning.
The last module tracks and compares experiments with TensorBoard, W&B and PyTorch Profiler, ending in a hands-on ViT vs. CNN Showdown with fine-tuning in timm.
Course page: 2 weeks at 10 hours a week, updated May 2026, taught in English with 9 subtitle languages. The specialization page separately lists this course as Course 1, 21 hours.
For people already training models who want modern backbones.
Coursera recommends working knowledge of PyTorch, CNNs and standard training loops. Familiarity with transformers is helpful but not mandatory.
Beyond ResNet
Deeper knowledge of modern vision backbones: ConvNeXt and Vision Transformers.
CNN vs. ViT trade-offs
How each backbone works, and how to choose between them.
Efficient training
Mixed precision, gradient accumulation and DDP/FSDP on limited GPUs.
Fair comparisons
Profiling, experiment tracking and controlled model comparisons.
Four capabilities, one per module.
The four outcomes Coursera lists for the course, in the order the modules build them.
Build modern backbones
Build and fine-tune ConvNeXt and Vision Transformer models using PyTorch Lightning and the timm library
Stabilize transformers
Apply RMSNorm, SwiGLU, and Rotary Position Embeddings (RoPE) in modern transformer architectures
Train efficiently
Implement mixed precision, gradient accumulation, and DDP/FSDP for efficient multi-GPU training
Benchmark experiments
Design, track, and benchmark CNN vs. ViT experiments using TensorBoard, W&B, and PyTorch Profiler
Learn deep learning with someone in your corner.
A dedicated coach alongside the course, with guidance shaped around the goal you bring.
Book a demo session- A coach matched to your goal in 24 hours
- A written roadmap you keep
- Weekly sessions for as long as you want them
A separate, optional Board Infinity service - not part of Coursera enrollment or the course certificate. 60-minute demo session · 100% refundable · no obligation to continue
A dedicated coach
One person who follows your progress alongside the course.
Goal-oriented guidance
Sessions shaped around the goal you set, not a fixed script.
Practice beyond the quizzes
Extra practice set for you, then reviewed with you.
Aim it at a role
Role-oriented direction on what to learn next.
Four modules, backbones to benchmarks.
Two weeks at 10 hours a week, fully self-paced. Open a module for what Coursera says it covers, what is inside it, and every lesson title.
- Module 01Backbones
ConvNeXt and Vision Transformers.
- Module 02Stabilize
RMSNorm, SwiGLU and RoPE.
- Module 03Train efficiently
Mixed precision and DDP/FSDP.
- Module 04Benchmark
Track, profile, compare.
- Explore the evolution of deep learning backbones from classical CNNs to ConvNeXt and Vision Transformers, understanding their mechanics, trade-offs, and industry relevance.
- Module 1 introduces modern backbone architectures, tracing the evolution from ResNets to ConvNeXt and Vision Transformers, covering patch embeddings, multi-head self-attention, and position encodings.
- Where Advanced Architectures Are Used Today11 minutes
- CNNs vs Transformers: Industry Reality8 minutes
- Skills You Need as a Vision Engineer7 minutes
- Why Classic CNNs Started Failing8 minutes
- What ConvNeXt Fixed in Old CNNs8 minutes
- ResNet vs ConvNeXt – Part 18 minutes
- ResNet vs ConvNeXt - Part 210 minutes
- How Images Become Tokens6 minutes
- What Attention Really Does6 minutes
- CNN vs ViT: Choosing the Right Backbone9 minutes
- Industry Landscape: Modern Backbones & Global Attention30 minutes
- Architectural Transition: From ResNet to ConvNeXt30 minutes
- Inside the ViT Forward Pass: Tokens, Attention & Positional Structure30 minutes
- Career Scope in Advanced Architectures30 minutes
- The Evolution Beyond ResNets30 minutes
- Vision Transformers Under the Hood30 minutes
- Modern Backbone Architectures (ConvNeXt & Vision Transformers)60 minutes
- Learn modern stabilization and efficiency techniques including RMSNorm, SwiGLU activations, and Rotary Position Embeddings that power state-of-the-art transformers.
- Module 2 dives into training dynamics and stabilization techniques including RMSNorm, SwiGLU activations, and Rotary Position Embeddings (RoPE) for stable, scalable training.
- Why Normalization Is Needed6 minutes
- BatchNorm vs LayerNorm vs RMSNorm8 minutes
- Practical Effects on Training Stability7 minutes
- Why ReLU Is Not Enough Anymore9 minutes
- GELU & SwiGLU Explained Visually8 minutes
- Practical Gains: Stability, Expressiveness, Convergence Speed11 minutes
- Why Position Encoding Matters8 minutes
- RoPE Explained Intuitively9 minutes
- Normalization Benchmarks in Modern Architectures30 minutes
- SwiGLU in Production Transformers30 minutes
- RoPE Explained: Sequence Extrapolation & Rotary Geometry30 minutes
- RMSNorm & Normalization Strategies30 minutes
- SwiGLU & Modern Activation Functions30 minutes
- Rotary Position Embeddings (RoPE)30 minutes
- Training Dynamics & Stabilization Techniques60 minutes
- Master practical techniques for training large models on limited hardware including mixed precision, gradient accumulation, and distributed training strategies.
- Module 3 focuses on efficient training on limited GPUs using mixed precision (FP16/BF16), gradient accumulation, efficient data pipelines, and distributed training with DDP/FSDP in Lightning.
- Why Mixed Precision Matters9 minutes
- FP16 vs BF16: When to Use What9 minutes
- Common Mixed Precision Failures8 minutes
- What Gradient Accumulation Really Does10 minutes
- Effective Batch Size Explained Clearly10 minutes
- Stability Issues with Large Batches9 minutes
- Single GPU vs Multi-GPU: When to Scale9 minutes
- DDP vs FSDP (Decision-Based)8 minutes
- Measuring Speed & Memory Correctly8 minutes
- AMP Benchmarks & Failure Patterns30 minutes
- Efficient Data Pipelines for Transformers & ViTs30 minutes
- Distributed Training on Commodity Hardware30 minutes
- Mixed Precision Training (FP16/BF16)30 minutes
- Gradient Accumulation & Large-Batch Simulation30 minutes
- Distributed Training with Lightning (DDP/FSDP)30 minutes
- Efficient Training on Limited GPUs60 minutes
- Learn to track experiments professionally and apply all course concepts in a hands-on ViT vs CNN Showdown project using fine-tuning with timm and PyTorch Lightning.
- Module 4 covers experiment tracking with TensorBoard and W&B, profiling FLOPs and throughput, and a hands-on ViT vs. CNN Showdown project with fine-tuning in timm.
- What to Track9 minutes
- Visualizing Loss Curves, Gradient Norms & Failure Modes9 minutes
- Profiling Memory & FLOPs10 minutes
- How Bad Comparisons Happen8 minutes
- Controlling Variables Properly8 minutes
- Forming Clear Hypotheses6 minutes
- Fine-Tuning ConvNeXt & ViT10 minutes
- Fine-Tuning ConvNeXt & ViT Part 26 minutes
- Fine-Tuning ConvNeXt & ViT Part 39 minutes
- Applying Mixed Precision & Efficiency Techniques Part -17 minutes
- Applying Mixed Precision & Efficiency Techniques7 minutes
- Interpreting Results Like an Engineer10 minutes
- Experiment Reproducibility & Performance Debugging30 minutes
- Backbone Design Patterns: Freezing, Unfreezing, Adapters, Head Tuning30 minutes
- Case Study: Fine-Grained Classification with Modern Backbones30 minutes
- Experiment Tracking (TensorBoard & W&B)30 minutes
- Designing a CNN vs ViT Experiment30 minutes
- The Hands-On Project - The ViT vs CNN Showdown30 minutes
- Experimentation, Tracking & The ViT vs CNN Showdown Project60 minutes
Titles, order and durations exactly as listed on the Coursera course page.
The tools you work in, and the skills you leave with.
The two official "Tools you'll learn" on the Coursera course page.
Vision Transformer (ViT)
Images as patch tokens with self-attention - built, fine-tuned and compared against CNNs.
PyTorch (Machine Learning Library)
The framework under everything, with PyTorch Lightning for training and the PyTorch Profiler for measurement.
Used throughout the course, but not part of Coursera's official tools list.
The nine skills Coursera lists for this course, grouped by the kind of work.
Architectures
Knowing what the backbone is doing.
- Convolutional Neural Networks
- Deep Learning
- Embeddings
Training
Getting a model to train well, and fit.
- Model Training
- Fine-tuning
- Model Optimization
Evaluation
Measuring what actually changed.
- Model Evaluation
- Performance Tuning
- Artificial Intelligence and Machine Learning (AI/ML)
Proof you can train modern backbones.
Complete the course on Coursera and earn a shareable career certificate from Board Infinity.
A course certificate issued through Coursera - not a degree, university credit or accreditation, and not a guarantee of employer recognition.
LinkedIn, resume or CV
Coursera says you can add it to your LinkedIn profile, resume, or CV.
Share it anywhere
Coursera also lists sharing it on social media and in your performance review.
Earned on completion
You earn it when you complete the course work on Coursera.
Two names on it
Board Infinity as the provider, Coursera as the platform.
Skills for training and evaluating vision models.
Where these skills may apply - career directions, not guaranteed outcomes.
Computer vision engineering
Choosing and fine-tuning CNN or ViT backbones for vision tasks.
Deep learning model development
Modern transformer components: RMSNorm, SwiGLU and RoPE.
Model training & optimization
Mixed precision, gradient accumulation and DDP/FSDP on limited GPUs.
Vision architecture evaluation
Weighing CNN and ViT trade-offs with controlled comparisons.
ML experimentation & profiling
Tracking runs in TensorBoard and W&B, profiling memory and FLOPs.
Training infrastructure
Measuring speed and memory correctly when scaling to multiple GPUs.
Papers and docs are free. So what is this?
Research papers, library docs and tutorials cover every topic here, and nobody is pretending otherwise. What a course adds is the order you learn it in, the checking that makes it stick, and a controlled comparison at the end.
This is the start of a three-course path.
Advanced Deep Learning Architectures Specialization - a 3-course series, intermediate level, 12 weeks at 5 hours a week.
Deep Learning: Advanced Backbones and Efficient GPU Training
21 hours · specialization listingConvNeXt and Vision Transformers, modern transformer components, GPU-efficient training and experiment benchmarking.
View course Course 2 · Efficient fine-tuningGenerative AI: Fine-Tuning LLMs and Diffusion Models
19 hours · specialization listingDecoder-only transformer pipelines with KV caching, LoRA and QLoRA fine-tuning, and diffusers pipelines with ControlNet.
View course Course 3 · Inference & deploymentDeploying Deep Learning: Quantization, Serving, and Edge AI
21 hours · specialization listingINT4/INT8 quantization (AWQ, GPTQ, GGUF), vLLM and NVIDIA Triton, ONNX Runtime and Llama.cpp, and a Dockerized inference API.
View courseCourse titles and hours as listed on the specialization page.
Know exactly what you're signing up for.
Common questions
Everything learners ask before enrolling, answered from the Coursera course page.
Yes. You should have working knowledge of PyTorch, CNNs, and standard training loops. Familiarity with transformers is helpful but not mandatory.
You'll work with PyTorch Lightning, the timm library, Weights & Biases, TensorBoard, and the PyTorch Profiler throughout the course.
Coursera lists it at 2 weeks to complete at 10 hours a week, on a flexible schedule you set yourself. The four modules are listed at 5, 5, 5 and 6 hours, and the specialization page lists the course at 21 hours.
The Coursera course page says the course prepares you for roles such as Deep Learning Engineer, Computer Vision Engineer, ML Research Engineer, and AI Infrastructure Engineer. Outcomes depend on the learner; no job or role is guaranteed.
According to Coursera, to access course materials, assignments, and earn a Certificate, you'll need to purchase the Certificate experience when you enroll in a course. Eligible learners may also have the option to start with a Free Trial. Some courses may also offer a Full Course, No Certificate option, which lets you access course materials, submit required assessments, and receive a final grade, but not earn or purchase a Certificate. Check the course page for the options available to you.
Coursera says that when you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate is added to your Accomplishments page, from where you can print it or add it to your LinkedIn profile.
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can't afford the enrollment fee. If financial aid or a scholarship is available for your learning program selection, you'll find a link to apply on the description page.
Yes - this page is Board Infinity's own overview of the course. Enrollment, lessons, grading and the certificate all happen on Coursera, where the course is hosted. Optional 1:1 coaching is a separate Board Infinity service.
Want someone building this with you?
Board Infinity 1:1 coaching pairs you with a practitioner while you take the course.
Modern backbones, trained efficiently.
ConvNeXt and Vision Transformers, stable transformer training, and GPU-efficient techniques - compared properly.