Blog

Overview This blog walks through two crucial DeepSpeed updates: (1) a PyTorch-identical backward API that…

Introduction As introduced in a previous blog post, Helion is a high-level DSL that empowers…

We’re excited to share that PyTorch now leverages Pyrefly to power type checking across our…

Summary In this post, we discuss how we optimized the Mamba-2 State-Space Dual (SSD) module…

What is an accumulator in an accelerator's GEMM engine and why does it matter? GPUs…

Why Choose PyTorch for Recommendation System PyTorch has emerged as the de facto framework in…

Recently, the PyTorch team released Helion, a new domain-specific and PyTorch-based language to make the…

Interactive image segmentation has become a defining mobile experience across the world’s most popular apps.…

We are excited to announce the release of PyTorch® 2.10 (release notes)! This release features…

Scaling reinforcement learning (RL) for post-training large language models (LLMs) is notoriously difficult. While running…

The Triton compiler aims to generate performance-portable code and runtime across hardware for AI kernels.…

Overview The most recent LLM serving frameworks and models increasingly adopt attention variants, such as…

If you want powerful on-device AI that doesn’t blow your memory budget or turn your…

What is cluster launch control (CLC)? Blackwell brings in cluster launch control (CLC) to enable…

Training massive Mixture-of-Experts (MoE) models like DeepSeek-V3 and Llama 4-Scout efficiently is one of the…

Introduction The PyTorch community is actively working to build a growing ecosystem of specialized accelerators…