00:00:30 - Why Inference Matters
00:03:19 - Inference Challenges and Key Metrics
00:08:31 - Innovations for Faster Inference
00:15:17 - Focusing on Inference and GPU Performance
00:16:12 - How Requests Are Routed and Processed
00:22:02 - Optimizing Inference with Kernels, Parallelism, and Quantization
00:30:19 - Quantization and Model Quality
00:32:35 - MiniMax Quantization Success Story
00:34:08 - Recap and Audience Q&A
00:45:32 - Challenges of Routing Across Inference Providers
00:50:30 - Monitoring and Evaluating Model Quality
00:56:00 - Improving Open Model Serving Consistency
01:00:36 - Quantizer Evaluations and Model Quality
01:01:45 - Local Models and the Future of Ollama
01:04:30 - Equality Saturation for AI Compilers
01:15:43 - Multi-Level IR and Kernel Fusion
01:17:38 - Hardware-Based Search and Cost Modeling
01:20:37 - Q&A on Compiler Design and Search
01:31:22 - Why Radix Attention Was Created
01:35:04 - Scheduling and Overlapping GPU Work
01:38:26 - KV Caching and Unified Radix Trees
01:46:22 - Unified Memory Pools and Compaction
01:49:24 - Miles Inference in the Training Loop
01:50:19 - XLA Summit and Q&A