Quantization, parallelism, and trade-offs
Quantization reduces bytes per parameter; tensor and pipeline parallelism distribute compute and memory. Each choice moves memory, quality, communication, and latency.
Links containing ?t= open the video at a specific second.
Video summary
The ideas to retain
Quantization does not mean that the whole model has one precision
Common notation already carries important scope information.
The basic operation introduces representation error
For a simple symmetric quantizer, we can write:
Weight-only and weight-plus-activation quantization target different bottlenecks
Key moments
Jump directly to a section
Next step
