Paul Janson
Paul Janson
Home
Publications
Contact
Light
Dark
Automatic
pipeline parallelism
Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
MAPL treats inter-stage activation compression in pipeline parallelism as a learnable orthogonal projection under Stiefel manifold constraints, letting each stage adapt its own task-optimal subspace. Combined with factorized anchor embeddings and residual vector quantization, it achieves high compression with negligible performance loss across LLaMA models from 150M to 1B parameters.
Paul Janson
,
Edouard Oyallon
,
Eugene Belilovsky
PDF
Cite
Cite
×