1

A PyTorch library that makes learned optimizers accessible to the broader ML community with CUDA-accelerated implementations with substantial speedups (5x improvement for ViT training) and integrates seamlessly with existing PyTorch workflows, enabling practical application of learned optimization to real-world large-scale tasks.

We demonstrate that infinite learning rate schedules consistently outperform widely-used repeated cosine decay for continual pre-training under distribution shifts across both vision and language models, providing a more effective alternative for large-scale self-supervised learning without catastrophic forgetting.

This study investigates the capabilities of video diffusion models in generating human motion from text prompts, revealing their strengths in common motions and limitations in rare or complex movements.

Learning novel concepts, remembering previous knowledge, and adapting it to future tasks occur simultaneously throughout a human’s …

Adding knowledge to the model without destroying its generalization by finetuning small set of parameters

Continual zero-shot learning involves learning seen classes incrementally while improving the ability to recognize unseen or …

A baseline that performs better without training in continual learning benchmarks