Topics

Long Context

Models and evaluations for reasoning over very large text, audio, video, or code contexts.

Gemini 1.5: The Long-Context Bet Becomes a Product-Scale Model

Gemini 1.5 made million-token multimodal context feel less like a demo trick and more like a practical interface for long documents, video, audio, and code.

Sequence Modeling · Carnegie Mellon University

Mamba: A Serious Attempt to Challenge Attention on Long Sequences

Mamba makes state space models selective, letting them decide what to remember or forget from the input while scaling linearly with sequence length.