Brenda Z. Yang

Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Serving

Generative conversational interfaces powered by large language models (LLMs) typically stream output token-by-token at a rate determined by computational budget, often neglecting actual human reading speeds and the cognitive load associated with the content. This mismatch frequently leads to inefficient use of computa…