Captions carry the words but rarely restore tone or urgency

What it is

Captions carry linguistic content but rarely restore tone or urgency. Once a line is transcribed, "I'm fine" is textually complete, yet the fatigue, reluctance, or warning in it is gone. What deaf and hard-of-hearing readers lose is not the words but how they were said.

Why it happens

Tone lives in prosody: pitch contour, stress placement, rate, and pauses jointly express attitude and urgency, and writing has no equivalent notation. Without explicit annotation, those dimensions drop to zero during transcription. The consequence is sharpest where urgency must be judged: a line that sounds like a shout in audio can read as a flat statement in text, so readers set their threshold for immediate action too conservatively.

Studying it

Compare audio-only, caption-only, and audiovisual presentation for judgments of speaker emotion, urgency, and need to act, measuring accuracy and consistency. Variables include whether prosodic annotation is present (parenthetical descriptions, emphasized punctuation, pace hints) and the emotional intensity of the content. Include the direction of errors, since mistaking urgent for calm and calm for urgent do not cost the same.

Where it stops holding

When content is purely informational and the speaker's attitude is irrelevant—a gate change announcement—lost prosody matters little. If emotion is carried by the wording itself ("leave now"), the text suffices and prosody only reinforces it. Captions also cannot substitute for what music and sound effects convey; that lies in the territory requiring additional description.

Applying it

• Add brief prosodic annotations where tone affects comprehension, such as "(hesitant)" or "(urgently)."
• Reinforce urgency through punctuation and line breaks instead of flattening a shout into a statement.
• For content requiring immediate action, state the action explicitly so readers need not infer it from tone.
• Verification: have caption-only readers judge speaker emotion and need to act, compare with audio-only listeners, and treat a systematic conservative bias as evidence that tone is missing.

Related

• Within the group: D2.12.2 Non-speech sound needs description, not only dialogue transcription · D2.12.4 Equivalence is not full replacement; some sound information is lost in transcription
• Adjacent: J2.03 Coverage of captions and text alternatives · D2.06.1 Speech conveys meaning well but structure poorly
• Search terms: captions · prosody · accessibility