Information from one sense can be encoded into another channel

What it is

Information from one sense can be encoded into another channel. Visual images can become tactile dot patterns or sound patterns; the direction of a sound can become the location of a vibration. Substitution changes the carrier, not the information, so users can access content otherwise unavailable to them.

Why it happens

Feasibility rests on the brain's ability to abstract information: as long as the encoding preserves enough distinguishing features, the receiving channel can carry what the original channel sent. The mapping matters—a direct dimensional correspondence (brightness to vibration intensity) is easy to grasp but limited in capacity, while an abstract code (images as note sequences) carries more but requires learning. Effectiveness depends on whether the encoding is stable, extensible, and preserves the features the task needs, rather than reproducing the original experience.

Studying it

Have users perform tasks dependent on the original channel and measure performance: for example, blind users using tactile substitution for spatial localization or reading, recording accuracy, completion time, and the learning curve. Variables include the encoding scheme, training duration, and task type. Outcomes include task accuracy and trials to stability, the latter determining practicality at scale.

Where it stops holding

Substitution usually cannot restore the full capability of the original channel, especially fine spatial resolution or timbre discrimination. When a task depends on only a few categories, substitution works well; when it depends on high-resolution detail, its practical value is limited. Users need sustained training to reach stable performance, so brief trials do not reflect real ability.

Applying it

• Define the specific tasks the substitution must support and choose the encoding accordingly.
• Preserve the features the task requires rather than pursuing a full replica of the original experience.
• Provide a structured training program and measure the training needed to reach stable performance.
• Verification: have users perform the target task through the substitute and measure accuracy and learning curve, confirming the training load is practical.

Related

• Within the group: D5.05.2 The bandwidth of a substitute channel is usually far below the original · D5.05.3 Substitution demands significant learning
• Adjacent: D3.09.1 Haptics can substitute visual confirmation for blind users · J3.02 Sensory substitution technology in accessibility
• Search terms: sensory substitution · cross-sensory encoding · learning curve