You are here: Home Projects Modality-specific crosstalk in multitasking: Influence of modality compatibility in task switching (2015-2018; 2018-2021)
Document Actions

Modality-specific crosstalk in multitasking: Influence of modality compatibility in task switching (2015-2018; 2018-2021)


Iring Koch

Prof. Dr. Iring Koch                         

Principal Investigator

contact email

Denise Stephan

Dr. Denise Nadine Stephan           

Principal Investigator

contact email

 Erik Friedgen

Erik Friedgen 

PhD Candidate

contact email



Former Team Members

Edina Fintor

Edina Fintor                          

PhD Candidate 

contact email




Abstract (2018-2021)

Experimental studies revealed robust performance costs in human multitasking. In the present project in the context of the Priority Program (PP) 1772 (“Human Performance under Multiple Cognitive Task Requirements: From Basic Mechanisms to Optimized Task Scheduling”), we focus on serial multitasking using variations of the task-switching paradigm. In this paradigm, the basic finding is that performance is worse in task switches than in repetitions. These switch costs have been taken to index the involvement of cognitive control processes at the level of higher-order task representations (“task sets”). The present individual PP project aims at contributing to the goals of the PP by examining issues pertaining to the structure, flexibility, and plasticity of the task representations underlying serial task switching, with a specific emphasis on modality-specific effects (“modality compatibility,” MC). We define MC as occurring to the degree that the stimulus modality corresponds to the modality of the intended response effect (e.g., auditory-vocal task mappings would be modality compatible because vocal responses are typically intended to produce auditory effects). In the first funding period, we found that switch costs are larger when switching between two incompatible modality mappings (e.g., auditory-manual and visual-vocal) than between two compatible mappings. Our theoretical account proposes that such effects of MC in multitasking derive from “ideomotor” learning (i.e., learning response-effect contingencies and anticipation of response effects). Rooting the influence of MC in the functional constraints of human action control invites a broad empirical approach that raises research issues referring to the context-specific flexibility and learning-dependent plasticity of human action control and task set. Specifically, this research aims at understanding better (1) how tasks and mappings are represented (structural issue), (2) how tasks and mappings can be selected voluntarily in “free choices” (flexibility), and (3) how task sets can change with short-term and long-term practice (plasticity). In the second project phase, we aim at deepening our understanding of modality-specific interactions in multitasking. First, we generalize the notion of MC to different processing requirements and experimental paradigms, including conditions in which the modality mappings are divided across participants in a “task sharing” situation. Second, we explore the role of MC in free (“voluntary”) response modality choices. Third, we examine the boundary conditions for transfer from single-mapping and mixed-mapping practice both in serial task (i.e., mapping) switching and simultaneous dual tasks. Finally, we also aim to extend existing and develop new cooperative links within the PP.

Abstract (2015-2018)

Human behavior is usually governed by multiple task constraints. Yet, performance is most often worse in multitasking than in single-task conditions. Specifically, frequent task switching usually leads to switch costs relative to task repetitions. According to reconfiguration accounts, these costs are partly caused by a central bottleneck-like process of reconfiguring the higher-order representation of the cognitive and motor task requirements (i.e., task set).

The present project is aimed at examining the nature of this central reconfiguration process. A common conception of reconfiguration is that it refers to an abstract, amodal process that controls modality-specific sensory and motor processes. However, recent evidence revealed that switch costs can vary substantially depending on the specific modality mapping (i.e., compatibility) even if the degree of sensory and motor interference is equated across switching conditions. We propose that this influence of modality compatibility is not due to intrinsic features of stimulus processing itself but due to constraints of human action control. Specifically, based on the notion that action is preceded by the anticipation of its sensory consequences (ideomotor theory), stimuli that match the anticipated response consequences in terms of modality are modality compatible (e.g., vocal responses generate auditory effects, so that an auditory-vocal mapping represents a modality-compatible task). Accordingly, switching between modality-compatible tasks should lead to less switch costs than switching between incompatible tasks because the latter invoke more between-task crosstalk at the central mapping level.

In three sets of experiments, we intend to examine (1) structural aspects of task set more deeply by varying the modality mappings.

We also examine (2) the flexibility of task-set control by assessing the effects of advance task preparation and, additionally, of inhibition of competing tasks on the influence of modality compatibility. We measure inhibition as n-2 task repetition costs (backward inhibition) when switching between tasks defined by three different stimulus and response modalities.

Finally, we examine (3) the plasticity of the strength of modality mappings by manipulating short-term practice schedules and, additionally, the influence of age-related cognitive changes as well as of specific expertise with self-produced action effects (i.e., in musicians).

Together, this project is aimed at contributing to the goals of the research framework set by the Priority Program 1772, which seeks to integrate theoretical perspectives differing in terms of their focus on the structure, the flexibility, and the plasticity of task set and its control in human multitasking performance.


Project Output

Fintor, E., Stephan, N. D., & Koch, I. (2018). Short-term practice of modality-compatibility: Modality incompatible single-task practice reduces between-task crosstalk in task-switching. Revised manuscript is in progress.


Fintor, E., Poljac, E., Stephan, N. D., & Koch, I. (2018). Modality compatibility biases voluntary choice of response modality in task switching. Psychological Research, (Advanced online publication) doi.org/10.1007/s00426-018-1040-5

Fintor, E., Stephan, N. D., & Koch, I. (2018). The interplay of crossmodal attentional preparation and modality compatibility in cued task switching. Quarterly Journal of Experimental Psychology, (Advanced online publication) doi: 10.1177/1747021818771836

Fintor, E., Stephan, D. N., & Koch, I. (2018). Emerging features of modality mappings in task switching: Modality compatibility requires variability at the level of stimulus and response modality. Psychological Research, 82, 121-133.

Stephan, N. D., Hensen, S., Fintor, E., Krampe, R., & Koch, I. (2018). Influences of postural control on cognitive control in task switching. Frontiers in Psychology, doi: 10.3389/fpsyg.2018.01153


Personal tools