Serpil Karabüklü

Serpil Karabüklü

she/her

Language Across Channels

Sign Language Linguistics
Human Language & Cognition
Computational Modeling

Language Across Channels

How does language organize meaning when multiple channels can express information simultaneously?

Human language is not limited to a single channel of expression. Sign languages make this particularly visible: the hands, face, body, and space can contribute distinct linguistic information, often at the same time. Yet these channels do not necessarily convey the same information, nor must they always align.

My research investigates the linguistic and cognitive principles governing how information is distributed, coordinated, and interpreted across channels. Drawing on American Sign Language (ASL), Turkish Sign Language (TİD), and Nicaraguan Sign Language (NSL), I combine theoretical linguistics, experimental methods, quantitative movement analysis, and computational modeling.

Through three interconnected research strands, I examine what the organization of sign languages reveals about linguistic structure, representation, processing, and human cognition.

Linguistic structure Linguistic structure, simultaneity, and meaning

How do multiple articulators contribute to linguistic meaning, and what determines how information is distributed across them?

Sign languages can encode linguistic distinctions simultaneously through manual and nonmanual articulators. However, the contributions of these articulators are not necessarily redundant or uniformly coordinated. Understanding how they interact offers insights into the relationship between linguistic structure, meaning, and modality.

My research on Turkish Sign Language (TİD) examines the expression of certainty, focus, tense, aspect, and modality. My work on certainty investigates how linguistic meaning is distributed across simultaneous articulations, while research on focus shows the importance of manual duration and the variable contribution of nonmanual cues to prosodic prominence.

Together, these studies raise broader questions about how languages allocate linguistic functions across available articulators, whether simultaneous structures are represented independently or jointly, and how modality shapes the realization of grammatical distinctions.

Selected publications

Linguistic structure Language Experience, Visual Attention, and Cognition

How does language experience shape the production, perception, and interpretation of linguistic information across channels?

Understanding simultaneous linguistic structure requires investigating not only how information is encoded, but also how language experience shapes its production and interpretation.

My research on Turkish Sign Language (TİD) reveals age-of-acquisition differences in linguistic production and judgment. Later-acquiring signers produced longer sign durations and more nonmanual articulations, showed less differentiation across semantic categories, and were more likely to accept linguistically unacceptable forms. These findings suggest that language experience influences not only individual linguistic cues but also how distinctions are organized and interpreted.

Complementing this work, my research examines how sign language learning relates to the temporal resolution of visual attention, connecting language experience to broader questions about perceptual processing. I also investigate how the kinematic properties of head movements distinguish linguistic functions in ASL and TİD, using pose estimation and dynamic time warping to move beyond categorical descriptions of nonmanual behavior.

Building on these findings, my ongoing research asks how age of acquisition affects the integration and interpretation of simultaneous linguistic structures. In particular, how do signers with different language acquisition histories use information distributed across manual and nonmanual articulators?

Together, these studies contribute to understanding the relationship between language experience, linguistic representation, perception, and processing.

Selected publications and preprints

Linguistic structure Computational Modeling and Linguistic Evaluation

Can computational models capture the linguistic distinctions expressed across multiple channels in sign languages?

Recent advances in computational sign language processing provide new opportunities to investigate how linguistic information is represented and interpreted. However, strong performance on conventional translation benchmarks does not necessarily demonstrate sensitivity to the grammatical distinctions that human signers recognize.

My research develops linguistically grounded approaches to evaluating sign language models, with particular attention to their sensitivity to manual and nonmanual information. Through the ASL Minimal Translation Pairs (ASL-MTP) benchmark, I investigate whether computational models distinguish linguistically meaningful contrasts and how their performance changes when information from different articulators is available or restricted.

Our analyses of SHuBERT-based sign language translation models reveal uneven sensitivity to linguistic distinctions and substantial reliance on manual information, raising questions about how current architectures represent simultaneous structure.

I also use computational tools, including pose estimation and movement-trajectory analysis, to identify linguistic properties that may be difficult to capture through categorical annotation alone.

Together, these approaches allow me to investigate both what computational models learn about sign languages and how computational methods can contribute to linguistic theory and the study of human language processing.

Selected publications and resources

Toward a Broader Understanding of Human Language

These research strands converge on a central question: What can the organization of linguistic information across multiple channels reveal about the architecture of human language?

By investigating how sign languages distribute meaning across articulators, how language experience shapes the interpretation of simultaneous structures, and how computational models represent these distinctions, my research seeks to identify principles of linguistic organization that extend beyond individual languages and modalities.

This perspective treats simultaneity not merely as a property of visual communication, but as a window into the relationship between linguistic structure, cognition, and the expressive capacities of human language.

Explore my full publication list or recent research activities.