Spoken language properties can influence downstream motor planning during writing, but how paralinguistic voice characteristics shape handwriting initiation and accuracy remains underexplored. In our Chinese dictation-to-writing experiment, native Cantonese-speaking participants (N=124) heard 835 two-character Chinese words produced by one of the five AI-generated Cantonese voices and wrote the words on a digitizing pad. The voices varied in speaker’s gender and pitch, with the assignment of a voice to each of the 835 words being counterbalanced across participants. Trials were randomized across voices; item and participant effects were controlled using linear mixed‑effects modeling. Dependent measures were first‑stroke reaction time from word offset (FSRT) and writing accuracy. Predictor variables include voice’s gender and pitch, controlling for participant’s gender and word’s duration. Results showed that words with higher pitch predicted lower writing accuracy. Male voice predicted lower writing accuracy yet faster FSRT than female voice, indicating that participants’ response criterion was more lenient to words spoken by male speaker than by female speaker. The Gender x Pitch interaction showed that male voice triggered slower FSRT for words with higher pitch than those with lower pitch, but this did not occur for female voice. These suggest that voice gender and acoustic features might influence the perceptual processing, impact motor output, and in turn, writing performance in the dictation-to-writing task. AI-generated voice characteristics moderate writing reaction time and accuracy, revealing the need for further research to refine synthetic audio design.

Authors:
Xi Huang, The Chinese University of Hong Kong, Hong Kong
Xi Cheng, The Chinese University of Hong Kong, Hong Kong
Chi-Shing Tse, The Chinese University of Hong Kong, Hong Kong

About the Presenter(s)
Professor Chi-Shing Tse is a University Professor/Principle Lecturer at The Chinese University of Hong Kong in Hong Kong

Connect on Linkedin
https://www.linkedin.com/in/chi-shing-tse-44b7451b6/

See this presentation on the full schedule – Saturday Schedule

Bookmark