Researching and assessing speech and language in a digital world
Analyzing natural speech and language samples of children is a well-known source of insights when conducting research in the field of speech and language acquisition. The process of collecting, manual transcription and analysis of these data however is extremely time-consuming and costly. Because of that, the data basis for many milestones in speech and language development is scarce due to small sample sizes.
Meanwhile speech recognition and processing technology has been developed to a point where use for research purposes in linguistics and speech-language-pathology seems possible. For the recognition of adult language, technology has evolved to mainstream applications like Siri, Alexa or Dragon Speech. However processing child utterances is much more challenging due to their acoustic and language properties.
The TALC project develops a hard- and software tool which enables recording as well as (semi)automatized transcription and linguistic analysis of natural speech samples. Researching speech and language development is transformed digitally on the interface of linguistics, computer linguistics, speech and language pathology / pedagogy and computer science:
- Big data access is possible by making the process of recording, transcription and analysis more applicable
- Knowledge drawn from data is based on longer sequences of natural communication
- Transferring results into intervention is facilitated by analyzing individual environments for speech and language acquisition
- TALC data can provide an alternative in evaluating change in everyday communication (as demanded by the ICF)
What is TALC-LSA?
TALC-LSA is a small, wearable tool, which allows recordings of several hours of natural communication. The matching software in a first step identifies individual speakers (e.g. in a child-parent or child-teacher interaction) and (semi)automatically transcribes the sample. The transcript is then analyzed using specific descriptive and linguistic parameters (e.g. word count, conversational turns, lexical diversity, word classes). In addition, acoustic information from the environment (e.g. electronic media) will be analyzed. The TALC-LSA project aims to be able to provide all these features also in multilingual and institutional contexts (e.g. kindergarten or school).
TALC Project Status
TALC-LSA has two parallel project branches in Germany and South Africa. Complementary teams of the disciplines Speech and Language Therapy, (Computer) Linguistics, Information Science and Electric/Electronic Engineering are working at both locations.
In both countries the talc pilot studies have started in 2019. Initially TALC-LSA is being developed in German and Afrikaans. In South Africa the TALC-LSA team has already started collecting data in Sesotho; data collection in isiXhosa is planned starting in 2024. In the future TALC-LSA will be extended to other relevant languages of multilingual children growing up in Germany as well.
Prof. Jeannie van der Linde from the University of Pretoria is the TALC-LSA PI in South Africa.
Publications
2023
- Gebauer, C., Rumberg, L., Ehlert, H., Lüdtke, U. & Ostermann, J. (2023). Exploiting Diversity of Automatic Transcripts from Distinct Speech Recognition Techniques for Children’s Speech. Proceedings INTERSPEECH -- 24th Annual Conference of the International Speech Communication Association, August 2023, 4578-4582.
- Rumberg, L., Gebauer, C., Ehlert, H., Wallbaum, M., Lüdtke, U. & Ostermann, J. (accepted). Uncertainty Estimation for Connectionist Temporal Classification Based Automatic Speech Recognition. Proceedings INTERSPEECH -- 24th Annual Conference of the International Speech Communication Association, August 2023, 4583-4587.
- Lüdtke, U., Ehlert, H., Gaigulo, D., & Bornam, J. (2023). Research on the Methodology of LSA with Preschool Children: A Scoping Review. Clinical Archives of Communication Disorders, 8(2), 29-46.
- Ehlert, H., Beaulac, E., Wallbaum, M., Gebauer, C., Rumberg, L., Ostermann, J., & Lüdtke, U. (2023). Collecting and Annotating Natural Child Speech Data - Challenges and Interdisciplinary Perspectives. Elektronische Sprachsignalverarbeitung, Tagungsband der 34. Konferenz, München, März 2023, 72-78.
- Gebauer, C., Rumberg, L., & Ostermann, J. (2023). Pronunciation Modelling for Children's Speech. Elektronische Sprachsignalverarbeitung, Tagungsband der 34. Konferenz, München, März 2023, 79-86.
- Lüdtke, U., Bornman, J., de Wet, F., Heid, U., Ostermann, J., Rumberg, L., van der Linde, J., & Ehlert, H. (2023). Multi-disciplinary perspectives on automatic analysis of children's language samples: Where do we go from here? Folia Phoniatrica et Logopaedica, 75(1), 1-12. doi: 10.1159/000527427
2022
- Rumberg, L., Gebauer, C., Ehlert, H., Lüdtke, U. & Ostermann, J. (2022). Improving Phonetic Transcriptions of Children’s Speech by Pronunciation Modelling with Constrained CTC-Decoding. Proceedings INTERSPEECH -- 23th Annual Conference of the International Speech Communication Association, September 2022, 1357-1361.
2021
- Rumberg, L., Ehlert, H., Lüdtke, U. & Ostermann, J. (2021). Age-Invariant Training for End-to-End Child Speech Recognition using Adversarial Multi-Task Learning. Proceedings INTERSPEECH -- 22th Annual Conference of the International Speech Communication Association, August 2021, 3850-3854.