Automatic Language Identification in Texts

This book provides readers with a brief account of the history of Language Identification (LI) research and a survey of the features and methods most used in LI literature. LI is the problem of determining the language in which a document is written and is a crucial part of many text processing pipelines. The authors use a unified notation to clarify the relationships between common LI methods. The book introduces LI performance evaluation methods and takes a detailed look at LI-related shared tasks. The authors identify open issues and discuss the applications of LI and related tasks and proposes future directions for research in LI. - In addition, this book reviews the history of LI research, including the challenges that have renewed interest in researching the topic - Compares and contrasts the features and methods commonly used for LI, as well as LI performance evaluation methods - Highlights the applications of language identificationand identifies areas for future research in LI About the Authors: Tommi Jauhiainen, Ph.D., is a Post-doctoral Researcher at The University of Helsinki. Marcos Zampieri, Ph.D., is an Assistant Professor at George Mason University. Timothy Baldwin, Ph.D., is the Acting Provost and Chair of the Department of Natural Language Processing at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) in addition to being a Melbourne Laureate Professor in the School of Computing and Information Systems at The University of Melbourne. Krister Lindén, Ph.D., is the Research Director of Language Technology at the University of Helsinki in addition to the National Coordinator of FIN-CLARIN, the Finnish Node of CLARIN ERIC, which is a European research infrastructure for Social Sciences and the Humanities.