Deep Neural Networks For Named Entity Recognition On Social Media

Abstract

Named entity recognition (NER) on noisy data, specifically user-generated content (e.g. on- line reviews, tweets) is a challenging task because of the presence of ill-formed text. In this regard, while studies on morphologically-poor languages such as English has been rapidly advancing in recent years, studies on morphologically-rich languages such as Turkish has fallen behind for noisy data. This is mostly due to Turkish being an agglutinative language, having a rich morphology and also having scarce annotated data. Existing studies on Turkish both for noisy and formal (e.g. news text) data still make use of hand-crafted features and/or external domain-specific resources (e.g. gazetteers). In this thesis, we investigate the effects of neural architectures without the help of any external domain-specific resources and/or manually-constructed features. So that the proposed model can also be used for different morphologically-rich languages and for different domains. Moreover, we also experimented with different word and sub-word level (e.g. morpheme, character or character n-gram level) embedding techniques and we argue that sub-word level embeddings provide better word representations for morphologically-rich languages syntactically and semantically. For this purpose, we propose a transfer learning model that is an extension of a baseline, bidirectional LSTM-CRF architecture. The model is trained on two different datasets simultaneously for ithe purpose of transfer learning from formal to noisy data and it exploits morpheme-level, character n-gram level and orthographic character-level embeddings as its feature set. Con- sequently, we have obtained an F1 score of 65.72% on Turkish tweet dataset and 41.97% on English WNUT’17 dataset.