Named Entity Extraction

A parallel corpus is a valuable resource in natural language processing and computational linguistics. It is used in machine translation, lexicography, statistical language analysis, cross language information retrieval, among others. This study aimed to build a parallel corpus of Philippine tourism data and evaluated the corpus using a statistical machine translation system. The corpus is to be used by the Philippine component of the ASEAN-MT  project.  ASEAN-MT is a network-based ASEAN language public translation service.  It aims to automatically translate among ASEAN languages and English, with English as the pivotal language. 

The source texts were in English and were gathered from four websites about Philippine tourism. The Filipino translation was done manually. Translations were by sentence. Guidelines to translating were followed like, multiple translations must be avoided and the commonest word order must be followed. Aside from the translations, the corpus is also annotated with the named entities, such as people’s names, group names, company names, currency units, temporal entities, language names, locations, products, and artistic creations.

The parallel corpora has 21, 491 sentences with 370,910 English words and 416,290 Filipino words. Each sentence was from varying lengths, from one word to approximately 30 words per sentence.  The corpora are categorized into Festivals and Events, Provincial Profile, Tourist Attractions and General Information. These categories were based on the source websites and the information of the text.

A lexicon was created by manual extraction of named-entities from the different Philippine tourism websites. Most of the named-entities for the lexicon were retrieved from lists provided by the different Philippine tourism websites also. The named-entities were manually translated and classified.

The corpora was assessed based on the following factors – (1) its overall BLEU when used on a statistical machine translation system, (2) BLEU score of the sub-corpora per translator, (3) BLEU per category, and (4) BLEU score of the sub-corpora per translator per category.

We found out that the number of function words, named-entities and numbers, ambiguity and alignment are the main factors which affect the quality of the machine translation. Function words add to the number of words in the translated sentence which causes misalignment while named-entities and numbers causes “penalty” to the BLEU score because they were not translated.

Keywords: Statistical machine translation systems, corpus building, tourism, BLEU score and named-entity


References
Evans, D. (2007). Corpus building and investigation for the humanities. University of Nottingham. http://www. corpus. bham. ac. uk/corpus-building. shtml. Accessed 10 Feb. 2015
Mukherjee, J., Kunstler, V., Maiwald, P., Saage, S. 2007. A guide to corpus linguistics. http://fss.plone.unigiessen.de/fss/faculties/f05/engl/ling/help/materials/restricted/clguide.pdf/file/A%20Guide%20to%20Corpus%20Linguistics.pdf. Accessed 10 Feb. 2015
Pumikalek, J. 2007. Building parallel corpora from the web.
Sridhar,V., Barbosa, L., and Bangalore,S. 2006. A scalable approach to building a parallel corpus from the web. http://www.research.att.com/export/sites/att_labs/techdocs/TD_100424.pdf. Accessed 10 Feb. 2015
Tan, L., and Bond, F. 2011. Building and annotating the linguistically diverse.  NTU-MC (NTU- Multilingual Corpus).

The Centre for Excellence in the Humanities to the University of Sofia, Bulgaria, has the pleasure to invite, for a fourth time, experts in the fields of Digital Humanities and Information Technologies to an Advanced Summer School in Digital Humanities.
The event will take place in September 2017 in a nice mountain retreat near Sofia, Bulgaria (tbc). The summer school relies on the cooperation of world-leading instructors from academic institutions in the UK and USA

Named Entity Recognition serves as the basis for many other areas in Information Management such as Semantic Annotation, Question Answering, Ontology Population or Opinion Mining. However, it is unclear what the meaning of Named Entity is, and yet there is a general belief that Named Entity Recognition is a solved task with success ratios over 95%. This question has been pointed out in other works, but it has never been fully analyzed. In this paper we do so by examining the evolution of the field and the several definitions that have been proposed both from a theoretical and practical point of view. We argue that the task is actually far from solved and show the consequences for the development and evaluation of tools. We discuss topics for further research with the goal of bringing the task back to the research scenario and develop appropriate evaluation techniques to reassess how well the tools perform for the current wealth of user needs.

DID-ACTE: Digging into Data: Automating Chinese Text Extraction

The Automating Data Extraction from Chinese Texts Project aims to provide humanists and social scientists with a means of transforming 2200 years of Chinese texts into structured data. The project will develop an open-source platform (MARKUS) that allows users to apply sophisticated text-mining techniques to a wide variety of historical and literary texts. Users will be able to tag and extract personal names, dates, place names, official titles and postings, kinship ties, other social relationships, and other user-defined content. The platform will be tested against 2000 local histories spanning an 800-year period and roughly 20,000 letters and 500 notebooks dating from the seventh through the thirteenth century. Data extracted from the sample repositories will be used to enrich text-mining applications and will also be made available for research through open-access online databases and data archives.

In this paper, we describe our approach for Named Entity rEcognition and Linking Challenge (NEEL) at the #Micro-posts2016. The task is to automatically recognize entities and their types from English microposts, and link them to corresponding DBpedia 2015 entries. If the resources do not exist, we use NIL identifiers instead. The task is unique as twitter data is informal in nature with non-conformational spellings, random contractions and various other noises. For this task, we developed our system using a hybrid model. We have used various existing named entity recognition (NER) systems and combined them with our classifier to improve the results.