Information Extraction and Classification on Journal Papers

Authors

Lei Yu, University of Nebraska-LincolnFollow

First Advisor

Prof. Stephen D. Scott

Second Advisor

Prof. Ashok Samal

Third Advisor

Prof. Vinodchandran Variyam

Date of this Version

11-2021

Document Type

Article

Citation

Yu, Lei, "Information Extraction and Classification on Journal Papers" (2021).

Comments

A THESIS Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfilment of Requirements For the Degree of Master of Science, Major: Computer Science, Under the Supervision of Professor Stephen D. Scott. Lincoln, Nebraska: November, 2021

Copyright © 2021 Lei Yu

Abstract

The importance of journals for diffusing the results of scientific research has increased considerably. In the digital era, Portable Document Format (PDF) became the established format of electronic journal articles. This structured form, combined with a regular and wide dissemination, spread scientific advancements easily and quickly. However, the rapidly increasing numbers of published scientific articles requires more time and effort on systematic literature reviews, searches and screens. The comprehension and extraction of useful information from the digital documents is also a challenging task, due to the complex structure of PDF.

To help a soil science team from the United States Department of Agriculture (USDA) build a queryable journal paper system, we used web crawler to download articles on soil science from the digital library. We applied named entity recognition and table analysis to extract useful information including authors, journal name and type, publish date, abstract, DOI, experiment location in papers and highlight the paper characteristics in a computer queryable format in the system. Text classification is applied on to identify the parts of interest to the users and save their search time. We used traditional machine learning techniques including logistic regression, support vector machine, decision tree, naive bayes, k-nearest neighbors, random forest, ensemble modeling, and neural networks in text classification and compare the advantages of these approaches in the end.

Advisor: Stephen D. Scott