Learning Hierarchically Decomposable Concepts with Active Over-Labeling

Authors

Yuji Mo, University of Nebraska-LincolnFollow
Stephen Scott, University of Nebraska-LincolnFollow
Doug Downey, Northwestern UniversityFollow

Date of this Version

2016

Citation

Published in 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain 12-15 Dec. 2016.
DOI: 10.1109/ICDM.2016.0045

Abstract

Many classification tasks target high-level concepts that can be decomposed into a hierarchy of finer-grained subconcepts. For example, some string entities that are Locations are also Attractions, some Attractions are Museums, etc. Such hierarchies are common in named entity recognition (NER), document classification, and biological sequence analysis. We present a new approach for learning hierarchically decomposable concepts. The approach learns a high-level classifier (e.g., location vs. non-location) by seperately learning multiple finer-grained classifiers (e.g., museum vs. non-museum), and then combining the results. Soliciting labels at a finer level of granularity than that of the target concept is a new approach to active learning, which we term active over-labeling. In experiments in NER and document classification tasks, we show that active overlabeling substantially improves area under the precision-recall curve when compared with standard passive or active learning. Finally, because finer-grained labels may be more expensive to obtain, we also present a cost-sensitive active learner that uses a multi-armed bandit approach to dynamically choose the label granularity to target, and show that the bandit-based learner is robust to differences in label cost and labeling budget.