Towards low-budget real-time active learning for text classification via proxy-based data selection

Abstract

Training data is typically the bottleneck of supervised machine learning applications, heavily relying on cost-intensive human annotations. Active Learning proposes an interactive framework to efficiently spend human efforts in the training data generation process. However, re-training state-of-the-art text classifiers is highly computationally intensive, leading to long training cycles that cause annoying interruptions to humans in the loop. To enhance the applicability of Active Learning, we investigate low-budget real-time Active Learning via Proxy-based data selection in the domain of text classification. We aim to enable fast interactive cycles within a minimal labelling effort while exploiting the performance of state-of-the-art text classifiers. Our results show that Proxy-based Active Learning can increase the F1-score of a lightweight classifier compared to a traditional budget Active Learning approach up to ~19%. Our novel Proxy-based Active Learning approach can be carried out time-efficiently, requiring less than 1 second for each learning iteration.