Pattern Recognition 40 (2007) 3728–3739
www.elsevier.com/locate/pr
EROS: Ensemble rough subspaces
Qinghua Hu∗, DarenYu, Zongxia Xie, Xiaodong Li
Harbin Institute of Technology, Harbin, China
Received 24 May 2006; received in revised form 28 March 2007; accepted 29 April 2007
Abstract
Ensemble learning is attracting much attention from pattern recognition and machine learning domains for good generalization. Both
theoretical and experimental researches show that combining a set of accurate and diverse classifiers will lead to a powerful classification
system. An algorithm, called FS-PP-EROS, for selective ensemble of rough subspaces is proposed in this paper. Rough set-based attribute
reduction is introduced to generate a set of reducts, and then each reduct is used to train a base classifier. We introduce an accuracy-guided
forward search and post-pruning strategy to select part of the base classifiers for constructing an efficient and effective ensemble system. The
experiments show that classification accuracies of ensemble systems with accuracy-guided forward search strategy will increase at first, arrive
at a maximal value, then decrease in sequentially adding the base classifiers. We delete the base classifiers added after the maximal accuracy.
The experimental results show that the proposed ensemble systems outperform bagging and random subspace methods in terms of accuracy
and size of ensemble systems. FS-PP-EROS can keep or improve the classification accuracy with very few base classifiers, which leads to a
powerful and compact classification system.
2007 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Attribute reduction; Ensemble learning; Multiple classifier system; Rough set; Selective ensemble
1. Introduction
Ensemble learning refers to training a set of base predictors
for a given classification or regression task and then combining
their outputs with a fusion strategy. This is also called multi-
ple classifier systems [1], expert committee [2], decision fores