DIVERSITY-BASED SELECTION OF LEARNING ALGORITHMS: A BAGGING APROACH

Abstract

Nowadays, classification problems are becoming increasingly important in many real-world applications. As the problems
become more complex and the consequences of a bad decision are more serious, more advanced techniques, as the combination
of classifiers, need to be applied. When combining classifiers, it is important to ensure diversity between them as it does not
make sense to combine classifiers whose classification is the same. There are several techniques to ensure diversity in systems
like these and generally it consider modify the data set, use different learning algorithms or make a process of improvement or
learning on the individual classification. Although the relationship between diversity and system accuracy has not been fully
established, it is clear that diversity remains a factor to be taken into account in the construction of multiclassifiers. In this paper
we present a modification to the bagging algorithm to consider different learning algorithms during the training process and
optimize the classifiers built to obtain diverse systems and as accurate as possible. Executed simulations suggest the use of the
Double Failure pairwise measure to quantify the diversity of the system. With respect to the number of classifiers used, it was
observed that the systems built had approximately half of the total classifiers they should have. After, the superiority of the
proposed method with respect to five state-of-the-art multiclassifiers was verified and it is suggested the incorporation of a
learning process like the one executed in Stacking. Finally, are shown results in biochemical real applications and the general
conclusions are exposed.