Self-adaptive heterogeneous random forest

Random Forest RF is an ensemble learning approach that utilises a number of classifiers to contribute though voting to predicting the class label of any unlabelled instances. Parameters such as the size of the forest N and the number of features used at each split $M$, has significant impact on the performance of the RF especially on instances with very large number of attributes. In a previous work Genetic Algorithms has been used to dynamically optimize the size of RF. This study extends this genetic algorithm approach to further enhance the accuracy of Random Forests by building the forest out of heterogeneous decision trees, heterogeneous here means trees with different $M$ values. The approach is termed as Heterogeneous Genetic Algorithm based Random Forests HGARF. As Random Forests generates a typical large number of decision trees with randomisation over the feature space when splitting at each node for all the trees, this has motivated the development of a genetic algorithm based optimisation. Typically, HGARF accepts as an input a forest RF of N trees, the initial population is randomly generated from RF as a number of smaller random forests rf_i where each one has a number n_i of trees. This population of forests is then evolved through a number of generations using genetic algorithms. Our extensive experimental study has proved that Random Forests performance could be boosted using the genetic algorithm approach.