Comparison of hybrid approaches in classification using decision trees and neural networks

Abstract

Approaches combining methods based on decision trees and neural networks are an important examples of hybrid strategies. The problem of classification of the table-based data is considered. There is a number of studies sharing the idea of unifying neural network and decision tree models. Besides the most common idea of straightforward using the ensemble of these two algorithms, there are Deep Neural Decision Trees (DNDF) – a notion for a neural decision trees with the split functions realised as a randomized multilayer perceptrons. In the applications where the trees approach is feasible, forest of such trees also can be applied as a generalization. There are many approaches in constructing a forest of trees and different methods using the forest of these decision trees, like Neural Decision Forests (NDF), Neural Random Forests (NRF), Neural Deep Forests. The research deals with the elaboration and implementation of these methods. Further on, all presented methods are to be compared with each other, as well as with the original algorithms themselves– the decision tree, ensemble of trees and the multilayer neural network. It is important to note that the comparison is not reduced to answering the question, which of them have better result in different examples, because such properties are already studied and presented using different datasets. Instead of that we are looking for the "stability” of the result. It is a known fact that for different examples specially selected different approaches are better – in one case it might be a decision tree, in another case it might be multilayer perceptron. So the idea for comparison is artificial creation of datasets with gradation from first case to second one. The more robustly the algorithm works on the aggregation of all sets the better we consider it.