Minimum description length and multi-criteria decision analysis in predictive modelling

Abstract

Accurate model selection is essential in predictive modelling across various domains, significantly impacting decision-making and resource allocation. Despite extensive research, the model selection process remains challenging. This work aims to integrate the Minimum Description Length principle with the Multi-Criteria Decision Analysis to enhance the selection of forecasting machine learning models. The proposed MDL-MCDA framework combines the MDL principle, which balances model complexity and data fit, with the MCDA, which incorporates multiple evaluation criteria to address conflicting error measurements. Four datasets from diverse domains, including software engineering (effort estimation), healthcare (glucose level prediction), finance (GDP prediction), and stock market prediction, were used to validate the framework. Various regression models and feed-forward neural networks were evaluated using criteria such as MAE, MAPE, RMSE, and Adjusted R2. We employed the Analytic Hierarchy Process (AHP) to determine the relative importance of these criteria. We conclude that the integration of MDL and MCDA significantly improved model selection across all datasets. The cubic polynomial regression model and the multi-layer perceptron models outperformed other models in terms of AHP score and MDL criterion. Specifically, the MDL-MCDA approach provided a more nuanced evaluation, ensuring the selected models effectively balanced complexity and predictive accuracy.