Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
A Generalized Stacking Method Using Matrix Ensemble Kalman Filter-Based Multi-Arm Neural Network
Abstract
Deep learners (DLs) have turned out to be the state-of-the-art method for predictive inference. Since we do not have widely applicable generalization error bounds for DLs, we can prevent over-confident inferences and predictions from a single “best performing” DL by creating an ensemble of such models and then performing model averaging. Such model averaging has been shown to increase robustness in the realm of deep learning techniques. This increase in robustness could be partially attributed to the fact that with model averaging, we no longer ignore the uncertainty due to model choice. Stacking is one of the most popular model-averaging protocols. In its standard form, stacking uses the output of base learners as non-stochastic inputs to a meta-learner. However, that ignores the uncertainty in the predictions generated by these base DLs. This practice is problematic because DLs are often trained with dropout layers, which induce uncertainty in their predictions. Consequently, a meta-learner should process that uncertainty hardwired into the base models.In this dissertation, we derive a novel methodology that can perform model averaging and propagate the uncertainty associated with the base models more coherently. We utilize Matrix Ensemble Kalman Filters to design a multi-arm artificial neural network that drives stochastic weights and performs model averaging in every filter update and batch update step. By default, our method produces realizations from one-step ahead predictive distribution, enabling the construction of prediction intervals from averaged models. We demonstrate that our methodology can be utilized for transfer learning and potentially identify a specific form of mean non-stationarity in the underlying data-generating model. We apply our model to cancer drug response predictions and classification of gut microbiota. All codes used in this dissertation can be obtained from: https://github.com/Ved-Piyush/UNL Thesis Codes VP/tree/main.
Subject Area
Statistics|Computer Engineering|Artificial intelligence
Recommended Citation
Piyush, Ved, "A Generalized Stacking Method Using Matrix Ensemble Kalman Filter-Based Multi-Arm Neural Network" (2023). ETD collection for University of Nebraska-Lincoln. AAI30814015.
https://digitalcommons.unl.edu/dissertations/AAI30814015