Date of this Version
Shrinkage methods are estimation techniques based on optimizing expressions to find which variables to include in an analysis, typically a linear regression. The general form of these expressions is the sum of an empirical risk plus a complexity penalty based on the number of parameters. Many shrinkage methods are known to satisfy an ‘oracle’ property meaning that asymptotically they select the correct variables and estimate their coefficients efficiently. In Section 1.2, we show oracle properties in two general settings. The first uses a log likelihood in place of the empirical risk and allows a general class of penalties. The second uses a general class of empirical risks and a general class of penalties obtaining limiting behavior for a large class of smooth likelihoods. The second contribution of this thesis is to realize that shrinkage techniques with oracle properties are asymptotically the same, but differ in their finite sample properties. To address this, in Section 2.1, we propose selection of a shrinkage method based on a stability criterion. Part of our analysis in Section 2.2 is a computational comparison of several specific shrinkage methods. In future work, we hope to optimize a stability criterion directly to derive a data driven shrinkage method using techniques from genetic algorithms. We describe this in Section 2.3 as future work.
Adviser: Bertrand Clarke