Academic Journals Database
Disseminating quality controlled scientific knowledge

The theory and application of penalized methods or Reproducing Kernel Hilbert Spaces made easy

Author(s): Nancy Heckman

Journal: Statistics Surveys
ISSN 1935-7516

Volume: 6;
Start page: 113;
Date: 2012;
Original page

Keywords: Penalized likelihood | Reproducing Kernel Hilbert Space | splines.

The popular cubic smoothing spline estimate of a regression functionarises as the minimizer of the penalized sum of squares $sum_j(Y_j -mu(t _j))^2 + lambda int_a^b [ mu''(t)]^2~dt$, where the data are $t_j,Y_j$, $j=1,ldots, n$. The minimization is taken over aninfinite-dimensional function space, the space of all functions with square integrable second derivatives. But the calculations can be carried out in a finite-dimensional space. The reduction from minimizing over an infinite dimensional space to minimizing over a finite dimensional space occurs for more general objective functions: the data may be related to the function $mu$ in another way, the sum of squares may be replaced by a more suitable expression, or the penalty, $int_a^b [ mu''(t)]^2~dt$, might take a different form. This paper reviews the Reproducing Kernel Hilbert Space structure that provides a finite-dimensional solution for a general minimization problem. Particular attention is paid to the construction and study of the Reproducing Kernel Hilbert Space corresponding to a penalty based on a linear differential operator. In this case, one can often calculate the minimizer explicitly, using Green's functions.

Tango Jona
Tangokurs Rapperswil-Jona

RPA Switzerland

Robotic Process Automation Switzerland