Convex Optimization Approaches for Model Selection

Kristiaan Pelckmans and Johan Suykens

K.U. Leuven, ESAT-SCD

In this talk we review recent progress in solving model selection tasks by using convex optimization techniques. We present different ways of characterizing the training problem of a regularized learning machine with respect to the choice of tuning parameters, and the subsequent optimization of these tuning parameters with respect to a model selection criterion. To make this approach practically feasible, relaxations of the resulting multi-level inference problems are derived in terms of a convex optimization problem. The ideas are illustrated for the use of least squares support vector machines, and extensions are given for other learning machines.