Sepp Hochreiter
hochreit@cs.tu-berlin.de
We introduce a new feature selection method, called ``Potential Support
Vector Machine'' (P-SVM), which is based on the support vector
machine technique and extracts features from a data set which are important
for constructing a classifier. Like standard SVMs, the P-SVM
is based on the structural risk minimization principle but in contrast
to standard SVMs it minimizes a new objective function. This new
objective expresses an upper bound on the generalization error which is
obtained by computing bounds through covering numbers. Another difference
to standard SVMs is that the P-SVM's class separating hyperplane
is described by feature vectors (the so-called ``support features'') which
formally assume the role of support vectors. Therefore, feature selection
is simply the identification of support vectors. To introduce feature vectors,
the P-SVM treats the given data matrix as dot product matrix between feature
vectors and vectors to classify. For example a gene expression matrix is
considered as a matrix of dot products between fixed gene vectors and variable
tissue vectors. To ensure that a data matrix is indeed a dot product
matrix, the measurement of a feature in an object must obey a given protocol
so that the expected feature value for an object remains constant. The
performance of the P-SVM feature selection method is demonstrated on data
sets obtained from patients with certain types of cancer (brain tumor,
lymphoma, and breast cancer), where the outcome of a chemo- or radiation
therapy must be predicted based on the gene expression profile. For
classification after P-SVM feature selection, generalization performance
is improved compared to previously proposed methods. Additionally,
the P-SVM extracts genes which may be important for drug development.