This file was created by the Typo3 extension
sevenpack version 0.7.14
--- Timezone: CEST
Creation date: 2017-06-28
Creation time: 21-18-10
--- Number of references
22
article
6038
Falsificationism and Statistical Learning Theory: Comparing the Popper and Vapnik-Chervonenkis Dimensions
Journal for General Philosophy of Science
2009
7
40
1
51-58
We compare Karl Poppers ideas concerning the falsifiability of a theory with similar notions from the part of statistical learning theory known as VC-theory. Poppers notion of the dimension of a theory is contrasted with the apparently very similar VC-dimension. Having located some divergences, we discuss how best to view Poppers work from the perspective of statistical learning theory, either as a precursor or as aiming to capture a different learning activity.
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Schölkopf
http://www.springerlink.com/content/n2420j2072h67722/fulltext.pdf
Biologische Kybernetik
Max-Planck-Gesellschaft
en
10.1007/s10838-009-9091-3
dcorfieldDCorfield
bsBSchölkopf
vapnikVVapnik
article
1436
Choosing Multiple Parameters for Support Vector Machines
Machine Learning
2002
46
1
131-159
The problem of automatically tuning multiple parameters for pattern recognition Support Vector Machines (SVM) is considered. This is done by minimizing some estimates of the generalization error of SVMs using a gradient descent algorithm over the set of parameters. Usual methods for choosing parameters, based on exhaustive search become intractable as soon as the number of parameters exceeds two. Some experimental results assess the feasibility of our approach for a large number of parameters (more than 100) and demonstrate an improvement of generalization performance.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/pdf1436.pdf
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/ps1436.ps
http://www.kyb.tuebingen.mpg.de
Biologische Kybernetik
Max-Planck-Gesellschaft
chapelleOChapelle
vapnikVVapnik
bousquetOBousquet
SMukherjee
article
2165
Model Selection for Small Sample Regression
Machine Learning
2002
48
1-3
9-23
Model selection is an important ingredient of many machine
learning algorithms, in particular when the sample size in
small, in order to strike the right trade-off between overfitting
and underfitting. Previous classical results for linear regression
are based on an asymptotic analysis. We present a new
penalization method for performing model selection for
regression that is appropriate even for small samples.
Our penalization is based on an accurate estimator of the
ratio of the expected training error and the expected
generalization error, in terms of the expected eigenvalues
of the input covariance matrix.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/ps2165.ps
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Biologische Kybernetik
Max-Planck-Gesellschaft
chapelleOChapelle
vapnikVVapnik
YBengio
article
2158
Bounds on Error Expectation for Support Vector Machines
Neural Computation
2000
12
9
2013-2036
We introduce the concept of span of support vectors (SV) and show
that the generalization ability of support vector machines (SVM) depends on this
new
geometrical concept. We prove that the value of the span is always smaller
(and can be much smaller) than the diameter of the smallest sphere containing th
e support
vectors, used in previous bounds. We also demonstate
experimentally that the prediction of the test error given by the span is very
accurate and has direct application in model selection (choice of the optimal
parameters of the SVM)
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/ps2158.gz
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Biologische Kybernetik
Max-Planck-Gesellschaft
vapnikVVapnik
chapelleOChapelle
article
2160
SVMs for Histogram Based Image Classification
IEEE Transactions on Neural Networks
1999
9
Traditional classification approaches generalize poorly on image
classification tasks, because of the high dimensionality of the feature space.
This paper shows that Support Vector Machines (SVM) can generalize well on
difficult image classification problems where the only features are high
dimensional histograms. Heavy-tailed RBF kernels of the form
$K(mathbf{x},mathbf{y})=e^{-rhosum_i |x_i^a-y_i^a|^{b}}$ with $aleq 1$
and $b leq 2$ are evaluated on the classification of images extracted from
the Corel Stock Photo Collection and shown to far outperform traditional
polynomial or Gaussian RBF kernels. Moreover, we observed that a simple
remapping of the input $x_i rightarrow x_i^a$ improves the performance of
linear SVMs to such an extend that it makes them, for this problem, a valid
alternative to RBF kernels.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/ps2160.gz
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Biologische Kybernetik
Max-Planck-Gesellschaft
chapelleOChapelle
PHaffner
vapnikVVapnik
article
378
Comparing support vector machines with Gaussian kernels to radial basis function classifiers
IEEE Transactions on Signal Processing
1997
11
45
11
2758-2765
The support vector (SV) machine is a novel type of learning machine, based on statistical learning theory, which contains polynomial classifiers, neural networks, and radial basis function (RBF) networks as special cases. In the RBF case, the SV algorithm automatically determines centers, weights, and threshold that minimize an upper bound on the expected test error. The present study is devoted to an experimental comparison of these machines with a classical approach, where the centers are determined by X-means clustering, and the weights are computed using error backpropagation. We consider three machines, namely, a classical RBF machine, an SV machine with Gaussian kernel, and a hybrid system with the centers determined by the SV method and the weights trained by error backpropagation. Our results show that on the United States postal service database of handwritten digits, the SV machine achieves the highest recognition accuracy, followed by the hybrid system. The SV approach is thus not only theoretically well-founded but also superior in a practical application.
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Bülthoff
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=650102&tag=1
Biologische Kybernetik
Max-Planck-Gesellschaft
10.1109/78.650102
bsBSchölkopf
KSung
CBurges
FGirosi
PNiyogi
TPoggio
vapnikVVapnik
inproceedings
3916
Inference with the Universum
2006
6
1009-1016
In this paper we study a new framework introduced by Vapnik (1998) and Vapnik (2006) that is an alternative capacity concept to the large margin approach. In the particular case of binary classification, we are given a set of labeled examples, and a collection of "non-examples" that do not belong to either class of interest. This collection, called the Universum, allows one to encode prior knowledge by representing meaningful concepts in the same domain as the problem at hand. We describe an algorithm to leverage the Universum by maximizing the number of observed contradictions, and show experimentally that this approach delivers accuracy improvements over using labeled data alone.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/icml_3916[0].pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Schölkopf
http://portal.acm.org/citation.cfm?id=1143971&jmp=abstract&coll=GUIDE&dl=GUIDE&CFID=1380099&CFTOKEN=72073021#abstract
Cohen, W. W., A. Moore
ACM Press
New York, NY, USA
Biologische Kybernetik
Max-Planck-Gesellschaft
Pittsburgh, PA, USA
23rd International Conference on Machine Learning (ICML 2006)
en
1-59593-383-2
10.1145/1143844.1143971
westonJWeston
RCollobert
fabeeFSinz
LBottou
vapnikVVapnik
inproceedings
2053
Kernel Dependency Estimation
2003
10
873-880
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/NIPS-2002-Weston.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Schölkopf
http://books.nips.cc/nips15.html
Becker, S. , S. Thrun, K. Obermayer
MIT Press
Cambridge, MA, USA
Advances in Neural Information Processing Systems 15
Biologische Kybernetik
Max-Planck-Gesellschaft
Vancouver, BC, Canada
Sixteenth Annual Conference on Neural Information Processing Systems (NIPS 2002)
0-262-02550-7
westonJWeston
chapelleOChapelle
andreAElisseeff
bsBSchölkopf
vapnikVVapnik
inproceedings
2164
Feature Selection for SVMs
2001
4
668-674
We introduce a method of feature selection for Support Vector Machines. The method is based upon finding those features which minimize bounds on the leave-one-out error. This search can be efficiently performed via gradient descent. The resulting algorithms are shown to be superior to some standard feature selection algorithms on both toy data and real-life problems of face recognition, pedestrian detection and analyzing DNA microarray data.
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/NIPS-2000-Weston.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://books.nips.cc/nips13.html
Leen, T.K. , T.G. Dietterich, V. Tresp
MIT Press
Cambridge, MA, USA
Advances in Neural Information Processing Systems 13
Biologische Kybernetik
Max-Planck-Gesellschaft
Denver, CO, USA
Fourteenth Annual Neural Information Processing Systems Conference (NIPS 2000)
0-262-12241-3
westonJWeston
SMukherjee
chapelleOChapelle
MPontil
TPoggio
vapnikVVapnik
inproceedings
2163
Vicinal Risk Minimization
2001
4
416-422
The Vicinal Risk Minimization principle establishes a bridge between generative models and methods derived from the Structural Risk Minimization Principle such as Support Vector Machines or Statistical Regularization. We
explain how VRM provides a framework which integrates a number of existing algorithms, such as Parzen windows, Support Vector Machines, Ridge Regression, Constrained Logistic Classifiers and Tangent-Prop. We then show how the
approach implies new algorithms for solving problems usually associated with generative models. New algorithms are described for dealing with pattern recognition problems with very different pattern distributions and dealing
with unlabeled data. Preliminary empirical results are presented.
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/NIPS-2000-Chapelle.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://books.nips.cc/nips13.html
Leen, T.K. , T.G. Dietterich, V. Tresp
MIT Press
Cambridge, MA, USA
Advances in Neural Information Processing Systems 13
Biologische Kybernetik
Max-Planck-Gesellschaft
Denver, CO, USA
Fourteenth Annual Neural Information Processing Systems Conference (NIPS 2000)
0-262-12241-3
chapelleOChapelle
westonJWeston
LBottou
vapnikVVapnik
inproceedings
2161
Model Selection for Support Vector Machines
2000
6
230-236
New functionals for parameter (model) selection of Support Vector Machines are introduced based on the concepts of the span of support vectors and rescaling of the feature space. It is shown that using these functionals, one can both predict the best choice of parameters of the model and the relative quality of performance for any value of parameter.
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/NIPS-1999-Chapelle.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://books.nips.cc/nips12.html
Solla, S.A. , T.K. Leen, K-R Müller
MIT Press
Cambridge, MA, USA
Advances in Neural Information Processing Systems 12
Biologische Kybernetik
Max-Planck-Gesellschaft
Denver, CO, USA
Thirteenth Annual Neural Information Processing Systems Conference (NIPS 1999)
0-262-19450-3
chapelleOChapelle
vapnikVVapnik
inproceedings
2162
Transductive Inference for Estimating Values of Functions
2000
6
421-427
We introduce an algorithm for estimating the values of a
function at a set of test points $x_1^*,dots,x^*_m$ given a set of training points $(x_1,y_1),dots,(x_ell,y_ell)$ without estimating (as an intermediate step) the regression function. We demonstrate that this direct (transductive) way for estimating values of the regression (or classification in pattern recognition) is more accurate than the traditional one
based on two steps, first estimating the function and then
calculating the values of this function at the points of interest.
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/NIPS-1999-Chapelle-2.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://books.nips.cc/nips12.html
Solla, S.A. , T.K. Leen, K-R Müller
MIT Press
Cambridge, MA, USA
Advances in Neural Information Processing Systems 12
Biologische Kybernetik
Max-Planck-Gesellschaft
Denver, CO, USA
Thirteenth Annual Neural Information Processing Systems Conference (NIPS 1999)
0-262-19450-3
chapelleOChapelle
vapnikVVapnik
westonJWeston
inproceedings
MullerSRSKV1999
Using support vector machines for time series prediction
1999
243-253
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://dl.acm.org/citation.cfm?id=299107&CFID=751886038&CFTOKEN=40994378
Schölkopf, B. , C.J.C. Burges, A.J. Smola
MIT Press
Cambridge, MA, USA
Advances in kernel methods: support vector learning
Denver, CO, USA
Eleventh Annual Conference on Neural Information Processing (NIPS 1997)
0-262-19416-3
klausK-RMüller
smolaAJSmola
raetschGRätsch
bsBSchölkopf
JKohlmorgen
vapnikVVapnik
inproceedings
798
Prior knowledge in support vector kernels
1998
6
640-646
We explore methods for incorporating prior knowledge about a problem at hand in Support Vector learning machines. We show that both invariances under group transformations and prior knowledge about locality in images can be incorporated by constructing appropriate kernel functions.
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/NIPS-1997-Schoelkopf.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Bülthoff
http://books.nips.cc/nips10.html
Jordan, M.I. , M.J. Kearns, S.A. Solla
MIT Press
Cambridge, MA, USA
A Bradford Book
Advances in Neural Information Processing Systems 10
Biologische Kybernetik
Max-Planck-Gesellschaft
Denver, CO, USA
Eleventh Annual Conference on Neural Information Processing (NIPS 1997)
0-262-10076-2
bsBSchölkopf
PSimard
smolaAJSmola
vapnikVVapnik
inproceedings
1823
Support Vector methods in learning and feature extraction
1998
2
72-78
The last years have witnessed an increasing interest in Support Vector (SV) machines, which use Mercer kernels for efficiently performing computations in high-dimensional spaces. In pattern recognition, the SV algorithm constructs nonlinear decision functions by training a classifier to perform a linear separation in some high-dimensional space which is nonlinearly related to input space. Recently, we have developed a technique for Nonlinear Principal Component Analysis (Kernel PCA) based on the same types of kernels. This way, we can for instance efficiently extract polynomial features of arbitrary order by computing projections onto principal components in the space of all products of n pixels of images. We explain the idea of Mercer kernels and associated feature spaces, and describe connections to the theory of reproducing kernels and to regularization theory, followed by an overview of the above algorithms employing these kernels. 1. Introduction For the case of two-class pattern.
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.42.1638
Downs, T. , M. Frean, M. Gallagher
University of Queensland
Brisbane, Australia
Biologische Kybernetik
Max-Planck-Gesellschaft
Brisbane, Australia
Ninth Australian Conference on Neural Networks (ACNN'98)
1-86499-026-0
bsBSchölkopf
smolaAJSmola
klausK-RMüller
CBurges
vapnikVVapnik
inproceedings
416
Predicting time series with support vector machines
1997
10
999-1004
Support Vector Machines are used for time series prediction and compared to radial basis function networks. We make use of two different cost functions for Support Vectors: training with (i) an e insensitive loss and (ii) Huber's robust loss function and discuss how to choose the regularization parameters in these models. Two applications are considered: data from (a) a noisy (normal and uniform noise) Mackey Glass equation and (b) the Santa Fe competition (set D). In both cases Support Vector Machines show an excellent performance. In case (b) the Support Vector approach improves the best known result on the benchmark by a factor of 29%.
http://www.kyb.tuebingen.mpg.defileadmin/user_upload/files/publications/ICANN-1997-Mueller.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Bülthoff
http://www.springerlink.com/content/563446j604043340/fulltext.pdf
Gerstner, W. , A. Germond, M. Hasler, J.-D. Nicoud
Springer
Berlin, Germany
Lecture Notes in Computer Science ; 1327
Artificial Neural Networks - ICANN '97
Biologische Kybernetik
Max-Planck-Gesellschaft
Lausanne, Switzerland
7th International Conference on Artificial Neural Networks
3-540-63631-5
10.1007/BFb0020283
klausK-RMüller
smolaAJSmola
raetschGRätsch
bsBSchölkopf
JKohlmorgen
vapnikVVapnik
inproceedings
445
Comparison of view-based object recognition algorithms using realistic 3D models
1996
7
251-256
Two view-based object recognition algorithms are compared: (1) a heuristic algorithm based on oriented filters, and (2) a support vector learning machine trained on low-resolution images of the objects. Classification performance is assessed using a high number of images generated by a computer graphics system under precisely controlled conditions. Training- and test-images show a set of 25 realistic three-dimensional models of chairs from viewing directions spread over the upper half of the viewing sphere. The percentage of correct identification of all 25 objects is measured.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/pdf445.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Bülthoff
http://www.springerlink.com/content/p936668278821327/fulltext.pdf
von der Malsburg, C. , W. von Seelen, J. C. Vorbrüggen and B. Sendhoff
Springer
Berlin, Germany
Lecture Notes in Computer Science ; 1112
Artificial Neural Networks - ICANN 96
Biologische Kybernetik
Max-Planck-Gesellschaft
Bochum, Germany
6th International Conference on Artificial Neural Networks
3-540-61510-5
10.1007/3-540-61510-5_45
volkerVBlanz
bsBSchölkopf
hhbHHBülthoff
CBurges
vapnikVVapnik
vetterTVetter
inproceedings
796
Incorporating invariances in support vector learning machines
1996
7
47-52
Developed only recently, support vector learning machines achieve high generalization ability by minimizing a bound on the expected test error; however, so far there existed no way of adding knowledge about invariances of a classification problem at hand. We present a method of incorporating prior knowledge about transformation invariances by applying transformations to support vectors, the training examples most critical for determining the classification boundary.
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Bülthoff
http://www.springerlink.com/content/p27724q228212166/fulltext.pdf
von der Malsburg, C. , W. von Seelen, J. C. Vorbrüggen, B. Sendhoff
Springer
Berlin, Germany
Lecture Notes in Computer Science ; 1112
Artificial Neural Networks - ICANN 96
Biologische Kybernetik
Max-Planck-Gesellschaft
Bochum, Germany
6th International Conference on Artificial Neural Networks
3-540-61510-5
10.1007/3-540-61510-5_12
bsBSchölkopf
CBurges
vapnikVVapnik
inproceedings
795
Extracting support data for a given task
1995
8
252-257
We report a novel possibility for extracting a small subset of a data base which contains all the information necessary to solve a given classification task: using the Support Vector Algorithm to train three different types of handwritten digit classifiers, we observed that these types
of classifiers construct their decision surface from
strongly overlapping small (k: 4%) subsets of the data base. This finding opens up the possibiiity of compressing data bases significantly by disposing of the data which is not important for the solution of a given task. In addition, we show that the theory allows us to predict the classifier that will have the best generalization ability, based solely on performance on the training set and characteristics of the learning machines. This finding is important for cases where the amount of available data is limited.
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Bülthoff
http://www.aaai.org/Papers/KDD/1995/KDD95-030.pdf
Fayyad, U.M. , R. Uthurusamy
AAAI Press
Menlo Park, CA, USA
Biologische Kybernetik
Max-Planck-Gesellschaft
Montréal, Canada
First International Conference on Knowledge Discovery & Data Mining (KDD-95)
0-929280-82-2
bsBSchölkopf
CBurges
vapnikVVapnik
techreport
3724
Popper, Falsification and the VC-dimension
2005
11
145
We compare Sir Karl Popper’s ideas concerning the falsifiability of a theory with similar notions from
VC-theory. Having located some divergences, we discuss how best to view Popper’s work from the perspective of
statistical learning theory.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/TR_145_[0].pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Schölkopf
Biologische Kybernetik
Max-Planck-Gesellschaft
Max Planck Institute for Biological Cybernetics, Tübingen, Germany
en
dcorfieldDCorfield
bsBSchölkopf
vapnikVVapnik
techreport
1864
Kernel Dependency Estimation
2002
8
98
We consider the learning problem of finding a dependency between a general class of objects and another, possibly different, general class of objects. The objects can be for example: vectors, images, strings, trees or graphs. Such a task is made possible by employing similarity measures in both input and output spaces using kernel functions, thus embedding the objects into vector spaces. Output kernels also make it possible to encode prior information and/or invariances in the loss function in an elegant way. We experimentally validate our approach on several tasks: mapping strings to strings, pattern recognition, and reconstruction from partial images.
http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/pdf1864.pdf
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Department Schölkopf
Biologische Kybernetik
Max-Planck-Gesellschaft
Max Planck Institute for Biological Cybernetics, Tübingen, Germany
westonJWeston
chapelleOChapelle
andreAElisseeff
bsBSchölkopf
vapnikVVapnik
techreport
1863
A New Method for Constructing Artificial Neural Networks
1995
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
http://www.kyb.tuebingen.mpg.de
Biologische Kybernetik
Max-Planck-Gesellschaft
AT & T Bell Laboratories
vapnikVVapnik
CJCBurges
bsBSchölkopf