XVAL is used for crossvalidation
[R,CC] = xval(D,classlabel)
.. = xval(D,classlabel,CLASSIFIER)
.. = xval(D,classlabel,CLASSIFIER,type)
.. = xval(D,{classlabel,W},CLASSIFIER)
.. = xval(D,{classlabel,W,NG},CLASSIFIER)
example:
load_fisheriris; %builtin iris dataset
C = species;
K = 5; NG = [1:length(C)]'*K/length(C);
[R,CC] = xval(meas,{C,[],NG},'NBC');
Input:
D: data features (one feature per column, one sample per row)
classlabel labels of each sample, must have the same number of rows as D.
Two different encodings are supported:
{-1,1}-encoding (multiple classes with separate columns for each class) or
1..M encoding.
So [1;2;3;1;4] is equivalent to
[+1,-1,-1,-1;
[-1,+1,-1,-1;
[-1,-1,+1,-1;
[+1,-1,-1,-1]
[-1,-1,-1,+1]
Note, samples with classlabel=0 are ignored.
CLASSIFIER can be any classifier supported by train_sc (default='LDA')
{'REG','MDA','MD2','QDA','QDA2','LD2','LD3','LD4','LD5','LD6','NBC','aNBC','WienerHopf', 'RDA','GDBC',
'SVM','RBF','PSVM','SVM11','SVM:LIN4','SVM:LIN0','SVM:LIN1','SVM:LIN2','SVM:LIN3','WINNOW'}
these can be modified by ###/GSVD, ###/sparse and ###/DELETION.
/DELETION removes in case of NaN's either the rows or the columns (which removes less data values) with any NaN
/sparse and /GSVD preprocess the data an reduce it to some lower-dimensional space.
Hyperparameters (like alpha for PLA, gamma/lambda for RDA, c_value for SVM, etc) can be defined as
CLASSIFIER.hyperparameter.alpha, etc. and
CLASSIFIER.TYPE = 'PLA' (as listed above).
See train_sc for details.
W: weights for each sample (row) in D.
default: [] (i.e. all weights are 1)
number of elements in W must match the number of rows of D
NG: used to define the type of cross-valdiation
Leave-One-Out-Method (LOOM): NG = [1:length(classlabel)]' (default)
Leave-K-Out-Method: NG = ceil([1:length(classlabel)]'/K)
K-fold XV: NG = ceil([1:length(classlabel)]'*K/length(classlabel))
group-wise XV (if samples are not indepentent) can be also defined here
samples from the same group (dependent samples) get the same identifier
samples from different groups get different classifiers
TYPE: defines the type of cross-validation procedure if NG is not specified
'LOOM' leave-one-out-method
k k-fold crossvalidation
OUTPUT:
R contains the resulting performance metric
R.OUTPUT classifier output
R.CL class labels
R.H confusion matrix
R.kappa Cohen's kappa coefficient
R.ACC accuracy
R.ERR error rate
... and a number of addition parameters obtained from the confusionm matrix [2]
R.noXV.{...} contains the result w/o crossvalidation
CC contains the classifier
plota(R) shows the confusion matrix of the results
see also: TRAIN_SC, TEST_SC, CLASSIFY, PLOTA
References:
[1] R. Duda, P. Hart, and D. Stork, Pattern Classification, second ed.
John Wiley & Sons, 2001.
[2] A. Schlögl, J. Kronegg, J.E. Huggins, S. G. Mason;
Evaluation criteria in BCI research.
(Eds.) G. Dornhege, J.R. Millan, T. Hinterberger, D.J. McFarland, K.-R.Müller;
Towards Brain-Computer Interfacing, MIT Press, 2007, p.327-342
Package: nan