FINITE-SIZE EFFECTS AND OPTIMAL TEST SET SIZE IN LINEAR PERCEPTRONS

Fluctuations in the test error are important in the learning theory of finite-dimensional systems as they represent how well the test error matches the average test error. By explicitly finding the variance of the test error due to randomness present in both the data set and algo rithm for a linear perceptron of dimension n, we are able to address s uch questions as the optimal test set size. Where exact results were n ot tractable, a good approximation is given to the variance. We find t hat the optimal test set size possesses a phase transition between lin ear and 2/3 power-law scaling in the system size n.