Random Search Toolbox

Introduction RS-toolbox

A major issue with many signal processing and machine learning algorithms is the lack of optimisation methods for determining the numerous hyper-parameters associated with the model as well as the knowledge of which hyper-parameters are relevant. These parameters are usually tunned by trial and error (manual search) or by grid search. In 2012 Bergstra et al. showed empirically that random search was performing similar or better than grid search while reducing the computational cost substantially.

RS-toolbox is contributed by Joachim Behar, Alistair Johnson, Julien Oster and Gari D Clifford. All authors are part of the institute of biomedical engineering at the University of Oxford. The source code is available in MATLAB rs-toolbox.zip

The RS-toolbox code provides functions for creating Efficiency Curves (EC) and Automatic Relevance Determination (ARD) plots. The functions can easily be run to reproduce the plots of this example. Start by simply launching the ESN_EfficiencyCurve.m script in order to produce a similar EC as Figure 1 of the example and then launch ESN_IntrinsicDimensionality.m in order to produce an ARD plot as Figure 2 of the example. This code together with its example of parameters search for training an Echo State Neural network should provide the users with a quick insight on how to use random search and produce EC and ARD from the searched parameters.

General remark on Random Search and ARD: be aware that the interval that you sample your variables from will have an impact on the ARD plots and the conclusions you will make from it!