Title: The effectiveness of derivative-free hybrid methods for black-box optimisation

Authors: G.A. Gray, K.R. Fowler

Addresses: Department of Quantitative Modeling and Analysis, Sandia National Laboratories, P.O. Box 969, MS 9159, Livermore, CA 94551-0969, USA. ' Department of Mathematics and Computer Science, Clarkson University, P.O. Box 5815, Potsdam, NY 13699-5815, USA

Abstract: Black-box optimisation problems are common in many applications of science and engineering. However, they are complicated in the sense that derivatives are unavailable and approximate derivatives are often unreliable. Moreover, evaluation the objective function and/or the constraints typically requires the results of a computationally expensive simulation. Derivative-free methods have emerged as invaluable for finding solutions to these problems. A wide variety of methods have been developed and each has distinct advantages and disadvantages. Therefore, we consider a hybrid approach to optimisation which allows the combining of beneficial elements of multiple methods in order to more efficiently search the design space. In this paper, we will first describe four derivative-free optimisation approaches – asynchronous parallel pattern search (APPS), implicit filtering, dividing rectangles (DIRECT), and a genetic algorithm (GA). We will also describe how statistical emulation can be used as an alternative to traditional optimisation. We will give the advantages and disadvantages of each approach. Then, we will explain how these five approaches are used to form the hybrids APPS-TGP, DIRECT-IFFCO, DIRECT-TGP, and EAGLS which exploiting the advantages and overcoming the disadvantages of the underlying methods. We also include references to papers which illustrate practical examples of these methods.

Keywords: black-box function; derivative-free optimisation; hybrid optimisation; mixed-variable optimisation; nonlinear optimisation; MINLP; black-box optimisation; derivatives; asynchronous parallel pattern search; APPS; implicit filtering; dividing rectangles; genetic algorithms; statistical emulation.

DOI: 10.1504/IJMMNO.2011.039424

International Journal of Mathematical Modelling and Numerical Optimisation, 2011 Vol.2 No.2, pp.112 - 133

Published online: 26 Mar 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article