Optimiser Algorithms

The optimiser algorithms are provided by the free/open source library NLopt developed and maintained by Steven G. Johnson who is professor of Applied Mathematics and Physics at MIT.

The library contains over 20 optimiser algorithms, however, for circuit simulation applications we have identified two algorithms in the library that give consistently superior results. These are:

  • Nelder-Mead - for unconstrained problems, that is problems that only define an objective function such as is usually the case for curve fitting. Nelder-Mead is also known as Downhill Simplex. The paper that first described this method was first published in 1966. Many researchers dismiss this algorithm as it is known to be capable of converging on a non-optimal point. While this is undoubtedly true, our experience is that it works well for many circuit simulation problems and is tolerant of the noise and discontinuities that can be present in that application.
  • COBYLA - for constrained problems. COBYLA (Constrained Optimization BY Linear Approximations) was developed by the late Michael Powell who was a professor of Mathematics at Cambridge university

Currently SIMetrix only supports "Derivative-free" optimisation algorithms. That is algorithms that do not require the calculation of the partial derivatives (or sensitivities) of each objective and constraint with respect to every parameter. The calculation of each derivative requires one additional simulation run for each parameter and so is in general expensive. However, algorithms that do require derivatives to be calculated, usually converge much more quickly.

In this topic:

Experimental Algorithms

Many of the other available algorithms may be used on an experimental basis. Check the Show experimental box and the algorithm drop down box will be populated with many more choices. Here are some details about each algorithm:

Algorithm Details
NELDER_MEAD_AL Supports constraints. Nelder-Mead coupled with Augmented Lagrangian, see notes below
BOBYQA Unconstrained only. Bound Optimization BY Quadratic Approximation. Developed by Michael Powell. This algorithm attempts to model the objective function as a quadratic which tends not to work that well in circuit simulation applications
BOBYQA_AL Supports constraints. BOBYQA coupled with Augmented Lagrangian, see notes below
SBPLX Unconstrained only. This is a variant of Nelder Mead and claims to be superior. However this is not what we have found in circuit simulation applications
SBPLX_AL Supports constraints. SBPLX coupled with Augmented Lagrangian, see notes below
PRAXIS Unconstrained only. Principal axis method developed by Richard Brent
PRAXIS_AL Supports constraints. PRAXIS coupled with Augmented Lagrangian, see notes below

Augmented Lagrangian

The augmented Lagrangian algorithm is a derivation of Lagrangian multipliers and is used to convert a constrained problem into an unconstrained problem. Each of the unconstrained local optimiser methods have augmented Lagrangian variations that have the suffix _AL. Important: the augmented Lagrangian method does not normalise the constraints. To get the constraint functions to work with any of these methods, it is necessary to apply appropriate scaling to each constraint function. For example, in the amp-700 circuit, the distortion constraint has almost no effect unless it is multiplied by about 1000, so the limit changes from 0.01 to 10. This is not the case with the COBYLA algorithm which supports constraints directly and does not require special scaling.

◄ Optimiser GUI Description   Overview ▶