# Optimization algorithms¶

The `<fitting>`

block in the **atomicrex** input file controls
the optimization procedure. The tag can be supplemented by two
optional attributes as shown in the following example:

```
<fitting enabled="true" output-interval="100">
...
</fitting>
```

Here, `enabled`

can be used to selectively activate (`True`

) or
deactivate (`False`

) model optimization without the need to remove
the entire block. The `output-interval`

attribute sets the number
of iterations, between which the code reports its current status
(i.e., potential parameters, property data etc.).

At present **atomicrex** *directly* supports the following
optimization algorithms:

the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) minimizer with constraints

the Spa minimizer for simply poking around

the split-Bregman minimizer (only useful for certain models)

the algorithms provided by the NLopt library

There are several *optional* parameters, which are provided as
attributes within the minimizer tag (see below for examples). They
allow one to modify the criteria, which cause the minimization loop to
be terminated.

The first parameter `conv-threshold`

defines a condition for the
maximum variation of the objective function between successive iterations \(k\) and
\(k+1\). The minimization is halted if

where \(c_t\) is set by `conv-threshold`

and \(p_m\) is the
machine precision, which is automatically generated by the code. The
default value is \(10^{-5}\).

If the number of iterations exceeds `max-iter`

the optimization is
terminated. The default value is 100.

The gradient of the objective function is evaluated numerically using central differences:

The step width \(\varepsilon\) is specified using the attribute
`gradient-epsilon`

and defaults to \(10^{-6}\).

## L-BFGS minimizer¶

**atomicrex** includes an implementation of the limited-memory
Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) minimizer, which is a
popular quasi-Newton method, with support for constraints. An
extensive description of the algorithm can be found on this Wikipedia
page. The
minimizer can be activated by inserting the `<BFGS>`

tag in the
`<fitting>`

block of the main input file with options provided via
attributes as illustrated by the following code block:

```
<fitting>
<BFGS conv-threshold="1e-16" max-iter="200" gradient-epsilon="1e-8" />
</fitting>
```

where the default values for the optional parameters described above have been overriden.

## Spa minimizer¶

The Spa minimizer is actually not a separate optimization algorithm but rather works as a wrapper. It generates randomized initial parameter sets and then invokes another minimization algorithm to optimize the starting parameter set. The following code block illustrates the use of the Spa minimizer. A complete example can be found here:

```
<fitting>
<spa max-iter="25" seed="120815">
<BFGS conv-threshold="1e-10" max-iter="50" />
</spa>
</fitting>
```

The `max-iter`

attribute specifies the number of times a new initial
parameter set is generated. The `seeds`

attribute allows one to set
the seed of the random number generator. Here, the L-BFGS minimizer
described above is used. Note that the maximum number
of iterations for the latter is set to a rather low value. This
procedure is recommended to avoid spending many iterations in the
inner loop on incrementally optimizing a parameter set rather than
sampling a larger number of different initial parameter sets. If a
parameter set has been found, whose objective function is smaller than for any of the previously found
sets, the new set is written to file and, depending on the value of
the `<verbosity>`

tag, standard output.

Warning

When using the Spa minimizer it is *strongly* advised to define
also bounds for properties such as the lattice
parameters. Otherwise the fit can run quickly away and yield
unreasonable results. Bounds on the properties are imposed by using
the `min`

and `max`

keywords of the respective element.

Warning

Note that optimizers that attempt a “global” minimum search
including the Spa minimizer as well as the global algorithms from
the NLopt library usually do *not* minimize
individual points very carefully. It is therefore usually strongly
advised to optimize the minima from these algorithms further using
conventional (local) minimizers with tight convergence criteria.

## Optimization via NLopt¶

**atomicrex** features an interface to the open-source library
NLopt for nonlinear optimization, which provides a
large number of algorithms for both
global optimization, local derivative-free optimization, and local
gradient-based methods. These algorithms can be invoked as illustrated
by the following example. A complete example can be found here:

```
<fitting>
<nlopt algorithm="LN_NELDERMEAD" stopval="1e-10"/>
</fitting>
```

The algorithm is selected using the mandatory `algorithm`

attribute,
which can assume any of the following string values:

Local gradient based algorithms:

`LD_LBFGS`

: limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm`LD_MMA`

: Method of moving asymptotes (MMA) and conservative convex separable approximation (CCSA)

Local, derivative free algorithms:

`LN_NELDERMEAD`

: Nelder-Mead Simplex`LN_SBPLX`

: Sbplx (based on Subplex)

Global algorithms

`GN_CRS2_LM`

: Controlled random search (CRS) with local mutation`GN_ESCH`

: ESCH (evolutionary algorithm)`GN_DIRECT`

: DIRECT and DIRECT-L`GN_ISRES`

: ISRES (Improved Stochastic Ranking Evolution Strategy)

Warning

Please note some of the algorithms in NLopt, in particular most of
the global-optimization algorithms, do not support unconstrained
optimization and will return an error if the user has not supplied
finite lower and upper bounds.
**Bounds for potential parameters** are set using the `min`

and
`max`

attributes of the elements of the `<fit-dof>`

block as
described here.

It is furthermore strongly advised to define also **bounds for
properties** such as the lattice parameters. Otherwise the fit can
run quickly away and yield unreasonable results. Bounds on the
properties are imposed by using the `min`

and `max`

keywords of
the respective element.

The `<nlopt>`

element can be further modified using several optional
attributes:

`seed`

(int): Set seed for pseudo-random number generator; more information can be found here.

`stopval`

(float): Stop when the objective function \(\leq\)`stopval`

.`maxeval`

(int): Stop when the number of function evaluations exceeds`maxeval`

.`maxtime`

(float): Stop when the optimization time (in seconds) exceeds`maxtime`

.`ftol_rel`

(float): Stop when an optimization step (or an estimate of the optimum) changes the objective function value by less than`ftol_rel`

multiplied by the absolute value of the function value.`ftol_abs`

(float): Stop when an optimization step (or an estimate of the optimum) changes the function value by less than`ftol_abs`

.`xtol_rel`

(float): Stop when an optimization step (or an estimate of the optimum) changes every parameter by less than`xtol_rel`

multiplied by the absolute value of the parameter.