Constraints in the Levenberg-Marquardt least-squares optimization

The standard Levenberg-Marquardt optimizer does not support box constraints. This is, for example the case for good MINPACK lmdif/lmder implementations that many optimization libraries use underneath. But in practice, it is often useful to limit the range of the variables, often because the objective might not be defined everywhere.

The R minpack.lm CRAN package allows the user to specify box constraints. How do they do it?

They choose a very straightforward projection approach to enforce the constraints: if the guess is outside the box, they place it at the closest boundary (see the source code). I believe this corresponds to the algorithm described in section 3 of the paper Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties.

Now this is very aggressive, especially if the gradient is computed numerically (lmdif): typically what happens is that the solution is then stuck at this boundary. For example, if the coordinate is \(x_k=-1\), and our box is the positive subspace, the projection will set \(x_k=0\) and then this coordinate is not going to be able to move as the gradient becomes 0. It behaves a bit better if the gradient is analytical at the boundary, or if we are careful to use forward/backward finite difference estimate depending for the lower/upper bound.

It is a much better idea to rely on a smooth and invertible transformation from the real line to a target interval, for example a sine transformation for a closed interval, or a square/square-root transformation for a semi-infinite interval as is suggested on this page. Of course, the transformation should also be included in the analytical gradient.