On optimization techniques for solving nonlinear inverse problems

This paper considers optimization techniques for the solution of nonlinear inverse problems where the forward problems, like those encountered in elec tromagnetics, are modelled by differential equations. Such problems are oft en solved by utilizing a Gauss-Newton method in which the forward model con straints are implicitly incorporated. Variants of Newton's method which use second-derivative information are rarely employed because their perceived disadvantage in computational cost per step offsets their potential benefit s of faster convergence. In this paper we show that, by formulating the inv ersion as a constrained or unconstrained optimization problem, and by emplo ying sparse matrix techniques, we can carry out variants of sequential quad ratic programming and the full Newton iteration with only a modest addition al cost. By working with the differential equation explicitly we are able t o relate the constrained and the unconstrained formulations and discuss the advantages of each. To make the comparisons meaningful we adopt the same g lobal optimization strategy for all inversions. As an illustration, we focu s upon a 1D electromagnetic (EM) example simulating a magnetotelluric surve y. This problem is sufficiently rich that it illuminates most of the comput ational complexities that are prevalent in multi-source inverse problems an d we therefore describe its solution process in detail. The numerical resul ts illustrate that variants of Newton's method which utilize second-derivat ive information can produce a solution in fewer iterations and, in some cas es where the data contain significant noise, requiring fewer floating point operations than Gauss-Newton techniques. Although further research is requ ired, we believe that the variants proposed here will have a significant im pact on developing practical solutions: to large-scale 3D EM inverse proble ms.