ACTIVE-SET STRATEGY BASED ON A GENERAL MODIFIED NEWTON-RAPHSON ALGORITHM FOR VARIABLE SELECTION IN HIGHLY ILL-POSED INVERSE PROBLEMS

Abstract

We propose a novel algorithm to perform efficient modified-Newton-Raphson optimization over the active set of selected features
(AMNR), and show that it allows to estimate multiple penalized least-squares (MPLS) models. MPLS models are used to find flexible
and adaptive least-squares solutions to highly ill-posed linear inverse problems, mainly requiring them to be simultaneously sparse
and smooth. This is relevant for applications where there is no ground truth, e.g., estimating electrophysiological sources. In this work, we step on a modified Newton-Raphson algorithm that can be interpreted as a generalization of the Minorization-Maximization algorithm to include combinations of several constraints, and derive the AMNR algorithm following an approach similar to that used for Least Angle Regression. This algorithm allows us to implement many different MPLS models, including novel models such as the Smooth Nonnegative Garrote and the Nonnegative Smooth LASSO. The performance of the algorithm is evaluated using simulated data from a simple ill-posed linear regression and from a realistic Electroencephalographic setup. Different models containing one or two penalty functions, and including sign constraints were evaluated in both cases. The algorithm allowed the recovering of solutions in a fast way and with adequate quality in simulated scenarios for different n/p ratios.