Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Cascade Computation of Second Order Information for Efficient Optimization of Multi-Argument Objective Functionals

Carlsson, Marcus LU ; Bacca, Jorge ; Wendt, Herwig and Nikitin, Viktor (2025) 2025 IEEE 10th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2025 p.341-345
Abstract

We present a general technique for efficient computation of second order information for optimization functionals involving many variables of different nature, frequently appearing in modern applications. This technique relies on the bilinear Hessian, and while the main tools for this task have been published elsewhere, we here focus on the problem of how to organize the computations in the multi-argument situation, relying on so called cascade optimization and the chain rule for the bilinear Hessian. For concreteness, we demonstrate our approach by focusing on three concrete problems; phase retrieval, multidistance nano-holotomography and training of neural networks. In the experimental section we present results where the proposed... (More)

We present a general technique for efficient computation of second order information for optimization functionals involving many variables of different nature, frequently appearing in modern applications. This technique relies on the bilinear Hessian, and while the main tools for this task have been published elsewhere, we here focus on the problem of how to organize the computations in the multi-argument situation, relying on so called cascade optimization and the chain rule for the bilinear Hessian. For concreteness, we demonstrate our approach by focusing on three concrete problems; phase retrieval, multidistance nano-holotomography and training of neural networks. In the experimental section we present results where the proposed technique significantly speeds up first order methods, and in particular we obtain faster training results than the Adamalgorithm when training the so called SIREN neural network.11Supported by Swedish Research Council (grant no.

(Less)
Please use this url to cite or link to this publication:
author
; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
bilinear Hessian, Cascade optimization, fast optimization, second order methods, Wirtinger calculus
host publication
2025 IEEE 10th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2025 - Proceedings
pages
5 pages
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
2025 IEEE 10th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2025
conference location
Punta Cana, Dominican Republic
conference dates
2025-12-14 - 2025-12-17
external identifiers
  • scopus:105035995004
ISBN
9798331526696
DOI
10.1109/CAMSAP66162.2025.11423868
language
English
LU publication?
yes
id
84614bbd-bbc2-4b24-857f-6ba4064342e9
date added to LUP
2026-06-26 13:09:48
date last changed
2026-06-26 13:10:05
@inproceedings{84614bbd-bbc2-4b24-857f-6ba4064342e9,
  abstract     = {{<p>We present a general technique for efficient computation of second order information for optimization functionals involving many variables of different nature, frequently appearing in modern applications. This technique relies on the bilinear Hessian, and while the main tools for this task have been published elsewhere, we here focus on the problem of how to organize the computations in the multi-argument situation, relying on so called cascade optimization and the chain rule for the bilinear Hessian. For concreteness, we demonstrate our approach by focusing on three concrete problems; phase retrieval, multidistance nano-holotomography and training of neural networks. In the experimental section we present results where the proposed technique significantly speeds up first order methods, and in particular we obtain faster training results than the Adamalgorithm when training the so called SIREN neural network.<sup>1</sup><sup>1</sup>Supported by Swedish Research Council (grant no.</p>}},
  author       = {{Carlsson, Marcus and Bacca, Jorge and Wendt, Herwig and Nikitin, Viktor}},
  booktitle    = {{2025 IEEE 10th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2025 - Proceedings}},
  isbn         = {{9798331526696}},
  keywords     = {{bilinear Hessian; Cascade optimization; fast optimization; second order methods; Wirtinger calculus}},
  language     = {{eng}},
  pages        = {{341--345}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  title        = {{Cascade Computation of Second Order Information for Efficient Optimization of Multi-Argument Objective Functionals}},
  url          = {{http://dx.doi.org/10.1109/CAMSAP66162.2025.11423868}},
  doi          = {{10.1109/CAMSAP66162.2025.11423868}},
  year         = {{2025}},
}