GIANT Networks: Very Deep Fully Connected Neural Networks Applied to Microwave Problems

Artikel i vetenskaplig tidskrift, 2026

We present the Gradient-Informed Attentive Normalisation Training (GIANT) framework with the objective to create very deep fully connected neural networks, which we use as surrogate models in the context of microwave problems. As the central component of the GIANT framework, we introduce a novel dynamic reparameterisation procedure for the weight-bias parameter space by means of a low-variance preserving normalisation layer for each fully connected layer and we refer to this construction as Attentive Normalisation (AttNorm). As part of AttNorm, we also introduce a new and tailored updating scheme that improves the convergence during training. To efficiently train very deep fully connected neural networks, we exploit Sobolev training with gradient information, which is computed at a very low computational cost by means of continuum sensitivity analysis. We test our novel approach on two microwave applications: (i) a six-port microwave cavity with a random medium and (ii) an H-plane waveguide filter optimised under geometrical uncertainty. For these examples, we demonstrate successful training of neural networks with up to 30 layers, which are sufficiently accurate and expressive to serve as excellent surrogate models.