Attention On What Is Important: Improving Neural Encoders for Routing Problems

Licentiate thesis, 2024

Virtually all deep learning-based frameworks trying to solve routing problems have a neural encoder architecture. In this work, we explore the sparsification of graphs representing instances of routing problems. By this sparsification, we allow the neural encoder architectures of learning-based frameworks to focus on the parts of the routing problem that are most promising to be part of the problem solution. As a result, the encoders can produce better encodings that represent the problems in the neural framework. Since these good prob- lem representations are fundamental for the overall learning pipeline, good encodings improve the overall performance.

In particular, in this thesis, we focus on graph neural network (GNN) and transformer encoders applied to instances of the traveling salesman problem (TSP). We propose two different procedures to determine the most promising edges of a TSP, i.e., the edges that are likely to be part of the optimal TSP tour. The first method is the simple k-nearest neighbor heuristic, where each node in the TSP instance is only connected to the k closest other nodes af- ter sparsification. The second method is based on minimum spanning trees (MSTs) and offers the advantage of guaranteeing connected sparse graphs.

Furthermore, we propose ensemble methods of different sparsification levels. This means that each TSP instance is represented several times, each time as a graph with either more or less edges of the original TSP graph being kept. By combining very sparse graphs with only the most promising edges and dense graphs with a high amount of edges, we allow the encoder architecture to focus on the most important parts of the problem only while minimizing the risk of completely deleting optimal TSP tour edges in the sparsification process. The encodings produced on the TSP graphs of different sparsification levels are merged afterwards, creating encodings that can be incorporated easily into existing learning-based routing frameworks.